How Can You Measure The Learning Effectiveness Of Online Courses And Create A Positive ROI?

While it is obvious that Learnability or learning effectiveness of online courses has an impact on ROI, its measurement is rather a tall order. This article outlines how you can use our framework to predict, measure, and validate the learnability or learning effectiveness of your online courses.

What Is Learnability Or Learning Effectiveness Of Online Courses?

Learnability or learning effectiveness of online courses can be defined as the ease and speed with which the learners can acquire the required information/knowledge/skill. There is an associated aspect that you need to be mindful of, that is, the scope of learnability. Once this measured, we can assess the progress from knowledge acquisition to application reflected as performance gain.

The scope of learnability evaluation can be looked at in two ways:

1. Initial Learnability: Initial Learnability applies to the performance over a single, short term usage period.
2. Extended Learnability: Extended Learnability applies to the performance change over time.

What Is The Significance Of Learnability Or Learning Effectiveness Of Online Courses?

The significance of learnability is evident. Only well crafted courses with the right learnability can help learners perform better and help organizations meet their goals.

Studies have revealed that learners lose up to 40% of their time due to “frustrating experiences”with courses. The common causes being missing, hard to find, unusable, and irrelevant features of the course.

Similarly, a study carried out by Federico Borges found that learners get frustrated due to a variety of reasons including the learning strategy used, lack of prompt feedback, the company culture, the learning material that was designed, and so on.

Does Learnability Impact The Computation Of ROI On Training? Is This Impact Direct Or Indirect?

Most of us are familiar with Kirkpatrick’s model of evaluation as shown here:

• Level 1 – Learner Reaction – Was the course relevant, useful, and worth my time?
• Level 2 – Learning – Did the course lead to increase in skill/knowledge? Did the course meet the required cognition level?
• Level 3 – Application of learning – Was there a performance improvement or change in behavior based on the learning?
• Level 4 – Business impact – Was there a tangible and demonstrable value after the training?

This model can be extended to measurement of ROI on training by mapping monetary value to the final gain (business impact).

Impact of Learnability on ROI on training: Learnability or learning effectiveness impacts each level of Kirkpatrick’s model, eventually helping you maximize ROI on training.

Specifically:

• Learnability impacts level 1 and 2 directly and enables application of learning in level 3. Once these three are in place, the business impact is likely to be met. Subsequently, the ROI on training can be calculated (mapping the business impact to a monetary gain).

How does this really work to impact the ROI?

1. A better and meaningful user experience increases the stickiness of learning.
2. It also increases the learners’ motivation to take the course leading to higher completion rates.
3. Once these are in line, you can expect to see an increased application of the learning and this in turn will lead to the desired impact on the learners and business.
4. By evaluating your online courses for learnability, you can predict its impact as well as identify measures to increase it. This will go a long way in achieving a positive ROI on your training spend.

NOTE: Validation of the learning experience is important when it comes to determining ROI on training. As suggested by Keri Bennington of UNC Kenan-Flager Business School in a report, “ROI measures should be related to performance after the Learning and Development experience and, according to some, tied to a dollar figure. For example, time saved or increased output (or both) as a result of improved performance following participation in a development program can then be compared to a dollar figure.”

Can Learnability Be Measured?

Yes, learnability or learning effectiveness of online courses can be measured but there are several challenges associated with this exercise:

1. There is a maze of parameters that influence it.
2. Adding to the volume of parameters that can influence learnability is the second challenge that not all parameters are measurable.
3. The significance of each parameter changes with each training type.
4. Finally, the feedback can be objective (easy to implement) as well as subjective (often difficult to fix).

How Do We At EI Design Measure Learnability?

In spite of the fact that measurement of learnability is a tall order, at EI Design, we offer a unique framework that you can use to predict, measure, and validate the Learnability of online courses (for new or legacy courses).

Our Approach: We began our exercise by looking at various building blocks of a course and how learners view them or interact with them. Using these cues, we created 6 metrics which map to each of these to help us evaluate the learnability or learning effectiveness as shown here:

• Metrics 1: Interface Design.
• Metrics 2: Course Information and Instructions (navigation).
• Metrics 3: Content Structuring (to meet the required level of cognition).
• Metrics 4: Task Performance (to interact and learn).
• Metrics 5: Usability (overall experience).
• Metrics 6: Feedback on Design Elements.

Measurement: Our framework also allows you to dynamically assign the criticality based on the nature of the course, content, or learners. Then the aggregated Learnability Index is generated.

Take a look at an example of the Learnability Index of an existing course and how we upped it by enhancing it through our Learnability guidelines.

Validation: This can be validated by the learners and the feedback can be used to enhance the learning design further, if required.

Take a look at some reports that show how the course fared with learners (validation of the predictive learnability).

How Can You Use Our Framework To Maximize ROI On Training?

You can use our framework to predict the learnability of the courses you are developing. You can also use it to evaluate the Learnability of your legacy courses and then uplift it. Through our learnability framework, you will be able to:

1. Get access to a predictive methodology to build learnability in your new courses.
2. Use it to evaluate the learnability of your existing courses: You can diagnose issues that reduce the learning effectiveness and hamper performance and step up the Learnability Index to the required levels.
3. Validate your assumptions with learners: You can check if your courses pass the litmus test by validating the impact of the courses from the folks who matter – the learners!
4. Gain crucial insights for future learning interventions that can enhance the effectiveness of the training curriculum and performance for the learners.
5. Improve learning retention and meet the prescribed cognition level.

With these, you will be able to have greater clarity on demonstrable gains for learners as well as the business and maximize your ROI on training.

How Can Our Learnability Framework Help Learning And Development Teams?

Learning and Development teams can use our learnability framework to:

1. Predict, measure, and continuously enhance the learning effectiveness of trainings.
2. Have a robust framework in place to constantly improve learning and performance.
3. Have a data driven approach to address user experience related issues in learning (actionable learner feedback).
4. Demonstrate a positive impact on ROI on training.

You can take a look at this video of our Webinar on How To Predict, Measure, And Validate The Learnability Of Online Courses for more in-depth insights:

I hope this article provides cues on how you can use our framework to measure and enhance the learnability or learning effectiveness of your online courses, how you can validate the predictive Learnability with your learners, and see if you are on target or what further measures will help you hit the bull’s-eye. If you have any queries on this, do contact me at apandey@eidesign.net.

Source: https://www.eidesign.net/can-measure-learning-effectiveness-online-courses-create-positive-roi/