Learning Measurement 5 -- E-Learning's Role in Measurement

What can e-learning add to measurement?

What can e-learning add to measurement? Does e-learning have unique capabilities that enable it to improve learning measurement? I think it does. Here’s a short list:

1. E-learning can capture more data more easily than classroom training.
2. E-learning can capture data during the learning program—not just at the end of the learning event—in a manner that the learners feel is a seamless and natural part of the event.
3. E-learning can track incoming proficiency through the use of pretests to determine whether the learning program actually meets a need, or determine who it meets a need for.
4. E-learning can collect data in a manner that can give learners comparison data while they complete an assessment.
5. E-learning can collect data on learner behaviors during the learning (for example, the click journey, time per screen, etc.)
6. E-learning can track pretest to posttest changes.
7. E-learning can randomly assign learners to program versions, making methodological comparisons possible. For example, a program version that uses immediate feedback can be compared to a program version using delayed feedback to determine which method is more effective.
8. E-learning can capture on-the-job performance data, including learners’ self-ratings, manager ratings, direct-report ratings, etc. This capability puts the focus on on-the-job performance, where benefits can accrue from management oversight and coaching, self-initiated development, and peer learning.
9. E-learning, because it can access and track learners at more than one point in time, can measure how well the learning intervention has performed in creating long-term remembering.
10. E-learning can capture data even when learners don’t know the learning program is being assessed. For example, the learning program can capture data when the learners think they are simply getting practice on the learning material.
11. E-learning can track learners as they move from the training event to the workplace. For example, e-learning programs can track learners’ goals to implement what they have learned to see how successful they have been in transferring the learning to the job.

With this power, comes responsibility, and a damn fun challenge. You can read my call-to-action later in this series and in the e-learning Guild Research Report as well.

The Measurement Series Continues Tomorrow...

Comments

[Carsten]

>E-learning can capture more data more easily than classroom training.

That is a claim worth taking a deep look at. It depends on what kind of data you are talking about. True, you can capture quantitative data about how long a page was opened in a browser (whether the student read it or went to grab a coffee, you don't know), what resource were looked at, performance in exercises etc., but you are missing quite important data every teacher in a classroom is able to give you.
The best example is motivation. I've been doing some work in Artificial Intelligence and education. It is extremely hard to make any sensible estimation of the motivational state of the learner based on the learner's interactions with the e-learning system alone. It is already hard to accurately asses the learner's knowledge or competencies, but with affective values it is nearly impossible. But still, a good teacher knows it. One reason certainly is that the teacher can use more modalities than just click-behavior (such as reading facial expressions).
So I would say that the general claim that "e-learning can capture more data more easily than classroom training" is a bit too optimistic. But then, I did not read the complete report.

[Will Thalheimer]

Carsten,

Of course, you are right, the kind of real-time data in the classroom is very important and great instructors can do great things with that information, even when they are processing student responses unconsciously. Audience response systems in the classroom have great promise as well in capturing additional data. Perhaps I should have been more precise in my statement. On the other hand, there is information that is not typically captured in the classroom that e-learning can, in practice, capture more easily. If you’re working in artificial intelligence, you probably are concocting additional good ways to capture information. What I’m focused on is the common experiences of today in the workplace classroom. Artificial intelligence is a long way off. In fact, the starting line for gathering data is pretty archaic, so my enthusiasm for e-learning’s potential to capture data comes from that real-world backdrop.

Thanks for your excellent comment!!

[GlennaRAY20]

Different people in the world take the personal loans from various creditors, because it is comfortable.