Jonathon Levy, currently Senior Learning Strategist at the Monitor Group, tells the story from his days as Vice President at Harvard Business School Publishing. The story is funny and sad at the same time, but it’s very instructive on several fronts.
Levy’s client decided that he would award end-of-year Christmas bonuses based on how successful his employees were in completing the Harvard online courses. Levy advised against it, but the client did it anyway.
The results were predictable, but they might never have been noticed if Jonathon’s Harvard team had not integrated into all their courses a tracking system to provide themselves with feedback about how learners used the courses. The tracking system showed that learners didn’t read a thing; they just scanned the course and clicked where they were required to click. They just wanted to get credit so they could maximize their bonuses.
Although very little learning took place, everyone was happy.
- Learners were happy because they got their bonuses.
- The client (training manager) was happy because he could show a remarkable completion rate.
- Line managers were happy, because the learners wasted very little time in training.
- Senior management was happy because they could demonstrate a higher rate of utilization of online learning.
What can we learn from this?
- Be careful what you reward, because you might get the specific behavior you reinforced (course-completion in this case).
- Completion rates are a poor measure of training effectiveness. Same as butts in seats.
- We need authentic measures of training effectiveness to prevent such silliness.
- Instructional-design activities benefit when good tracking and feedback mechanisms are built into our designs.
