Since 2002, I have offered the Learning Research Quiz on the Work-Learning Research website. Over 1200 people have taken that quiz. 991 people answered all 15 questions. There are 15 questions on the quiz. A blind monkey randomly guessing the answers would score about 24% (there are three to seven answer choices per question).
The questions are scenario-based questions presenting relatively realistic learning-design situations. The answer choices are plausible---there are no throwaway answer choices. The questions are thus not easy to answer. They are specifically designed to measure people's understanding of fundamental learning principles like repetition, spacing, feedback, retrieval practice, instructional objectives, etc.
People who answered the questions are self-selected from visitors to the Work-Learning Research website so we can't really be sure who this sample represents. In general, it is my experience that most people who come to the website are experienced learning professionals who are passionate about learning, training, teaching, and/or instructional design.
Here are the results for the 991 people who answered all 15 questions:
If 24% is a completely random score, what should we expect from those who take the quiz? Would you expect them to do twice as well (around 50% correct?) or do better, say 60%, 70%, 80%, 90%, or 100%?
On average our 991 participants scored 32% correct, barely better than chance responding!!
Scoring 1% to 25% correct were 279 respondents.
Scoring 25% to 50% correct were 610 respondents.
Scoring 50% to 75% correct were 101 respondents.
Scoring 75% to 100% correct was 1 respondent, who scored 80% correct.
Association affilitation had little impact on average correct:
ISPI members scored 36% correct.
ASTD members scored 33% correct.
Other trade association members 28% correct.
Research association members 30% correct.
Non-affilitated respondents scored 31% correct.
Age of respondents had little impact on average correct:
Under 21 years of age scored 26%
21-25 scored 28%
26-35 scored 32%
36-45 scored 33%
46-55 scored 30%
56-65 scored 35%
66-75 scored 33%
Education degree had little impact on average correct:
"Ph.D. in psychology, learning, etc." scored 35% correct.
"Ph.D. in other discipline" scored 34% correct.
"Masters in psychology, learning, etc." scored 33% correct.
"Masters in other discipline" scored 32% correct.
"Bachelors" scored 31% correct.
"Other" scored 29% correct.
Job Title had little impact on average correct:
"Trainer" scored 32% correct.
"Instructional Designer" scored 33% correct.
"Performance Consultant" scored 33% correct.
"Human Performance Technologist" 38% correct.
"E-Learning Specialist" scored 29% correct.
"Learning Technology Developer" scored 33% correct.
"Teacher" scored 31% correct.
"Professor (with little research activity)" scored 33% correct.
"Learning Researcher (professors too)" scored 31% correct.
"Manager of Instructional Development" scored 34% correct.
"Manager of Training" scored 34% correct.
"Student" scored 28% correct.
"Other" scored 29% correct.
Date of Taking the Quiz had little impact on average correct:
2002 scored 30%
2003 scored 33%
2004 scored 31%
2005 scored 33%
2006 scored 30%
2007 scored 33%
Conclusion
Because the sample of respondents is difficult to define, any conclusions must remain speculative. Still, the results suggest a massive competence gap. The 32% average score---and the stubborn lack of improvement regardless of experience, education, and age---suggests that most people in the learning-and-performance field are unprepared for roles as designers of learning, at least as far as their ability to apply knowledge of learning research.
Links
- The Quiz
- The Quiz Questions with Feedback