Will at Work Learning

Myths the Business Side Has About Learning: Result of Data Gathering

I've been gathering a list of Myths that the Business Side Has about Learning.

I reached out to my clients, to groups in LinkedIn, to my Brown Bag Learning participants. I also reviewed some books, including Stolovitch & Keeps "Telling Ain't Training"; Doyle's "The Manager's Pocket Guide to Training", Bell's "Managers as Mentors". I also brought to memory my own recollections from over a decade of work and research on learning.

I compiled a list of about 140 myths and then used a card-sort methodology to separate them into categories.

Here are the results:

Everybody Hold Myths

First, it became clear that the Business Side isn't the only group that holds myths. Learners and we as Learning Professionals have our own sets of myths. We can't demonize the Business Side. We have to go out of our way to understand and work with the business side to craft workable effective solutions for our organizations and all the people impacted.

Let me say that sometimes I kind of regret that a distinction has to be made between us as learning professionals and them as the business side. There's something wrong with that distinction (we are IN the business aren't we), yet the dichotomy makes some sense since we support others who do the actual work of the business.

The Most Popular Myths
(that the Business Side Has about Learning, according to Learning Professionals)

These are in order from my card-sorting categorization effort. The most-often cited are listed first.

1. Bad Learning Designs are Thought to be Good Learning Designs (big list below).
2. Training Alone Produces Improvements in On-the-job Performance.
3. Information Presentation is Sufficient as a Training Design.
4. Training & Instructional Design Require No Special Skills or Competencies.
5. Learners Know How to Learn.
6. Managers Think Learning & Development is a Low-Priority Part of their Role.

Other High-Importance Categories

• On-the-Job Learning is Forgotten or NOT Utilized or NOT Supported.
• It’s a Training Issue (a conclusion drawn before considering alternative causes).
• Formal Training has Little Impact.
• Experienced Workers Don’t Need Training.
• Development of Learning Interventions is Easy and Can be Shortened or Short-Changed.
• Measurement of Learning. Miscellaneous Issues thereof.
• Technology is Key to Learning Success.

Will's and Other Additions

• Learning Designs Don’t Need to specifically Minimize Forgetting (Enable Remembering).
• Content Doesn’t Need Validation.
• Particular Behaviors are Easy to Learn (e.g., It's easy to do customer service).
• Learning is Always Beneficial. It is Never Disruptive or Distracting. It Never Misinforms.
• Opportunity Costs of Learning Can be Ignored.
• We Don’t Have to Measure Learning.
• We Have to Measure ROI. 
• We can Avoid Measuring Retrieval.

Short List of the Bad Learning Designs that the Business Side (and others I might add) Think Are Good Learning Designs

• It is good to have new employee take all their elearning courses right away before starting work.
• Employees ONLY learn by doing.
• Reading is always bad, boring, and ineffective.
• Training can be just as effective if we make it as short as possible.
• Training doesn’t need pre-work or post-work.
• A large library of courses or books is the way to go.
• Employees need to know everything.
• We should and CAN cater to learning styles.
• Latest management craze (provide book to everyone).
• Six-hour online courses are fine.
• Some learning media are inherently better than other learning media.
• Best value in training is a 10 to 12 hour day.
• More information = More learning.
• People remember 10% of what they read, 20% of what they see…
• Most communication is by body language (57% is body language, only a small fraction communicated is in the actual learning messages).
• We need more exciting visual decorations to grab attention.
• Immediate feedback is always best.
• Etc.

The MOST IMPORTANT QUESTION:  What do we do?

The first thing to do is to demonize everyone and give ourselves kudos for our wisdom,  conscientiousness, and whimsical charm.

No.

The first thing to do is to take responsibility. Just as a speaker must take responsibility to ensure that his or her listeners are understanding the intended message (even though much is out of the speaker's control), we must take responsibility for ensuring that our business stakeholders (1) understand learning at a deep level, (2) understand how they can ensure that training is applied successfully on the job, and (3) understand how they can create a work-learning environment that supports employees in learning on their own, from each other, and from their managers.

I got started on this myth gathering as a way to help me build a course for a client (a very large company) to help them improve work-learning at their company top to bottom, including formal and on-the-job learning.

Will this be easy? No. Someone today at my Brown Bag Webinosh asked, "Haven't we been trying to bust these myths for decades?" Great question, and it goes to the difficulty of the task. Many of us have been trying for decades to make changes, but I think also that many of us are just doing our little part as order takers. We build learning interventions when asked. So, bottom line is that I think we could try harder. That's the first thing.

We need to try smarter as well. I've learned over the years, when I've tried to communicate complicated research-based information, that it is critical to find just the right metaphor, just the right visual model, just the right explanation that is both simple and robust to get the job done.

Maybe human learning and performance is just too complicated to enable this, but I think it's worth a try to build some better metaphors, models, and explanations.

We also need to continue to offer research, real-world examples, and valid evaluation results as evidence. We also need to understand our business partners and their mental models and build our case within their frameworks, so they get what we're saying. We need to build into our training-development process our stakeholder-education efforts and our stakeholder-understanding efforts.

Reaching Out

If your company has created a learning intervention to help your business managers better understand learning and their role in it, I'd love to learn more. Contact me.

If your company would like to utilize or co-develop such a learning intervention, feel free to contact me now.

Complete Lists of Myths That the Business Side Has About Learning
(according to Learning Professionals)
(Note that these are offered "as is" with typos, etc.)

1. "learning" is the accountability of the Training or Development Department or staff, rather than a leadership responsibility
2. 1 and done - one class and they'll know everything
3. 1 or 2 day management training seminar can turn an ineffective manager in to a high performing one.
4. A best practice is to "get all the PPLs out of the way"
5. A business gives a metrics pass to the learning group because “that stuff can’t be measured” and is then puzzled.
6. A learning buffet (large library of courses) is the way to go
7. A learning group is not integrated with those responsible for performance support.
8. a test is need to prove the learners ""know"" it
9. Any set of questions will do. There is no need to check to see which ones are good measures and which are not.
10. Anyone can train someone else therefore anyone can create a training course.
11. Asking a performance leader (someone good at their job) to deliver on the job training should not diminish that performer's output
12. Bad Learning Designs are Thought to be Good Learning Designs (big list below).
13. best value in training is a day 10 or 12 hours long.
14. Build it and it will run: it's vital to get IT involved
15. Build it before or without any needs assessment.
16. Butts in seats is all that matters
17. Content Doesn’t Need Validation.
18. Context doesn't matter; just teach everyone the right steps to a task
19. Courses without organizational support are okay.
20. customer service is easy to teach.
21. Delivering or presenting instructional content (via ILT or online courseware) is sufficient to elicit improved performance in the workplace.
22. Different media create different learning results.
23. Don't bother with objectives; just present the content.
24. E-learning development is fast
25. e-Learning isn't learning.
26. E-Learning takes 1/3 the time of classroom instruction, so it should only cost 1/3 as much to create
27. electronic learning is just as effective as in person learning
28. Employees can't manage their own learning successfully
29. employees need to know everything.
30. Employees only learn by doing.
31. everyone learns the same way (often the way that the manager best learns
32. Everyone learns the same way, so only one style of learning is required..
33. Experienced Workers Don’t Need Training.
34. Facilitators can develop great courses
35. Formal (scheduled, structured, SME-created) learning interventions are the best means of conveying knowledge and skills to our workforce
36. Formal Training has Little Impact.
37. Getting certified by taking a training class alone
38. Hands-on training is okay if it just enables situation-actions
39. Help mgmt solve problem, not just do workshop
40. I already know it so I don't need to go to training.
41. I attended a training class so I don't need to practice it.
42. I attended a training class so I must know how to do it.
43. I don't have to take part.
44. I don't need to go through training, I just need my people to
45. I know everyone had different learning styles, but I learn hands on.
46. I left them a to-do list--they should follow it. No follow up required.
47. I need new folks to start immediately. No time for training.
48. I should see immediate results on my bottom line the first day after training
49. idea sharing is a good form of learning
50. If ""they"" can do it, ""they"" can train it.
51. If I tell all of my people what to do in a meeting, they'll do it and won't need reminders or additional training
52. If someone doesn't know how to do something I will just do it myself because it's faster than teaching
53. If someone is trained on something they will be able to easily figure out how to apply it to their current job without any guidance
54. 'if we build it they will come
55. I'll figure it out on my own so therefore I don't need to go to training.
56. I'm a Director/VP so I don't need to go.
57. I'm a visual learner - I can only understand it if I see it.
58. in hard economic times it makes sense to cut training.
59. Information makes for learning
60. Information Presentation is Sufficient as a Training Design.
61. Interactive eLearning is only for Gen X or younger. Older folks won't get it.
62. It has to be interactive
63. IT training still needs vaildation if the training is presented from a task point of view. Must ensure that the steps taught are the steps needed to complete the task.
64. It’s a Training Issue.
65. Its a training issue
66. It's better if I just have someone show them how to do it.
67. It's easy for people to change if you train them right
68. It's okay for the training function to be order takers.
69. I've been promoted so I don't have to go to training.
70. Just send me the handouts/training materials and I'll figure it out.
71. Lack of cultural sensitivity for global audiences
72. Lack of performance results mostly from lack of skills or knowledge.
73. latest management book or craze (providing book to everyone)
74. Learners have misconception that they don't have responsibility to go beyond listening.
75. Learners Know How to Learn.
76. Learners know what they need
77. Learning Designs Don’t Need to specifically Minimize Forgetting (Enable Remembering).
78. Learning Development is Easy and Can be Shortened or Short-Changed.
79. Learning does not happen outside the classroom
80. learning is a luxury.  We hired smart people.  Just work.
81. Learning is Always Beneficial. It is Never Disruptive or Distracting. It Never Misinforms.
82. Learning P's. don't understand that learning happens on the job.
83. Learning should not take a lot of time away from work.  And people should be able to do self-study for almost everything
84. Learning/Training is the responsibility of other departments -- NOT the responsibility of the managers.
85. Let's give them a book or seminar on the topic and they'll be all better.
86. Live virtual programs (LVC) are most effective when they are recorded without an audience and made available for playback
87. Managers think it's more valuable to create multiple SMEs as opposed to structured learning.
88. Managers Think Learning & Development is a Low-Priority Part of their Role.
89. Measurement of Learning Misc. Issues.
90. Money not available
91. More information provided, more learning.
92. more/better training will solve the problem
93. Most communication is by body language (55%) and tone of voice (37%) rather than choice of words (7%). [This is a bastardization of Mehrabian's studies.]
94. My reports went through e-learning. I don't need to do more.
95. My time is valuable, I don't have time to take a training class.
96. need a class [to practice the stuff]; I already read it
97. Non-business people shouldn’t be involved in business decision making
98. Not just test scores!
99. On the job training happens without structure or reward or cost
100. one size fits all" approach
101. Only paper and pencil tests (i.e., multiple choice/true-false) are adequate for regulatory purposes to prove that the learner has mastered the content.
102. On-the-Job Learning is Forgotten or NOT Utilized or NOT Supported.
103. Opportunity Costs of Learning Can be Ignored.
104. Other High-Importance Categories
105. Particular Behaviors are Easy to Learn.
106. People can learn how to use software from a cheat sheet.
107. people can learn without being made self-aware about their own level of competence.
108. people know "how" to learn
109. People's overall learning doesn't matter, I just want them to do the task right
110. Performers should be assessed immediately after they have received the content from an instructor or from a courseware program.
111. PowerPoint with narration is good enough.
112. PPL completion rate is the way to measure quality of training.
113. PPLs and in-store activities are useless - we need to do hands-on training "instead".
114. presentation = training
115. Pyramid.
116. Quantify and communicate the value
117. Reading is always bad, boring, ineffective.
118. Regulatory and credentialling agencies create good tests.
119. Reports generated by a Learning Management System (LMS) are sufficient for monitoring the learning-to-assessment-to-performance continuum in our workplace.
120. Role plays are a waste of my time.
121. Seen IT buy ""learning"" w/o consultitng HR or Training dept
122. Six-hour online courses are just fine. i.e. no acknowledgement of information overload erasing what is learned.
123. SME's are the best trainers, and Trainers are always the SME's": Pulling an SME to deliver training just because they know the most isn't always the most effective approach.
124. SME's or developers make the best (or even competent) trainers.
125. So often what is perceived by mgmt as good training is attributed to the skills of a good presenter, not to training design.
126. successful performance during training usually results in improved otj performance.
127. Technology is Key to Learning Success.
128. Technology is key to learning success.
129. Tell me what I need to know and that's enough.
130. tell once, people know it.
131. tell them and they'll do it.
132. Telling is all we need to do."
133. Telling somebody once means they will remember it AND apply it to their work.
134. That "presentation" = "training".
135. That stakeholders will see imediate results (i.e. less than 1 year).
136. The best way to design is to use the "present and test method"
137. The biggest myth is that training alone will change people's behaviors.
138. The business believes that they can put an employee through training (be it live, web-based, etc.) and magically they will automatically put the skills into place
139. the course alone will solve the problem
140. the HR as a service provider model gives problems as your 'client' is your customer - and the customer is always right
141. The more slides, the better (death by PowerPoint)
142. The only way to learn is on-the-job-training; spending money on training programs is a waste
143. The skills of instructional designers and educators are pretty shallow and their key abilities are primarily related to instructional technology.
144. the training department can't help - they don't know our side of the business
145. the training is bad
146. there are learning styles
147. There are way too many PPLs... but we need a PPL on _____________.
148. There is no special knowledge needed to teach, design, or organize training
149. They can learn all they need to know in (pick arbitrary unit of time)
150. they don't realize the importance of reinforcement, repeat sessions, follow up
151. They have a college degree so they already know it.
152. They need a course in order to learn
153. Think in-person learning is more effective than online
154. too busy
155. Training & Instructional Design Require No Special Skills or Competencies.
156. Training Alone Produces Improvements in On-the-job Performance.
157. Training can be just as effective if we make it as short as possible (one day instead of three days)
158. training course will solve the problem.
159. Training determines job content and tasks, not the supervisor or work center.
160. training doesn't need follow-up
161. Training doesn't need pre-work or post-work
162. Training done to replace what managers should be doing
163. training fixes everything
164. Training is a cheap-quick-easy solution to a problem with my people
165. Training is common sense.
166. Training is the responsibility of the organization that sponsors it and the trainer who delivers it.
167. Training is time consuming and does not produce results
168. Training isn't very important in my responsibilities.
169. Training Just Happens
170. Training takes too long.
171. Training will automatically change behavior on the job
172. Training willing workers creates willing and able workers.
173. Training/teaching/telling = learning
174. Trainings are luxury and sometimes seen as a cookie for the staff at a time no one really need it. Let them have some legal fun
175. Try again, make sure you use the Access Code that is showing and follow by a # sign.
176. verbal responses (for example to customers) are easy.
177. We can Avoid Measuring Retrieval.
178. we can send them an email.
179. We can train people to do anything...
180. we can train people to instantly recall anything.
181. We can use common sense to guide training design.
182. We can't bring in outside help - our industry is too specialized and our needs are too unique.
183. We Don’t Have to Measure Learning.
184. We don't have to look at the performance situation.
185. We don't have to validate our content.
186. We don't need to learn! We just need to prove we meet the regulation.
187. We don't need to practice. Just tell them.
188. We have no time allocated for training in our budget so it doesn't happen (mgr may not realize that a lot of training happens on the job - not only as a formal process where the employee sits at the computer).
189. We Have to Measure ROI.
190. We only hire people who know what they are doing, they don't need to learn anything, and if they do, they'll pick it up on the job
191. we should automatically assume that an SME is ipso facto 'the best trainer'.
192. We should/can cater to learning styles.
193. When things are not going well it is clearly a lack of skills and knowledge - so TRAIN them
194. Why explain to all levels of employees how the organisation works, how the departments relate to one another, etc
195. Why would I train my employees if they are already doing it?
196. Why would I want to train my employees in specific sub-skills
197. You can develop a perfect course without SMEs.
198. you can fix anything with enough training.
199. You can’t teach people relationship skills (either they have them or they don’t)
200. You don't need objectives, any one can write training.
201. You either have the ability to learn or not.
202. You have competence or not, then you learn it on the job.
203. You need to use a technology to train people properly

Ideas Participants in My Brown-Bag Learning Event Offered on What We as Learning Professionals Ought to Do about the Myth Problem
(Note that these are offered "as is" with typos, etc.)

• Our responsibility - gently guide. Present the right solution when asked for the wrong one
• Give examples of whether X type of intervention has been successful
• Offer performance solutions: this is what we can do (beyond training)
• Bring out the research to dispel the myths
• Develop solid business acumen and work, plan, collaborate from there
• to educate clients
• We need to discuss the learning models and theories that we support when appropriate
• We should be advocates for learners
• We should questions their thinking, ask for evidence
• Provide real evidence of success.
• educate, communicate, inform
• We have to walk a fine line between sticking to the ""truths"" we know, yet dealing tactfully with management's myths.
• myth busters
• Don't be an order taker
• I have found that the RIGHT manager can make a difference. Sometimes change can come from within, by working to influence a middle manager.
• SHOW OUR VALUE
• Have proof/case studies of effects of good design and guidance.
• Don't wait to be invited to clarify them. Anticipate the reality and invite yourself to the table.
• Sell our clients on our skills and recommendations. It keeps coming down to convincing management about the value of what we have to offer.
• As learning professionals we need to promote the effort to focus on what is needed to improve performance.
• To have a clear focus and mission for learning in our organizations, and to be able to communicate clearly, with supporting information.
• Dealing with these myths is our reality and part of scoping a project and defining target and objectives realistically... all the time...

Another Example of the Bogus Percentages. Now by Qube Learning.

Below is another example of the misuse of the now-infamous bogus percentages by a speaker at a prominent international conference in the workplace learning field, this time in an online session in January 2009.

I have documented this problem starting in 2002. The following posts illustrate this problem.

• Original Post (Read this First):  People remember 10%, 20%...Oh Really?

• More Bogus Research Cited. This time at HRDQ
• Excellent Review of Dale's Cone and its Bastardizations
• More Bogus Percentages. This time on Wikipedia
• NTL continues its Delusions
  

A manager at Qube Learning joins the list of folks who have been fooled, and who foolishly and irresponsibly re-gift this faulty information. Point: If you can't verify the credibility of the so-called "research" you come across, don't share it.

And this follow-up slide:

It's a shame we have to keep revisiting this bogus information. I truly wish I didn't have to do this.

Of course, even if you and I wipe this bogus-information example off the face of the earth, there will be more misinformation we'll have to deal with. It's okay. It's the nature of living I think. The learning point here is that all of us in the learning-and-performance field must be vigilant. We must be skeptical of claims. We must build structures where we can test these bogus claims in the crucible of an evidence-based marketplace. It is only then that we will be able to build a fully-worthy profession.

Keep sending me your examples. Thanks to the helpful soul who sent me this example.

Interestingly, just today a major player in our field asked me permission to publish the original blog post (the one debunking the bogus-percentage myth) in their company newsletter (which goes out to over 100,000 people). They too had been using this misinformation in their work and now wanted to correct their mistake. I salute their action.

Excellent Review of Dale's Cone and its Bastardizations

Judith Gustafson just left an excellent comment on an earlier blog post. She let us know about a presentation at the Association of (AECT) Educational Communications and Technology conference in 2002.

Click here for the PPT presentation by Tony Betrus and Al Januszewski of the State University of New York at Potsdam that does a great job of describing what Edgar Dale meant to convey with his cone, AND shows numerous examples of how the cone has been used improperly with the numbers added.

Here is my original post on this.

More Bogus Percentages. This time on Wikipedia.

In what has become an eternal vigil against the myth that "People remember 10% of what they..." I just hit the jackpot with the help of Jay Banks who just sent me an email.

The Wikipedia entry for Edgar Dale had two incorrect references to the bogus numbers that I talk about so often (see my blog category Myths and Worse). I fixed it today, I hope for good.

Here's what it looked like:

And here's what it looked like in Wikipedia:

For those who are shocked that information on the internet might be wrong—or that Wikipedia might be wrong—see my previous entries about Wikipedia (1st Most-Recent).

Learning Styles Challenge 1-Year Update

It has been exactly one year since I offered $1,000 to anyone who could demonstrate that utilizing learning styles improved learning outcomes. Click here for the original challenge.

So far, no one has even come close.

For all the talk about learning styles over the last 15 years, we might expect that I was at risk of quickly losing my money.

Let me be clear, my argument is not that people don't have different learning styles, learning preferences, or learning skills. My argument is that for real-world instructional-development situations, learning styles is an ineffective and inefficient waste of resources that is unlikely to produce meaningful results.

Let me leave you with the original challenge:

"Can an e-learning program that utilizes learning-style information outperform an e-learning program that doesn't utilize such information by 10% or more on a realistic test of learning, even it is allowed to cost up to twice as much to build?"

The challenge is still on.

More Bogus Research Cited. This time at HRDQ.

I just came across another sighting of the mythologic numbers of memory retention, this time on the webpage of HRDQ.

Take a look:

They claim that, "Research shows people remember only 5% of what they hear in a lecture. But they retain 75% when they learn by doing." Bulloney!!

If you want to read my full debunking, click here.

If you want to see many bogus sightings, click here and scroll down.

Another Guru Sharing the Same Old Myth

And here's another example of a well-respected industry analyst lazily sharing the biggest myth in the learning field. This time it's from a Senior Industry Analyst with Forrester Research (October 19th, 2006). See recorded webinar.

Read my initial post describing how this myth got started, and how it harms our field and our learners.

The source of the offending PowerPoint slide claims the data as their own ("Source: Forrester Research"). Yeah, I guess if you find false information on the web, then change it around a little bit to help you make your point, that you ought to cite yourself. Is it plagiarism if you steal a lie?

Makes you wonder what other information Forrester has "researched."

To make it easier for the Forrester marketing and public relations folks to respond to this outing, I've developed a new logo for them. Instead of the name "Forrester" superimposed on the sea-green ellipsis, how about the following?

This constant myth-sharing should stop.

Do you think it would help if I started naming names? What about photographs? Email addresses?

Maybe sarcasm will work.

NTL continues its delusions

It's time to publicly vilify NTL Institute for Applied Behavioral Science for propagating the myth that learners remember 10% of what they read, 20% or what they see visually, etc. They continue to claim that they did this research and that it is accurate.

The research is NOT accurate, nor could it be. Even a casual observer can see that research results that end neatly in 5's or 0's (as in 5%, 10%, 20%, 30%) are extremely unlikely. To see a complete debunking of this hoax, click here.

Normally, I choose not to name names when it comes to the myths in our field. We all make mistakes, right? But NTL continues to harm our field by propagating this myth. Here is the document (Download pyramid_2.DOC) they send to people who inquire about the percentages. At least five separate people have sent me this document after contacting NTL on their own initiative.

I have talked to NTL staff people and emailed them (over a year ago), and even with my charming personality, I have failed to persuade them of the problems they are causing.

The people who write me about this are outraged (and frankly confused) that an organization would propagate such an obvious falsehood. Are you?

Here are claims that NTL makes in its letter that are false:

NTL: We know that in 1954 a similar pyramid with slightly different numbers appeared on p. 43 of a book called Audio-Visual Methods in Teaching, published by the Edgar Dale Dryden Press in New York.

Why false? There are NO numbers on page 43 of Edgar Dale's book.

NTL: We are happy to respond to your inquiry about The Learning Pyramid. Yes, it was developed and used by NTL Institute at our Bethel, Maine campus in the early sixties when we were still part of the National Education Association's Adult Education Division.

Very Intriguing: How could NTL have developed the pyramid in the 1960's, when a similar version was published by Edgar Dale in 1954? Professor Michael Molenda of Indiana University has found some evidence that the numbers first appeared in the 1940's. Maybe NTL has a time machine.

NTL: Yet the Learning Pyramid as such seems to have been modified and always has been attributed to NTL Institute.

No. It wasn't attributed to NTL by Dale. Dale thought it was his. And again, Dale did not use any numbers. Just a cone.

Okay, so now half of you hate NTL, and the other half of you hate me for being the "know-it-all kid" from 7th grade. Well, I'll take the heat for that. But still, is this the kind of field you want to work in?

And what is the advantage for NTL to continue the big lie?

Here's what NTL should write when people inquire:

Thanks for your inquiry to the NTL Institute. Yes, we once utilized the "Learning Pyramid" concept in our work, starting in the 1960's. However, we can no longer locate the source of the original information and recent research tends to debunk those earlier recommendations. We apologize for any harm or confusion we may have caused.

One more time -- People remember 10%, 20%...

Okay, here's another example of the same incorrect information that plagues our field. This is from a company named Percepsys:

Hopefully, sometime soon, the webpage on their site won't work because the vendor will smarten up and remove this misinformation.

It's NOT TRUE that people remember 10% of what they read, 20% of what they hear, etc. Moreover, if you know anything about learning, you'd know it would be impossible to pin down the amount of remembering. It depends on the materials, the learners, the duration of learning, the type of learning activities, the consistency between the learning situation and the retrieval situation, and the length of retention among other things. Finally, while this information (the 10%, 20%, 30% information) is often attached to Dale's Cone, Dale never actually had any numbers on his cone.

For the best review of the history of this misdirection, if I must say so myself, is here.

People remember 10%, 20%...Oh Really?

Publication Note

This article was originally published on the Work-Learning Research website (www.work-learning.com) in 2002. It may have had some minor changes since then. It was moved to this blog in 2006.

Introduction

People do NOT remember 10% of what they read, 20% of what they see, 30% of what they hear, etc. That information, and similar pronouncements are fraudulent. Moreover, general statements on the effectiveness of learning methods are not credible---learning results depend on too many variables to enable such precision. Unfortunately, this bogus information has been floating around our field for decades, crafted by many different authors and presented in many different configurations, including bastardizations of Dale's Cone. The rest of this article offers more detail.

My Search For Knowledge

My investigation of this issue began when I came across the following graph:

The Graph is a Fraud!

After reading the cited article several times and not seeing the graph---nor the numbers on the graph---I got suspicious and got in touch with the first author of the cited study, Dr. Michelene Chi of the University of Pittsburgh (who is, by the way, one of the world's leading authorities on expertise). She said this about the graph:

"I don't recognize this graph at all. So the citation is definitely wrong; since it's not my graph."

What makes this particularly disturbing is that this graph has popped up all over our industry, and many instructional-design decisions have been based on the information contained in the graph.

Bogus Information is Widespread

I often begin my workshops on instructional design and e-learning and my conference presentations with this graph as a warning and wake up call. Typically, over 90% of the audience raises their hands when I ask whether anyone has seen the numbers depicted in the graph. Later I often hear audible gasps and nervous giggles as the information is debunked. Clearly, lots of experienced professionals in our field know this graph and have used it to guide their decision making.

The graph is representative of a larger problem. The numbers presented on the graph have been circulating in our industry since the late 1960's, and they have no research backing whatsoever. Dr. JC Kinnamon (2002) of Midi, Inc., searched the web and found dozens of references to those dubious numbers in college courses, research reports, and in vendor and consultant promotional materials.

Where the Numbers Came From

The bogus percentages were first published by an employee of Mobil Oil Company in 1967, writing in the magazine Film and Audio-Visual Communications. D. G. Treichler didn’t cite any research, but our field has unfortunately accepted his/her percentages ever since. NTL Institute still claims that they did the research that derived the numbers. See my response to NTL.

Michael Molenda, a professor at Indiana University, is currently working to track down the origination of the bogus numbers. His efforts have uncovered some evidence that the numbers may have been developed as early as the 1940's by Paul John Phillips who worked at University of Texas at Austin and who developed training classes for the petroleum industry. During World War Two Phillips taught Visual Aids at the U. S. Army's Ordnance School at the Aberdeen (Maryland) Proving Grounds, where the numbers have also appeared and where they may have been developed.

Strange coincidence: I was born on these very same Aberdeen Proving Grounds.

Ernie Rothkopf, professor emeritus of Columbia University, one of the world's leading applied research psychologists on learning, reported to me that the bogus percentages have been widely discredited, yet they keep rearing their ugly head in one form or another every few years.

Many people now associate the bogus percentages with Dale's "Cone of Experience," developed in 1946 by Edgar Dale. It provided an intuitive model of the concreteness of various audio-visual media. Dale included no numbers in his model and there was no research used to generate it. In fact, Dale warned his readers not to take the model too literally. Dale's Cone, copied without changes from the 3rd and final edition of his book, is presented below:

Dale's Cone of Experience (Dale, 1969, p. 107)

You can see that Dale used no numbers with his cone. Somewhere along the way, someone unnaturally fused Dale's Cone and Treichler's dubious percentages. One common example is represented below.

The source cited in the diagram above by Wiman and Meierhenry (1969) is a book of edited chapters. Though two of the chapters (Harrison, 1969; Stewart, 1969) mention Dale's Cone of Experience, neither of them includes the percentages. In other words, the diagram above is citing a book that does not include the diagram and does not include the percentages indicated in the diagram.

Here are some more examples:

The "Evidence" Changes to Meet the Need of the Deceiver

The percentages, and the graph in particular, have been passed around in our field from reputable person to reputable person. The people who originally created the fabrications are to blame for getting this started, but there are clearly many people willing to bend the information to their own devices. Kinnamon's (2002) investigation found that Treichler's percentages have been modified in many ways, depending on the message the shyster wants to send. Some people have changed the relative percentages. Some have improved Treichler's grammar. Some have added categories to make their point. For example, one version of these numbers says that people remember 95% of the information they teach to others.

People have not only cited Treichler, Chi, Wiman and Meierhenry for the percentages, but have also incorrectly cited William Glasser, and correctly cited a number of other people who have utilized Treichler's numbers.

It seems clear from some of the fraudulent citations that deception was intended. On the graph that prompted our investigation, the title of the article had been modified from the original to get rid of the word "students." The creator of the graph must have known that the term "students" would make people in the training / development / performance field suspicious that the research was done on children. The creator of Wiman and Meierhenry diagram did four things that make it difficult to track down the original source: (1) the book they cited is fairly obscure, (2) one of the authors names is spelled wrong, (3) the year of publication is incorrect, (4) and the name Charles Merrill, which was actually a publishing house, was ambiguously presented so that it might have referred to an author or editor.

But Don't The Numbers Speak The Truth?

The numbers are not credible, and even if they made sense, they'd still be dangerous.

If we look at the numbers a little more closely, they are highly unconvincing. How did someone compare "reading" and "seeing?" Don't you have to "see" to "read?" What does "collaboration" mean anyway? Were two people talking about the information they were learning? If so, weren't they "hearing" what the other person had to say? What does "doing" mean? How much were they "doing" it? Were they "doing" it correctly, or did they get feedback? If they were getting feedback, how do we know the learning didn't come from the feedback---not the "doing?" Do we really believe that people learn more "hearing" a lecture, than "reading" the same material? Don't people who "read" have an advantage in being able to pace themselves and revisit material they don't understand? And how did the research produce numbers that are all factors of ten? Doesn't this suggest some sort of review of the literature? If so, shouldn't we know how the research review was conducted? Shouldn't we get a clear and traceable citation for such a review?

Even the idea that you can compare these types of learning methods is ridiculous. As any good research psychologist knows, the measurement situation affects the learning outcome. If we have a person learn foreign-language vocabulary by listening to an audiotape and vocalizing their responses, it doesn't make sense to test them by having them write down their answers. We'd have a poor measure of their ability to verbalize vocabulary. The opposite is also nonsensical. People who learn vocabulary by seeing it on the written page cannot be fairly evaluated by asking them to say the words aloud. It's not fair to compare these different methods by using the same test, because the choice of test will bias the outcome toward the learning situation that is most like the test situation.

But why not compare one type of test to another---for example, if we want to compare vocabulary learning through hearing and seeing, why don't we use an oral test and written one? This doesn't help either. It's really impossible to compare two things on different indices. Can you imagine comparing the best boxer with the best golfer by having the boxer punch a heavy bag and having the golfer hit for distance? Would Muhammad Ali punching with 600 pounds of pressure beat Tiger Woods hitting his drives 320 yards off the tee?

The Importance of Listing Citations

Even if the numbers presented on the graph had been published in a refereed journal---research we were reasonably sure we could trust---it would still be dangerous not to know where they came from. Research conclusions have a way of morphing over time. Wasn't it true ten years ago that all fat was bad? Newer research has revealed that monounsaturated oils like olive oil might actually be good for us. If a person doesn't cite their sources, we might not realize that their conclusions are outdated or simply based on poor research. Conversely, we may also lose access to good sources of information. Suppose Teichler had really discovered a valid source of information? Because he/she did not use citations, that research would remain forever hidden in obscurity.

The context of research makes a great deal of difference. If we don't know a source, we don't really know whether the research is relevant to our situation. For example, an article by Kulik and Kulik (1988) concluded that immediate feedback was better than delayed feedback. Most people in the field now accept their conclusions. Efforts by Work-Learning Research to examine Kulik and Kulik's sources indicated that most of the articles they reviewed tested the learners within a few minutes after the learning event, a very unrealistic analog for most training situations. Their sources enabled us to examine their evidence and find it faulty.

Who Should We Blame?

The original shysters are not the only ones to blame. The fact that many people who have disseminated the graph used the same incorrect citation makes it clear that they never accessed the original study. Everyone who uses a citation to make a point (or draw a conclusion) ought to check the citation. That, of course, includes all of us who are consumers of this information.

What Does This Tell Us About Our Field?

It tells us that we may not be able to trust the information that floats around our industry. It tells us that even our most reputable people and organizations may require the Wizard-of-Oz treatment---we may need to look behind the curtain to verify their claims.

The Danger To Our Field

At Work-Learning Research, our goal is to provide research-based information that practitioners can trust. We began our research efforts several years ago when we noticed that the field jumps from one fad to another while at the same time holding religiously to ideas that would be better cast aside.

The fact that our field is so easily swayed by the mildest whiffs of evidence suggests that we don't have sufficient mechanisms in place to improve what we do. Because we're not able or willing to provide due diligence on evidence-based claims, we're unable to create feedback loops to push the field more forcefully toward continuing improvement.

Isn't it ironic? We're supposed to be the learning experts, but because we too easily take things for granted, we find ourselves skipping down all manner of yellow-brick roads.

How to Improve the Situation

It will seem obvious, but each and every one of us must take responsibility for the information we transmit to ensure its integrity. More importantly, we must be actively skeptical of the information we receive. We ought to check the facts, investigate the evidence, and evaluate the research. Finally, we must continue our personal search for knowledge---for it is only with knowledge that we can validly evaluate the claims that we encounter.

Our Citations

Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182.

Dale, E. (1946, 1954, 1969). Audio-visual methods in teaching. New York: Dryden.

Harrison, R. (1969). Communication theory. In R. V. Wiman and W. C. Meierhenry (Eds.) Educational media: Theory into practice. Columbus, OH: Merrill.

Kinnamon, J. C. (2002). Personal communication, October 25.

Kulik, J. A., & Kulik, C-L. C. (1988). Timing of feedback and verbal learning. Review of Educational Research, 58, 79-97.

Molenda, M. H. (2003). Personal communications, February and March.

Rothkopf, E. Z. (2002). Personal communication, September 26.

Stewart, D. K. (1969). A learning-systems concept as applied to courses in education and training. In R. V. Wiman and W. C. Meierhenry (Eds.) Educational media: Theory into practice. Columbus, OH: Merrill.

Treichler, D. G. (1967). Are you missing the boat in training aids? Film and Audio-Visual Communication, 1, 14-16, 28-30, 48.

Wiman, R. V. & Meierhenry, W. C. (Eds.). (1969). Educational media: Theory into practice. Columbus, OH: Merrill.