People remember 10%, 20%...Oh Really?

Publication Note

This article was originally published on the Work-Learning Research website (www.work-learning.com) in 2002. It may have had some minor changes since then. It was moved to this blog in 2006.

Introduction

People do NOT remember 10% of what they read, 20% of what they see, 30% of what they hear, etc. That information, and similar pronouncements are fraudulent. Moreover, general statements on the effectiveness of learning methods are not credible---learning results depend on too many variables to enable such precision. Unfortunately, this bogus information has been floating around our field for decades, crafted by many different authors and presented in many different configurations, including bastardizations of Dale's Cone. The rest of this article offers more detail.

My Search For Knowledge

My investigation of this issue began when I came across the following graph:

The Graph is a Fraud!

After reading the cited article several times and not seeing the graph---nor the numbers on the graph---I got suspicious and got in touch with the first author of the cited study, Dr. Michelene Chi of the University of Pittsburgh (who is, by the way, one of the world's leading authorities on expertise). She said this about the graph:

"I don't recognize this graph at all. So the citation is definitely wrong; since it's not my graph."

What makes this particularly disturbing is that this graph has popped up all over our industry, and many instructional-design decisions have been based on the information contained in the graph.

Bogus Information is Widespread

I often begin my workshops on instructional design and e-learning and my conference presentations with this graph as a warning and wake up call. Typically, over 90% of the audience raises their hands when I ask whether anyone has seen the numbers depicted in the graph. Later I often hear audible gasps and nervous giggles as the information is debunked. Clearly, lots of experienced professionals in our field know this graph and have used it to guide their decision making.

The graph is representative of a larger problem. The numbers presented on the graph have been circulating in our industry since the late 1960's, and they have no research backing whatsoever. Dr. JC Kinnamon (2002) of Midi, Inc., searched the web and found dozens of references to those dubious numbers in college courses, research reports, and in vendor and consultant promotional materials.

Where the Numbers Came From

The bogus percentages were first published by an employee of Mobil Oil Company in 1967, writing in the magazine Film and Audio-Visual Communications. D. G. Treichler didn’t cite any research, but our field has unfortunately accepted his/her percentages ever since. NTL Institute still claims that they did the research that derived the numbers. See my response to NTL.

Michael Molenda, a professor at Indiana University, is currently working to track down the origination of the bogus numbers. His efforts have uncovered some evidence that the numbers may have been developed as early as the 1940's by Paul John Phillips who worked at University of Texas at Austin and who developed training classes for the petroleum industry. During World War Two Phillips taught Visual Aids at the U. S. Army's Ordnance School at the Aberdeen (Maryland) Proving Grounds, where the numbers have also appeared and where they may have been developed.

Strange coincidence: I was born on these very same Aberdeen Proving Grounds.

Ernie Rothkopf, professor emeritus of Columbia University, one of the world's leading applied research psychologists on learning, reported to me that the bogus percentages have been widely discredited, yet they keep rearing their ugly head in one form or another every few years.

Many people now associate the bogus percentages with Dale's "Cone of Experience," developed in 1946 by Edgar Dale. It provided an intuitive model of the concreteness of various audio-visual media. Dale included no numbers in his model and there was no research used to generate it. In fact, Dale warned his readers not to take the model too literally. Dale's Cone, copied without changes from the 3rd and final edition of his book, is presented below:

Dale's Cone of Experience (Dale, 1969, p. 107)

You can see that Dale used no numbers with his cone. Somewhere along the way, someone unnaturally fused Dale's Cone and Treichler's dubious percentages. One common example is represented below.

The source cited in the diagram above by Wiman and Meierhenry (1969) is a book of edited chapters. Though two of the chapters (Harrison, 1969; Stewart, 1969) mention Dale's Cone of Experience, neither of them includes the percentages. In other words, the diagram above is citing a book that does not include the diagram and does not include the percentages indicated in the diagram.

Here are some more examples:

The "Evidence" Changes to Meet the Need of the Deceiver

The percentages, and the graph in particular, have been passed around in our field from reputable person to reputable person. The people who originally created the fabrications are to blame for getting this started, but there are clearly many people willing to bend the information to their own devices. Kinnamon's (2002) investigation found that Treichler's percentages have been modified in many ways, depending on the message the shyster wants to send. Some people have changed the relative percentages. Some have improved Treichler's grammar. Some have added categories to make their point. For example, one version of these numbers says that people remember 95% of the information they teach to others.

People have not only cited Treichler, Chi, Wiman and Meierhenry for the percentages, but have also incorrectly cited William Glasser, and correctly cited a number of other people who have utilized Treichler's numbers.

It seems clear from some of the fraudulent citations that deception was intended. On the graph that prompted our investigation, the title of the article had been modified from the original to get rid of the word "students." The creator of the graph must have known that the term "students" would make people in the training / development / performance field suspicious that the research was done on children. The creator of Wiman and Meierhenry diagram did four things that make it difficult to track down the original source: (1) the book they cited is fairly obscure, (2) one of the authors names is spelled wrong, (3) the year of publication is incorrect, (4) and the name Charles Merrill, which was actually a publishing house, was ambiguously presented so that it might have referred to an author or editor.

But Don't The Numbers Speak The Truth?

The numbers are not credible, and even if they made sense, they'd still be dangerous.

If we look at the numbers a little more closely, they are highly unconvincing. How did someone compare "reading" and "seeing?" Don't you have to "see" to "read?" What does "collaboration" mean anyway? Were two people talking about the information they were learning? If so, weren't they "hearing" what the other person had to say? What does "doing" mean? How much were they "doing" it? Were they "doing" it correctly, or did they get feedback? If they were getting feedback, how do we know the learning didn't come from the feedback---not the "doing?" Do we really believe that people learn more "hearing" a lecture, than "reading" the same material? Don't people who "read" have an advantage in being able to pace themselves and revisit material they don't understand? And how did the research produce numbers that are all factors of ten? Doesn't this suggest some sort of review of the literature? If so, shouldn't we know how the research review was conducted? Shouldn't we get a clear and traceable citation for such a review?

Even the idea that you can compare these types of learning methods is ridiculous. As any good research psychologist knows, the measurement situation affects the learning outcome. If we have a person learn foreign-language vocabulary by listening to an audiotape and vocalizing their responses, it doesn't make sense to test them by having them write down their answers. We'd have a poor measure of their ability to verbalize vocabulary. The opposite is also nonsensical. People who learn vocabulary by seeing it on the written page cannot be fairly evaluated by asking them to say the words aloud. It's not fair to compare these different methods by using the same test, because the choice of test will bias the outcome toward the learning situation that is most like the test situation.

But why not compare one type of test to another---for example, if we want to compare vocabulary learning through hearing and seeing, why don't we use an oral test and written one? This doesn't help either. It's really impossible to compare two things on different indices. Can you imagine comparing the best boxer with the best golfer by having the boxer punch a heavy bag and having the golfer hit for distance? Would Muhammad Ali punching with 600 pounds of pressure beat Tiger Woods hitting his drives 320 yards off the tee?

The Importance of Listing Citations

Even if the numbers presented on the graph had been published in a refereed journal---research we were reasonably sure we could trust---it would still be dangerous not to know where they came from. Research conclusions have a way of morphing over time. Wasn't it true ten years ago that all fat was bad? Newer research has revealed that monounsaturated oils like olive oil might actually be good for us. If a person doesn't cite their sources, we might not realize that their conclusions are outdated or simply based on poor research. Conversely, we may also lose access to good sources of information. Suppose Teichler had really discovered a valid source of information? Because he/she did not use citations, that research would remain forever hidden in obscurity.

The context of research makes a great deal of difference. If we don't know a source, we don't really know whether the research is relevant to our situation. For example, an article by Kulik and Kulik (1988) concluded that immediate feedback was better than delayed feedback. Most people in the field now accept their conclusions. Efforts by Work-Learning Research to examine Kulik and Kulik's sources indicated that most of the articles they reviewed tested the learners within a few minutes after the learning event, a very unrealistic analog for most training situations. Their sources enabled us to examine their evidence and find it faulty.

Who Should We Blame?

The original shysters are not the only ones to blame. The fact that many people who have disseminated the graph used the same incorrect citation makes it clear that they never accessed the original study. Everyone who uses a citation to make a point (or draw a conclusion) ought to check the citation. That, of course, includes all of us who are consumers of this information.

What Does This Tell Us About Our Field?

It tells us that we may not be able to trust the information that floats around our industry. It tells us that even our most reputable people and organizations may require the Wizard-of-Oz treatment---we may need to look behind the curtain to verify their claims.

The Danger To Our Field

At Work-Learning Research, our goal is to provide research-based information that practitioners can trust. We began our research efforts several years ago when we noticed that the field jumps from one fad to another while at the same time holding religiously to ideas that would be better cast aside.

The fact that our field is so easily swayed by the mildest whiffs of evidence suggests that we don't have sufficient mechanisms in place to improve what we do. Because we're not able or willing to provide due diligence on evidence-based claims, we're unable to create feedback loops to push the field more forcefully toward continuing improvement.

Isn't it ironic? We're supposed to be the learning experts, but because we too easily take things for granted, we find ourselves skipping down all manner of yellow-brick roads.

How to Improve the Situation

It will seem obvious, but each and every one of us must take responsibility for the information we transmit to ensure its integrity. More importantly, we must be actively skeptical of the information we receive. We ought to check the facts, investigate the evidence, and evaluate the research. Finally, we must continue our personal search for knowledge---for it is only with knowledge that we can validly evaluate the claims that we encounter.

Our Citations

Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182.

Dale, E. (1946, 1954, 1969). Audio-visual methods in teaching. New York: Dryden.

Harrison, R. (1969). Communication theory. In R. V. Wiman and W. C. Meierhenry (Eds.) Educational media: Theory into practice. Columbus, OH: Merrill.

Kinnamon, J. C. (2002). Personal communication, October 25.

Kulik, J. A., & Kulik, C-L. C. (1988). Timing of feedback and verbal learning. Review of Educational Research, 58, 79-97.

Molenda, M. H. (2003). Personal communications, February and March.

Rothkopf, E. Z. (2002). Personal communication, September 26.

Stewart, D. K. (1969). A learning-systems concept as applied to courses in education and training. In R. V. Wiman and W. C. Meierhenry (Eds.) Educational media: Theory into practice. Columbus, OH: Merrill.

Treichler, D. G. (1967). Are you missing the boat in training aids? Film and Audio-Visual Communication, 1, 14-16, 28-30, 48.

Wiman, R. V. & Meierhenry, W. C. (Eds.). (1969). Educational media: Theory into practice. Columbus, OH: Merrill.

Comments

[a kas]

thanks for this..i am on postgrad course doing an essay on planning 4 maths -- Dales cone interested me and I'm glad there are pple like u 2 put the record straight 4 pple like me! thank you again

[Dennis Coxe]

Wow. Talk about coincidence. I had just stumbled over that chart in an instructional design text book last night without any reference and I wondered at the equal displacement of learning increments.

Of course, the book did not have list any source for the chart, but did present it as gospel truth.

For one reason or another I had (fortunately) forgotten about it and continued on in my career. Thanks for shedding a great deal of light on it.

[Shron Burnell]

I came across a lovely colourful version fo the diagram - it just looks so nice. I thought to do a sarch to try and track downthe basis for the numbers - your writing is nicely though out and I enjoyed reading it. Thanks for all your effort in tracking references

[jj]

Wow...Thanks for your wonderful critique, which answered my questions about Dale's Cone of Experience. I did a google search on Dale's Cone of Experience and found so many different versions of the graphics but only very few of them have references. Thanks a lot.

[Matthias Drochner]

So, you killed and buried my beloved Cone of Experience. Yes, I'm one of those deceived by seeing it pop up everywhere and believing the masses of the quote-quoters, repeating error and myth.

BUT: Do you have any research evidence for the effectiveness of, say, practicing role-playing for sales people and counselors. Or research about the effectiveness of giving practice speeches for learning public speaking?

I have been looking for this kind of material, but somehow I'm stuck and can't seem to find firm statements; all I find are fuzzy recommendations, "It would be good if your students practiced this or that in a role-play."

Got any pointers or recommendations? Here in the jungle where I teach I'm kind of far away from any kind of library except my own, but I have some nice mail carriers who visit me one in a while down here.

Keep up the good work, err, hobby!

Matthias, from the jungles of Peru

[Melissa Spore]

Thank you, thank you, thnk you for this.

I've believed that damn pyramid was wrong from thefirst day I saw it. Now you have gioven me the argument & citations.

[Isaac Oyeyiola]

I regard this piece of investigative work for academic writings invaluable.
Weeldone! (intentional spellong error)

[Angie]

Thank you for this - I have been fooled too and have used the percentages in material I have prepared for my adult students. I was googling to get the correct quote and reference when I came across your blog. Really interetsing to show students how we easy it is to be mislead and how we need to carefully verify sources of information.

[KPC]

This info needs to be publicized greatly. You have no idea how the percentages version of Edgar Dale's "Cone of Experience" is taught as unassailable truth & fact in most teacher/trainer certification programs. Even I was taken in by it at 1st, even though I myself am geared toward learning from print-based/ text-based reading. When I came across the info posted here & 1 other place [why only 1 other?], I felt hoodwinked. Of course, the historically accurate truth as presented here makes more sense. This info is what should really be taught.

[Noura Maalaoui]

Thank you f or this article; it really clarifies things. I was preparing a TOT and was looking for d’Edgar DALE's Piramide including the numbers and found this ienterting article.
Thanks for the corrections
Noura

[Dalize]

Point taken about the numbers in Edgar Dale's Piramide, but what is the suggested alternative? So many others have also "proven" the theory Knowles '70, Carl Rodgers, Barer-Stein and Draper '88, David Kolb '80.

[Renee]

So what are the real statistics in retention? Specifically in reading, hearing, seeing and doing - and the combination of some and all of those?

I would really like to know the real statistics or an average statistical suggestion based on real evidence.

thanks!!

Please email me!

[Renee]

So what are the real statistics in retention? Specifically in reading, hearing, seeing and doing - and the combination of some and all of those?

I would really like to know the real statistics or an average statistical suggestion based on real evidence.

thanks!!

Please email me!

[Cassandra]

Finally a site that sheds some light on a statistic I've heard for years, yet recently decided to investigate.

As a self-development expert and author that trains youth and adult audiences on a variety of topics, it's important to me that I know the facts of what I teach.

I thank you for providing this article with such a detailed breakdown of how this statistic is nothing more than a 'myth'.

I also agree with Renee, so what is the real statistic?

[Matt Kempke]

Thank you for your article. I am just writing a text for a seminar that I attended and was already wondering about the new versions of Dales Cone. Since our library doesn't own Dales original writing including the cone I was almost fooled by the forged new versions that you can find anywhere in the web. I almost believed it. Thanks a lot! Matt

[Guy B. Scandlen]

Thak you for this because I have been using this rather extensively in my training in communication for behaviour development with UNICEF.

Now that you have debunked this, I think it would also be highly professional of you to refer us to the research that DOES attribute some evidence of the efficacy of variuous learning strategies with statistics.

I am working on some neurological principles of learbing and behaviuour change, but in reducing my work to a training exercise, the Cone proved a useful way to deomstrate some of these principles.

What would you suggest?

Kindest regards.

Guy B. Scandlen
87/2 Moo 2
Tambol Dawn Gaew
Amphur Mae Rim
Chiangmai 50180
Thailand

[Chris]

Good instructional design will include visual , auditory and kinaesthetic surely?
This will help people with different learning styles.

[Jannette Finch]

I am Jannette Finch, one of the people influenced by the infamous "cone." My name is on the last graph pictured.

I now use my own work, along with the graph so beautifully reinterpreted by Elaine Montambeau, as an example for classes I give to students in evaluating quality websites. I invite you to do the same. Everything indicates that the website hosting the retention rates graph is legitimate, with known authors, acknowledgments, dates last updated, etc., but when you read closely, as the author of this blog does, and bother to track down the source, you will find that the evidence presented is based on faulty information.
It is a wonderful, personal example of how we must be VERY thorough and suspicious consumers of information.

[Will Thalheimer]

Jannette,

Thanks for your comment and let me commend you as an example of the kind of professional we need in the field of learning. None of us is going to be right 100% of the time. When we're wrong, we ought to admit our mistakes and share our newfound wisdom with others.

Thanks from the field!!

[JC Craver]

"Shams and delusions are esteemed as soundest truths, while reality is fabulous."

Can you point to credible data on this topic? Anyone? I want some industry support for best practices in adult learning theory based on credible research.

Keep up the good work. Thanks for the article.

[Pete Hybert]

This comment may be a little late but thanks for a solid treatment of a chronic problem in the training business.

The bummer is that it is really easy to remember and get heads nodding at "10% of what you read, 20% of what you hear...etc." It sounds like common sense. So, business people (vs. learning specialists) like it and like to use it...little maxims like that become part of the vocabulary and culture in the business.

Then, because it helps us when we are trying to convince someone that just sending out an email or writing something in a procedure manual won't effectively build capability, it is tempting to use that lever rather than take a more difficult path...and in the process you end up reinforcing it.

Clearly, the desire to make things seem beyond argument by citing bogus research is dangerous. Maybe the desire to make things too simple is just as bad.

Thanks for posting the article!

[Ken Spell]

While I am glad to hear this practical look at a long-assumed concept, the article says nothing about what is out there on learning retention. For me, this is kind of like saying this is wrong without providing a "this is right" alternative.

So what is out there that we designers of content can rely upon?

[Ken Spell]

While I am glad to hear this practical look at a long-assumed concept, the article says nothing about what is out there on learning retention. For me, this is kind of like saying this is wrong without providing a "this is right" alternative.

So what is out there that we designers of content can rely upon?

[William Thacker]

Thank you for this article. The last graph I saw cited Bruce Nyland or Bruce Hyland at the University of Wisconsin as revising and expanding Edgar Dale's work. However, I can not find information about Bruce Nyland/Hyland. Is this person another ruse?

[Philip Schoewe, DREI]

My 30 years of experience in the field of real estate education and training is summed up in my poor attempt to remember the little latin retained from college in the 1960s. "Solvitur ambulando."

Because people learn as differently as they eat, early in the first hours of any real estate course I observe each learner as they observe me and as they work "with" another learner on items cleverly requiring them to expose their learning tendency.

Like eating, when the social interaction becomes friendly enough to show their habits, learners indicate preferences in how they want to be fed information. I deliver the prepared material content in a most favorable way to the learners, often mixing the teaching method while staying on topic in the syllabus. This, of course, is not possible or credible with a large mass of adult learners unless you have developed an almost pendantic style of teaching masked and wrapped cleveryly in a highly energetic, boyant, interactive personality style.

At any rate, as the learners grasp core knowledge and are often unknowingly herded to mastery on a topic, staccato role play or longer, involved, real life application discussions move us all, including the teacher/instructor, into solutions by walking through the thinking and application process of what the day's topic mastery was agreed to be in the syllabus...however we get there!

Again, "Solvitur ambulando." We solve it by experiencing, by walking through the matter ourself. So, I, too find this article inwardly exploratory and a welcome input to my own experiences as I "walk along" with learners for I have come to believe that the atmosphere of learning is as important or perhaps more important than a pre-planned delivery formulated on hear, see and do percentage charts.

To conclude my positive remarks on the exposure of this article to more teacher/instructors, "the teacher appears when the learner is ready to learn." I strive to create a safe learning atmoshphere and to be prepared, myself, for that learning moment so that "solvitur ambulando" applies in each topic. Reading this article will better prepare me. Thank you.

As a postscript, when a teacher has class of all one type of leaners, and this will happen in your career with small, higher lever course topics, all the percentages of see, hear and do go out the window. What does not abandon the class space is the teacher's awareness of what type of learner is in front of teacher and the need for the teacher to reach across the learner-teacher knowledge barrier and deliver, like good food, well-prepared topically correct information in a way that it can be digested.

[Mike McCartney]

Dear Will,
Your excellent research reminds me of the caveat, "33.3% of all statistics are made up on the spot."
Thanks!
Mike

[Jay Banks]

I am a PhD candidate working on a course project that set out to cite and re-frame the original 3M/Wharton Business College research that I had been taught (and as you succinctly stated) incorrectly cited for years...WOW (best scholarly verbiage I could initially come up with)! Now this makes for a more interesting project...and I have several questions:
1. Has anyone done credible research that would attach numbers/percentages to teaching/learning styles? If not, might this be the topic of a "once and for all" study?
2. This is the kind of study that would make me put my current dissertation research on the back burner and "cook" this one up. What ingredients should I use and peruse through? Meaning, how might I conduct this research? Any suggestions on how to test Dale's Cone of Experience via numbers/percentages (i.e. elements of the study, methodology, etc.)? Anyone currently considering this research?

Jay Banks, M.Ed.
Capella University PhD Candidate

[Ruby]

nice article - thanks! Reminds me of the number of words Eskimos have for snow!
"For my part, I want to make one last effort to clarify that the chapter above isn't about Eskimo lexicography at all, though I'm sure it will be taken to be. What it's actually about is intellectual sloth.
Among all the hundreds of people making published contributions to the great Eskimo vocabulary hoax, no one had acquired any evidence about how long the purported list of snow terms really was, or what words were on it, or what criteria were used in deciding what to put on the list. The tragedy is not that so many people got the facts wildly wrong; it is that in the mentally lazy and anti-intellectual world we live in today, hardly anyone cares enough to think about trying to determine what the facts are. " Pullum, G "The Great Eskimo Vocabulary Hoax" 1991
http://ling.ed.ac.uk/~gpullum/index.html

[Ruby]

nice article - thanks! Reminds me of the number of words Eskimos have for snow!
"For my part, I want to make one last effort to clarify that the chapter above isn't about Eskimo lexicography at all, though I'm sure it will be taken to be. What it's actually about is intellectual sloth.
Among all the hundreds of people making published contributions to the great Eskimo vocabulary hoax, no one had acquired any evidence about how long the purported list of snow terms really was, or what words were on it, or what criteria were used in deciding what to put on the list. The tragedy is not that so many people got the facts wildly wrong; it is that in the mentally lazy and anti-intellectual world we live in today, hardly anyone cares enough to think about trying to determine what the facts are. " Pullum, G "The Great Eskimo Vocabulary Hoax" 1991
http://ling.ed.ac.uk/~gpullum/index.html

[Ruby]

nice article - thanks! Reminds me of the number of words Eskimos have for snow!
"For my part, I want to make one last effort to clarify that the chapter above isn't about Eskimo lexicography at all, though I'm sure it will be taken to be. What it's actually about is intellectual sloth.
Among all the hundreds of people making published contributions to the great Eskimo vocabulary hoax, no one had acquired any evidence about how long the purported list of snow terms really was, or what words were on it, or what criteria were used in deciding what to put on the list. The tragedy is not that so many people got the facts wildly wrong; it is that in the mentally lazy and anti-intellectual world we live in today, hardly anyone cares enough to think about trying to determine what the facts are. " Pullum, G "The Great Eskimo Vocabulary Hoax" 1991
http://ling.ed.ac.uk/~gpullum/index.html

[Ruby]

nice article - thanks! Reminds me of the number of words Eskimos have for snow!
"For my part, I want to make one last effort to clarify that the chapter above isn't about Eskimo lexicography at all, though I'm sure it will be taken to be. What it's actually about is intellectual sloth.
Among all the hundreds of people making published contributions to the great Eskimo vocabulary hoax, no one had acquired any evidence about how long the purported list of snow terms really was, or what words were on it, or what criteria were used in deciding what to put on the list. The tragedy is not that so many people got the facts wildly wrong; it is that in the mentally lazy and anti-intellectual world we live in today, hardly anyone cares enough to think about trying to determine what the facts are. " Pullum, G "The Great Eskimo Vocabulary Hoax" 1991
http://ling.ed.ac.uk/~gpullum/index.html

[tahiya]

My experience before HPT and training is special education. In special education programming the data is everything. And the fact that data are custom per student makes all the difference. Some day it won't matter what most subjects in a study do because we'll have the bandwidth and the tech to allow each person to be profiled as a learner BEFORE instructional materials are created for them. Every training will be customized, not to a learner population, but to the learner, like special ed is now. Data will still be the most important thing, but you'll know it's correct because it will only be about one person.

[Becky]

For those of you looking for a credible source of info, here's a possibility:
Lalley, J., & Miller, R. (2007). The learning pyramid: Does it point in teachers in the right direction? Education, 128(1), 64-79. Enjoy!

[Will Thalheimer]

Thanks for commenting on my blog and for bringing the article by Lalley and Miller to my attention.

Note that the authors conclude:

“No specific credible research was uncovered to support the pyramid, which is loosely associated with the theory proposed by the well-respected researcher, Edgar Dale. … While no credible research was uncovered to support the pyramid, clear research on retention was discovered regarding the importance of each of the pyramid levels: each of the methods identified by the pyramid resulted in retention, with none being consistently superior to the others and all being effective in certain contexts.” (p. 64)

This is pretty damning to the pyramid notion. There is no research support, AND there is no evidence that any of the components are better than any others. This suggests a cylinder, not a cone.

The authors, who cite my criticism in their first paragraph by the way, seem to want to support the pyramid’s components nonetheless. Unfortunately, looking for corroborating evidence is bad science. It’s subject to confirmatory bias. It’s just too easy to find evidence that things like “reading” are valuable. Is this really noteworthy?

The pyramid is still for cone heads.

Thanks again!! I'm so glad you found this article.

[Liz Walker]

Will, great research and comments. Studying Adult Learning at the moment and our tutor has just taken us through the Learning Pyramid emphasising the percentages. Your research will enlighten many.
Thanks
Liz, Wales (UK)

[Richard Kool]

I am aware of one test, you might say, based in part at least on Dale's Cone:
Peart, B. (1984). Impact of exhibit type on knowledge gain, attitudes and behavior. 27(3), 220-236.

Bob experimentally manipulated a museum exhibit to look at the influence of design that had only words, pictures and words, only objects, object and words (standard exhibit), and objects, words and sounds. These exhibit manipulations were explicitly linked to Dale's 1954 paper talking about abstract and concrete modes of learning.

Bob Peart's findings found a consistent pattern of increased learning from the exhibit, increased exhibit attracting power, exhibit holding power, and more interaction with the exhibit, as the exhibit became more 'concrete'.

It's not much, perhaps, but it is the only explicit test of the Cone model that I am aware of!

Richard Kool
[email protected]

[Liz]

Most informative - and somewhat disheartening. I'm a trainee teacher and I've lost track of how many of my lecturers have presented us with those figures. I shall post this up on our noticeboard.

[Johnny Bolster]

While I thank you for taking the time to critique these supposed truths, I must contend that this article of far less practical use to you intended audience without including at least some indication of more valid statistics. Your so close. Please just lead with the truth and we can all more easily make the corrections on our minds.

[Rob Wilson]

I came across your blog posting debunking the “10%, 20%, oh really” use of data for how people best learn. I am not surprised, but was very interested to read your findings since I was wondering exactly where the original data came from. I too have seen the data used numerous times in presentations. However, having just finished my Masters in Training and Development from Oakland University (Michigan), I was prone to wanting to see citations.

My question is, how do you normally think about the validity of posting such as yours that YOU come across on the web. What you say is well written and appears to be legitimate, but would it be possible for your disclaimer to be inaccurate. Said another way, are there more traditional articles in journals that say the same thing, or have you published these findings elsewhere?

Thanks for the posting.

[Kelly O'Leary]

I have long noted the omission of writing as part of Dale's Cone: the passive interpretation of verbal and visual symbols as opposed to those that are self-generated.

[Sheila Dymond]

I really don't have words to thank you for this article. I came across it 'accidentally' while researching theories for a class on gifted endorsement I am teaching. Since I teach high school, I can not even count the number of times this false information has been handed down as gospel truth-even by ME! I will now look more closely at information I receive which shapes how I teach children in my classes. Thank you. Now, can someone steer me in the CORRECT direction????

[Charles]

Did you see the recent Cisco report that corroborates your post here? It's at http://www.cisco.com/web/strategy/docs/education/Multimodal-Learning-Through-Media.pdf

[Lisa Deitch]

The numbers and the graphs may be bogus, but my personal experience is that I DO remember things better if I listen/read/recite out loud/write/draw/act out/teach to someone else whatever it is I'm trying to learn.

[Danny Alexander]

I had always heard this mess as I was growing within the field of Education, Instruction and Training. I could actually see this was NOT true by the results shown through student performance over an array of instructional formats.
I was so delighted to see that you took on the established norms and debunked them.
I wish all of us, including ME, would have taken the time and expended the energy to do this and do it properly.

Perhaps now all of us will devote more time to research.

Thanks Will!

[Laura Gibbs]

I just read this old post thanks to it being mentioned by Stephen Downes in OLDaily today. This is a great post, but cynically I have to say that it reinforces my perception as a mere "practitioner" that about the only thing I can do to to improve the quality of my work is to monitor my student outcomes carefully, and design my courses based on a continuous questioning of my own assumptions, relying on what I can learn from observing myself and my students closely each semester. I have never been impressed by what comes out of the Ed. School on my campus and this article sadly helps to reinforce that prejudice on my part. I just finished reading the book True Enough by Farhad Manjoo - a GREAT book, by the way - and the situation you have uncovered here fits perfectly into the scenario he describes so well of people seeing exactly what they want to see, regardless of any kind of actual search for the truth of the matter.

[Ellen Pham]

I'm so disappointed! That was such handy "proof" of hands-on learning for us elementary teachers! And for peer teaching, teaching being an activity where you remember 90% of what you learned! Damn...

[Jim Lalley]

The comment by Will Thalheimer that Lalley and Miller"seem to want to support the pyramid’s components nonetheless" is misleading. We support each of the methods listed in the pyramid because we found empirical evidence for them. We in no way support the structure of the pyramid or the retention rates purported for the methods. As we stated in our article, the numbers are speculation.

[Will Thalheimer]

Jim,

You say, "We support each of the methods listed in the pyramid because we found empirical evidence for them."

Yes, you looked for evidence of them and found it. But your methodology for finding evidence is flawed because it doesn't rule out (it actually encourages) confirmatory bias. In lay terms, you found what you wanted to find. Finding evidence, for example, that "seeing" something helps people learn doesn't seem very earth shattering. More importantly, why then even mention the cone? If you want teachers to know that "seeing" something has value, then you ought to review that research, determining both generalities and boundary conditions.

If we want to help teachers know how best to teach, is the cone the best way to teach them? Heck no!! Is it too simplified and misleading? Yes!! We have a responsibility to give teachers (and all learning professionals) better mental models of how learning works than the cone. Your article, while fair and balanced, unfortunately becomes an apologist appeal to a poor model. Teachers can invest a little more time to get to really understand learning at a deep level. We have a responsibility to help them in that regard.

While we try to wipe overly simplistic notions off the map, the Lalley and Miller (2007) article will only extend the life of the Edgar Dale Cone model and the bogus numbers often associated with them. Did you notice how the Lalley and Miller article was originally presented in the comments above?
Commenter Wrote: "For those of you looking for a credible source of info, here's a possibility." All through education land, people will point to the article as "credible evidence" that the cone is valuable.

Alas....

[Judith Gustafson]

Nobody here has mentioned the November 2002 AECT presentation delivered by Tony Betrus and Al Januszewski of SUNY-Potsdam. They make exactly the same points, especially regarding the "Cone of Experience." Their PPT is available at
www2.potsdam.edu/betrusak/AECT2002/dalescone_files/dalescone.html.ppt

[Will Thalheimer]

Thanks Judy!!

I didn't know about their presentation and they do such a nice job of describing what Dale DID say that more people ought to know about this!!

I just posted notification of their presentation and your comment about it at:

www.willatworklearning.com/2008/08/excellent-revie.html.

--Will

[Courtney Lee]

wow! thx for this awesome writing piece!
i can't believe i was absolutely fooled by the diagram!! i kinda like the cone though..

[Joe Kirby]

I enjoyed your article. I was somewhat taken aback by some of the posting comments about Dale's model. As you point out, Dale never presented the cone as a research result backed up by data: only as a theoretical model. Subsequently others took the model and tried to make it "real".

[T Lewis]

Wow!!! Professors, Docotoral Educators, Superintendents and many others have used the "Learning Pyramid" to reform teaching practices and learning experiences.

When I searched to find the origin of the research for the "Learning Pyramid" several years ago and I could not find what I definitive source for the research I thought it was because I had not dug deep enough.

Wow, I just think that somehow I always wanted to know how the research that accounts for the percentages.

I am grateful for this article.

[Chuck]

Great article, thanks. Let's work on that feedback loop so we can all stay informed. Any update on some good statistics. I've been asked by many of my clients about retention. Mostly about how long the learning will be retained if not used immediately. I'm a big proponent of JIT and I do primarily new technology training for adult learners. I've found many statistics but now am doubting them.

[Brian Jensen]

I first saw this in Washington State volunteering as a hunter safety instructor. Every time I was trained the pyramid came out from the NTL Institute. It's sad that this has been put out there by so many for so long, and with so many personal and professional changes...Ah, well. Nice job with the blog though. Has anyone tried to "replicate" these numbers with any kind of success?

[Alexandre]

A useful post. You could have spent more time on the credibility of "general statements on the effectiveness of learning methods," as it seems to be a central issue in your field.
One useful thing with source criticism (and critical thinking in general) is that it pushes us to think about authors' purposes and about why some ideas may "stick."
In this case, it seems likely that the reasons so few people have searched for the original data may have a lot to do with the usefulness of the core argument for those who seek to evaluate the effectiveness of different teaching methods. Not only is the model intuitive but it packages a number of ideas in an elegant and easy-to-explain graphical form. In folkloristics, this could be compared to a "motif" which is a narrative unit that can easily be transmitted. Educators tend to share a lot of things like these and these "motifs" can help in "confirming what we already suspect."
There's also a kind of "math envy" in education. Quantitative data points sound more impressive than qualitative ones and are more easily used to "prove" a point. The same people who so distrust anecdotal evidence may take numeric formulas at face value.

Altogether, this approach (common in some English-speaking educational contexts) is what I tend to call the "studies have shown" approach. As academics, we're trained not to trust any research by itself and we usually have no concept of research projects simply "proving" a given point. My personal favourite was something about studies concluding that "63% of students perform better if the presentation method is changed within 20 minutes because their attention span is so low." Or some such. I don't remember the precise details. Nor do I agree with the basic concept (that the only way to cope with students' attention span is to fragment out course meetings using some formula).

Going back to learning effectiveness, the main point in all of this...
Isn't there a notion, somewhere, that both learning and teaching are influenced by a large variety of factors and that there's no magic formula to account for learning?

Now, my perspective is probably a radical one. I get the impression that learning happens despite formal teaching. Retention, the main target of many of these studies, isn't something which matters so much in my field (ethnography). Those of us who really care about lifelong learning may even say that the true effects of the learning process only come up long after the course has ended and are extremely difficult to pinpoint. Sure, this all makes it much more difficult to justify our work, if we don't have "hard data" about learning effectiveness through a specific teaching method. But is research really meant to justify our work?

[James Grason]

Mistaken data are a problem.

Dangerous is a judgement and not helpful to the discussion.

Lack of data does not imply correctness or incorrectness in the application and use.

Certainly it is possible to find controlled tests to validate a theory. Your hyperbolic constructions of testing golf against boxing are simple rhetoric.

Leave the investigative reporting to the politics.

A much more powerful "debunking" is to "bunk" the truth. Given the widespread use over such a long period - collecting longitudinal data should not be a problem.

Most helpful would be to add data and new analysis which leaves us with a new assessment of the cone.

[Phlox Drummondii]

thx, thought it was too simple and you proved me right. What about Howard Gardner and his "Multiple intelligences", a framework in our country (Belgium) often used to answer diversity issues

[Ken Masters]

Thanks for a great article. Although I have not ever used the figures, I have heard them several times, and was certainly duped by them.

The real tragedy is that this is NOT an isolated incident. A few years ago, I wrote an article tracing how work from a World Bank author (Michael Gibbons) had been twisted and edited, and then mis-quoted (and usually attributed directly to him) by authors who had obviously not referred to the original source. The problem was then compounded by the fact that some highly-influential documents also mis-quoted Gibbons, and then these became primary sources in their own right. I am sure that many others must have found similar incidents in their fields. It would be an interesting piece of research to try to collate these and see just how much of this is actually happening.

[Bas Jongenelen]

I discovered recently the non-existence of the learning cone. I wrote a short article about it, which was published in a Dutch national newspaper: http://tinyurl.com/bas-in-trouw for the internetpage, http://www.basjongenelen.nl/files/leerpiramide%20in%20het%20onderwijs%20bestaat%20niet.pdf for a pdf of the paper version.

[Theo Selles]

Excellent article! Good work. It's far too easy for "soft science" type folk to throw crap out there just to benefit themselves and justify their importance.

[Lilia Bejec]

This is an amazing news to me after using this for many years. While it is true that there are different types of learning styles and multiple intelligences, there are also benefits we can derive from this theory. I've been teaching different levels of education and the more the students are exposed to concepts and contents the better they retain them.

[Brett]

I agree with a lot of this, but I guess my question is this. Did everyone really think some research study would produce such nice round numbers incrementing by 10%?

I always used and presented this as some sort of idealized hierarchy someone literally made up. To me he important point here is that the more actively engaged and responsible you are for any given bit of material the more likely you are to remember it. Also, really? Do people think they remember 90% of what they teach? Good luck.

I doubt that anyone would see one of these graphs and think "Oh I guess I'll get them to see if before I get them to read it." People naturally focus on the upper end. So the real danger here is that someone might use this in a general way to say experiential learning and being responsible to others is good for your learning. I guess one other danger, as mentioned in several other comments, is that people might think Education research can be flaky. Oh noes! What will we do?! Oh. wait. Like all academic fields it often is.

Anyway, excellent bit of work tracking down all the history. William Glasser indeed.

[Kimberly]

I am on a simliar hunt for the original research for the Transfer of Training Grid by John Newstrom and Mary Broad. Research indicates what the manager does before and after a training event has greater impact than the learner during the event.
Thoughts?

[Pablo]

For scientifics data about learning:

http://www.rogerschank.com/

[Adam Neaman]

I am a little blown away to find one of the industry's most prominent analysts using not this graph but his firm's own version of it... They designed a study to successfully generate these results (with a level of validity and reliability - to say nothing of specificity - comparable to that of the original research).

[Adam Neaman]

My bad. The data is actually drawn from the NTL study, though said analyst put a copyright into his book's graph of the NTL data citing to his own research. And, said analyst continues to use this data in reports that long post-date Will's original posting.

[Molly Cooke]

The cone showed up this week in the quarterly bulletin published by my high school and circulated to alumni. I had not seen it before (although I work in higher education, I don't do training). It struck me as far too good to be true. Thanks for the careful de-bunking. I sent it to the person who was citing it, percentages and all. Thanks for the careful investigation.

[dubai property]

I really enjoyed this article – the universal things fit into any situation at any time. How about: Works For Me, Won’t Fix – most common sentence spoken among my colleagues developers.

[Brigette Thomas]

I agree. Human brain has some ratio that it stores and that it washes away. Very beautiful article.

Thanks,
Brigette
www.dailytrader.com

[UFO]

The source of the data is not readily available to neither validate nor dispute the pyramid. Such is the case with many “truths" we base practices on. Having several citations as this Blog has (suppressed giggle), does not increase its validity. You will note that none of the persons cited contribute anything to refute the numbers.

In addition the pyramid implies teaching, which may employ all of the above methodologies and an understanding of the material taught; therefore it is logical the (average) rate of retention would be higher.

Finally it would be obviously unethical to do research to support or refute this war on the pyramid. So people will have to use their own experience and professional practice to determine the efficacy of experiential learning.
Keep using the pyramid with a disclaimer if it resonates with your professional practice; if you don't believe the pyramid is helpful, don't worry there a countless automatons ready for mass lectures and legions of budget conscious administrators wanting to end less profitable methodologies.

[John Richbourg]

Thanks for tracking this incorrect information down to its (apparent) source. As a doctoral candidate, it's a real dash of cold water to the face. I have been meticulous in finding and using cited materials for my responses. It's very sobering to have some of the naivete' peeled away with the understanding that possibly not all of the "experts" I have been learning from are as accurate as one would hope.
Keep up the great work. I have bookmarked your main site so I can review what you have been up to on a regular basis.

Thanks again!

[Divisi, retention specialist]

Last time I checked theory meant not proven. And this is a learning theory! I think any theory you have to take for what it is "a theory" and check it out for yourself. And yes, I agree learning has too many variables which is why this would be very hard to research and turn into a fact. However, speaking from experience, the more you interact with the learned material and live it the better you retain it. So if you are going to cite this, I agree be careful to give it only the credit it is due in theory!

[richel lawaguey]

it would be better if you also posted some possible solutions to this issue,.for us to have an idea on how to be guided by this matter,.

[Bill]

so then, does all this mean that the old saying, "the best way to learn is by DOING" is not true?

[Britt]

Of course it could be true, what our blogger is trying to say is, as a whole those numbers are a lie. There is no valid source or research to back up those numbers. However, it is very possible that you may only remember 10% of what you read or see. It all depends on the way an individual learns and with what learning style they're most confident with.

[Sue]

Very interesting! When I first read the cone-it almost made sense because I am more a visual learner but then to hear all the facts and blogs-who knows?

[A Researcher]

Sorry, but this is a poor critique. There is no alternative statistical evidence that has been provided to "disprove" this claim. As such, this argument can be used to claim that any research that is sufficiently old so as to be untraceable is a myth.

Unless alternative evidence is provided, such critiques should be dismissed.

[nike jordan shoes]

Perhaps the young season, the young messenger who really easy to reckless personal danger.

[Strani]

I wouldn't agree that only 10% goes to reading. At least 20 or 30, in my opinion.

[Maria Persson]

Informative and food for thought. I have always thought that if you get given info it is your responsibility to sieve through it, discuss it further, unravel and seek the original source to give it credibility!
Intellectual property and ownership of it is a fascinating area for discussion and with the era of e Learning and e Teaching we have to become more aware of the inundation of information and like you say - research!

[Pellet Mill Plant]

There are many cool sites that are even specific to each phone. They should be more informative too after the run of competitive market.

[swimnumbers]

Do you have studies showing results DIFFERENT than the ones in question? If not, then all we've done is questioned published results, without an alternative hypothesis.

[Disheartened]

There is an Edgar Cone Wikipedia page that cites this blog as an authentic source for Dale's Cone misleading data interpretations. Neither cite is authenticated and was first created in 2007 and last updated March 14, 2011.

Did Will Thalheimer create a Wikipedia page and cite himself as the credible source?

If this counts as an authentic source, then the PPT presentation from 2002 with the same data should have been credited, not this blog.

(www2.potsdam.edu/betrusak/AECT2002/dalescone.../dalescone.html.ppt)

http://en.wikipedia.org/wiki/Edgar_Dale

[cheap million dollar quartet tickets]

It is about as comical as all those Bush as Hitler signs and posters from a couple of years ago. We all know what happened to Hitler don't we.

[leather jackets]

I really appreciate every single piece of work that you publish and share with the big, wide world.t is always refreshing to read posts of other bloggers and learn something from them.Great!!!

[yankees tickets]

There is an Edgar Cone Wikipedia page that cites this blog as an authentic source for Dale's Cone misleading data interpretations. Neither cite is authenticated and was first created in 2007 and last updated March 14, 2011.

[dual saw]

This is the official information from articles I and my friends need. Thanks again!

[Leather Bags]

Great blog here, such a wealth of content.

[Lingerie]

That blog is so interesting. I must say that at first I thought it would be a waste of time, but after reading your post was impressed with its quality. I just hope that over time you continue maintaining the same quality, and most of the same passion at the time of writing.

[Motorcycle Boots]

I am New Here,My name is Jesse and this is my first entry here. feel sometimes, that I have to learn a lot, hope you guys have patient on me.Thank you for sharing the post.

[Stun Guns]

This is one of the best blogs I’ve ever read. I m absolutely excited to get to read such a well blog. The amount of data that I get is truly great. This is an good masterpiece. I am truly impressed. I would love to read more of your blogs. Please sustain posting. Keep up the good work.

[halloween fancy dress costumes]

This blog Is very informative , I am really pleased to post my comment on this blog . It helped me with ocean of knowledge so I really belive you will do much better in the future . Good job web master .

[Buy Essay online]

I completely agree with you. I have no point to raise in against of what you have said I think you explain the whole situation very well

[dr who jackets]

A very good looking site and very informative too.

[Natanie]

Thank you for the debunk!

I searched the internet and found Dale's cone. The one with percentage,that is. Something's wrong with the graph ie. it quoted percentage of retention but didn't specify the duration, that is why i jumped to wiki and found the page that link to here.

[עבודה לצעירים]

The reverse is also meaningless. People who learn the language in the light on the written page can not be fairly assessed by asking them to say words.