Will at Work Learning

How Much Do People Forget?

This blog post is excerpted from the full report, How Much Do People Forget? Click here to download the full report. You may also access the report---and many other reports---by going to my catalog page by clicking here.

Everybody Wants to Know—How Much Do People Forget?

For years, people have been asking me, “How much do people forget?” and I’ve told them, “It depends.” When I make this statement, most people scowl at me and walk away frustrated and unrequited. I also suspect that some of them think less of me—perhaps that I am just hiding my ignorance.

But I try. I try to explain the complexity of human learning. I explain that forgetting depends on many things, for example:

• The type of material that is being learned
• The learners’ prior knowledge
• The learners’ motivation to learn
• The power of the learning methods used
• The contextual cues in the learning and remembering situations
• The amount of time the learning has to be retained
• The difficulty of the retention test
• Etc.

More meaningful materials (like stories) tend to be easier to remember than less meaningful material (like nonsense syllables). More relevant concepts tend to be easier to remember than less relevant concepts. Learners who have more prior knowledge in a topic area are likely to be better able to remember new concepts learned in that area. More motivated learners are more likely to remember than less motivated learners. Learners who receive repetitions, retrieval practice, feedback, variety (and other potent learning methods) are more likely to remember than learners who do not receive such learning supports. Learners who are provided with learning and practice in the situations where they will be asked to remember the information will be better able to remember. Learners who are asked to retrieve information shortly after learning it will retrieve more than learners who are asked to retrieve information a long time after learning it.

I try to explain all this, but still people keep asking.

And then there are the statistics I keep hearing—that are passed around the learning field from person to person through the years as if they were immutable truths carved by Old Moses Ebbinghaus on granite stones. Here is some information so cited (as of December 2010):

• People forget 40% of what they learned in 20 minutes and 77% of what they learned in six days (http://www.festo-didactic.co.uk/gb-en/news/forgetting-curve-its-up-to-you.htm?fbid=Z2IuZW4uNTUwLjE3LjE2LjM0Mzc). 
• People forget 90% after one month. (http://www.reneevations.com/management/ebbinghaus-curve/)
• People forget 50-80% of what they’ve learned after one day and 97-98% after a month. (http://www.adm.uwaterloo.ca/infocs/study/curve.html)

Never mind that these immutable truths conflict with each other.

So, I will try one more time to convince the world that forgetting depends.

To accomplish this, I explored 14 research articles, examining 69 conditions to see how much forgetting occured, representing over 1,000 learners.

The following graph details the amount of forgetting for each of the 69 conditions:

Conclusions

This graph and the indepth analysis in the full article revealed four critical concepts in human learning—truths that every learning professional should deeply understand.

1. The amount a learner will forget varies depending on many things. We as learning professionals will be more effective if we make decisions based on a deep understanding of how to minimize forgetting and enhance remembering.
2. Rules-of-thumb that show people forgetting at some pre-defined rate are just plain false. In other words, learning gurus and earnest bloggers are wrong when they make blanket statements like, “People will forget 40% of what they learned within a day of learning it.”
3. Learning interventions can produce profound improvements in long-term remembering. In other words, learning gurus are wrong when they say that training is not effective.
4. Different learning methods produce widely different amounts of forgetting. We as learning professionals can be more effective if we take a research-based approach and utilize those learning methods that are most effective.

Telling Findings From the Research

1. People in the reviewed experiments forgot from 0% to 94% of what they had learned. The bottom line is that forgetting varies widely.
2. Even within a restricted time range, learners forgot at wildly differing rates. For example, in the 1-2 day range, learners forgot from 0 to 73%. Learners in the 2-8 year range forgot from 16% to 94%. The obvious conclusion here is that learning varies widely (and wildly) and cannot be predetermined (except perhaps by deities, of whom, I think, we have not even a few in the learning field). To be specific, when we hear statements like, “People will forget 60% of what they learned within 7 days,” we should ignore such advice and instead reflect on our own superiority and good looks until we are decidedly pleased with ourselves.
3. Even when we looked at only one type of learning material, forgetting varied widely. For example, in Bahrick’s classic 1979 experiment where learners were learning English-Spanish word pairs, learners forgot from 12% to 63%. Even more remarkably, if we include those cases where learners actually remembered more on the second test than the first test, learners’ “forgetting” varied from -41% to 63%, a swing of 104 percentage points! Again, we must conclude that forgetting varies widely.
4. Many of the experiments reviewed in this report showed clearly that learning methods matter. For example, in the Bahrick 1979 study, the best learning methods produced an average forgetting score of -29% forgetting, whereas the worst learning methods produced forgetting at 47%, a swing of 76% points. In Runquist’s 1983 study, the best learning method produced average forgetting at 34%, whereas all the other learning methods produced average forgetting of 78%. In Allen, Mahler, and Estes’ 1969 experiment, the learners given the best learning methods forgot an average of 2.3%, whereas the learners who got middling learning methods forgot an average of 14.3%, and learners given the worst learning methods forgot approximately 21.7%. The bottom line is that the learning methods we choose make all the difference!!

Check out the full report to learn more about the following:

• What you should do as a learning professional (in light of these findings).
• Whether the learning-curve notion still applies.
• What wisdom each of the 14 research articles revealed.
• The methodology used in the research.
• The calculation of forgetting.

 

Cognitive Load Theory Coming Under Withering Attacks

First I must explain that there is a difference between empirical research findings and the theoretical formulations that human researchers create to explain their findings. To reiterate, we have:

1. research findings (data)
2. theoretical explanations (rationales that researchers invent)

The data can be true, while the theoretical explanations can be wrong.

Academic researchers are paid the big bucks---and gain the highest psychic rewards---for developing theories.

As you know if you've followed my work for any length of time, I put much more faith in data than in theories. 

So, while I am about to share criticisms of a theory, I think the research findings are still sound.

Here are some recent criticisms of Cognitive Load Theory:

-----------------

Ton de Jong says:

What has cognitive load theory brought to the field of educational design? The three main recommendations that come from cognitive load theory are: present material that aligns with the prior knowledge of the learner (intrinsic load), avoid non-essential and confusing information (extraneous load), and stimulate processes that lead to conceptually rich and deep knowledge (germane load). These design principles have been around in educational design for a long time (see e.g., Dick and Carey 1990; Gagne´ et al. 1988; Reigeluth 1983). Work in cognitive load theory often denies the existence of this earlier research, as illustrated in the following quote by Ayres (2006a, p. 288): ‘‘Whereas strategies to lower extraneous load are well documented…methods to lower intrinsic load have only more recently been investigated’’ (p. 288). In his study, Ayres introduces part-tasks as one of the initial approaches to lower cognitive load. Describing this as a ‘‘recent’’ approach denies much of the history of instructional design.

de Jong, T. (2010). Cognitive load theory, educational research, and instructional design: Some food for thought. Instructional Science, 38(2), 105-134.

-----------------

Roxana Moreno says:

Under the light of CLT’s [Cognitive Load Theory's] fundamental limitations, I will make the argument that continuing to use the theory to frame instructional design research is instilling the idea that educational research cannot aspire to have the same scientific value as that of the hard sciences (Diamond 1987). The following are some reasons why this might be the case. When educational researchers are not able to demonstrate that they are making progress, they give further reasons to believe that the learning sciences are a lesser form of knowledge (Labaree 1998). Second, although a strength of CL research is the use of controlled experimental studies—one of the exemplary methods of scientifically based research (Eisenhart and Towne 2003)—it has failed to develop adequate methods that permit direct investigation of the research questions at stake. Science relies on measurements or observational methods that provide reliable and valid data across studies by the same or different investigators (National Research Council 2002).

Third, in any science, researchers construct towers of knowledge on the foundations of the work of others. de Jong raises a valid concern about the fact that CL research often ignores the existence of earlier research and theories that may better account for the findings than CLT. The dangers of this isolated approach to science are clearly stated by Labaree (1998) ‘‘At the end of long and distinguished careers, senior educational researchers are likely to find that they are still working on the same questions that confronted them at the beginning. And the new generation of researchers they have trained will be taking up these questions as well (p. 9).’’

Lastly, although bias may not be completely avoidable, scientists are expected to be aware of potential bias sources in their work. One safeguard against bias in any area of study is to be open to reflection and scrutiny. It is the professional responsibility of educational researchers to evaluate the state of current knowledge on a regular basis, identify knowledge gaps, and lay the scientific principles for future investigation. Engaging in this ‘effortful’ practice is key in fostering a scientific community and culture.

Moreno, R. (2010). Cognitive load theory: More food for thought. Instructional Science, 38(2), 135-141.

-----------------

Schnotz and Kurschner (2007) say:

Numerous empirical studies have demonstrated that traditional instruction can and should be re-designed according to principles of cognitive load theory, and that this re-design results in better learning. However, there are also numerous conceptual problems related to cognitive load theory, which sometimes make interpretation of empirical findings difficult. Although the concept of cognitive load has been frequently described in general terms and although definitions have been provided for different kinds of cognitive load, a closer look reveals that the exact nature of these different kinds of load is not sufficiently clear yet. Further clarification is needed regarding the relations between different kinds of cognitive load and whether they can and how they should be manipulated to enhance learning. Other open questions refer to the role of working memory in the process of learning. Although working memory is a key concept in cognitive load theory, it is not sufficiently clear to what extent working memory is in fact required for learning. Finally, further clarification is needed whether and in which way different kinds of cognitive load constrain each other, how they relate to the process of learning and, last not least, how they can be measured.

Schnotz, W., & Kurschner, C. (2007). A reconsideration of cognitive load theory. Educational Psychology Review, 19, 469–508.

-----------------

So, the theory is shaky, even though it has generated a slew of great research.

The data is still compelling, so, among other things, we can still use worked examples.

1. Worked examples are useful for novice learners.
2. Worked examples may hurt more experienced learners. Better to utilize practice problems for more experienced learners.

Are you being fooled by claims of brain-based learning?

What happens when people add scientific-sounding words to their arguments--even if the scientific-sounding words have nothing to do with those arguments?

Research now has the answer. Here's the research abstract from a recent publication:

Explanations of psychological phenomena seem to generate more public interest when they contain neuroscientific information. Even irrelevant neuroscience information in an explanation of a psychological phenomenon may interfere with people's abilities to critically consider the underlying logic of this explanation.

We tested this hypothesis by giving naïve adults, students in a neuroscience course, and neuroscience experts brief descriptions of psychological phenomena followed by one of four types of explanation, according to a 2 (good explanation vs. bad explanation) × 2 (without neuroscience vs. with neuroscience) design.

Crucially, the neuroscience information was irrelevant to the logic of the explanation, as confirmed by the expert subjects.

Subjects in all three groups judged good explanations as more satisfying than bad ones. But subjects in the two nonexpert groups additionally judged that explanations with logically irrelevant neuroscience information were more satisfying than explanations without.

The neuroscience information had a particularly striking effect on nonexperts' judgments of bad explanations, masking otherwise salient problems in these explanations.

So, if you're swayed by "brain-based learning" and other such scientifically-sounding arguments, you might ask yourself, was the argument really persuasive or was I just fooled into thinking it was.

Weisberg, Deena Skolnick; Keil, Frank C.; Goodstein, Joshua; Rawson, Elizabeth; Gray, Jeremy R. (2008). The seductive allure of neuroscience explanations. Weisberg, D. S., Keil, F. C., Goodstein, J., Rawson, E., & Gray, J. R. (2008). The seductive allure of neuroscience explanations. Journal of Cognitive Neuroscience, 20(3), 470-477.

The Displacement Hypothesis -- a tangentially-related research study

The Displacement Hypothesis says that one activity can displace another activity.

In the following research by Robert Weis and Brittany Cerankosky, boys given video games (1) did worse in school, (2) spent less time in other after-school activities, (3) had more behavioral problems, and (4) had lower reading and writing scores.

Abstract of the Research Article:

Young boys who did not own video games were promised a video-game system and child-appropriate games in exchange for participating in an “ongoing study of child development.” After baseline assessment of boys’ academic achievement and parent- and teacher-reported behavior, boys were randomly assigned to receive the video-game system immediately or to receive the video-game system after follow-up assessment, 4 months later. Boys who received the system immediately spent more time playing video games and less time engaged in after-school academic activities than comparison children. Boys who received the system immediately also had lower reading and writing scores and greater teacher-reported academic problems at follow-up than comparison children. Amount of video-game play mediated the relationship between video-game ownership and academic outcomes. Results provide experimental evidence that video games may displace after-school activities that have educational value and may interfere with the development of reading and writing skills in some children.

Analogs in Adult Learning

Are there analogs in adult learning? In a quick database review I couldn't uncover any research on the displacement hypothesis with adults, but here are some learning events that may displace other learning events:

1. Twitter
2. Social Networking
3. Web Surfing
4. Blogs
5. RSS following
6. Gaming

Postscript:

When I asked the lead author if he knew of any studies on adults regarding the displacement hypothesis, he said "no," but he pointed me to this article on college students.

Learning Styles Reviewed by Association for Psychological Science AND FOUND WANTING.

The Association of Psychological Science commissioned a review of the evidence for the benefits of using learning styles, and the report is clear.

We conclude therefore, that at present, there is no adequate evidence base to justify incorporating learning-styles assessments into general educational practice. Thus, limited education resources would better be devoted to adopting other educational practices that have a strong evidence base, of which there are an increasing number. However, given the lack of methodologically sound studies of learning styles, it would be an error to conclude that all possible versions of learning styles have been tested and found wanting; many have simply not been tested at all. (p. 105)

Research Citation:

Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008). Learning styles: Concepts and evidence. Psychological Science in the Public Interest, 9, 105-119.

You can access the article by clicking here.

You can access Richard Mayer's nice intro to the article---which stresses the benefits of research---by clicking here.

My $1,000 Learning Styles Challenge

Three and one half years ago I offered $1,000 to any person or group who could demonstrate the benefits of learning styles in a real-world practical training program. No one has collected the money yet.

Here was the challenge:

Can an e-learning program that utilizes learning-style information outperform an e-learning program that doesn't utilize such information by 10% or more on a realistic test of learning, even it is allowed to cost up to twice as much to build?

You can access my original Learning Styles Challenge by clicking here. 

You can access my three-year update on the challenge by clicking here.

Final Nail in the Coffin of Learning Styles?

Is this excellent research review by some of the most highly-respected researchers in the learning-research field a final nail in the coffin of learning styles?

Well, as a researcher I must always maintain openness to new information. Perhaps someday more research will demonstrate some specific benefits to learning styles. As the authors of the review say themselves:

Although we have argued that the extant data do not provide support for the learning-styles hypothesis, it should be emphasized that we do not claim that the same kind of instruction is most useful in all contexts and with all learners. An obvious point is that the optimal instructional method is likely to vary across disciplines. For instance, the optimal curriculum for a writing course probably includes a heavy verbal emphasis, whereas the most efficient and effective method of teaching geometry obviously requires visual–spatial materials. Of course, identifying the optimal approach for each discipline is an empirical question, and we espouse research using strong research methods to identify the optimal approach for each kind of subject matter.

Furthermore, it is undoubtedly the case that a particular student will sometimes benefit from having a particular kind of course content presented in one way versus another. One suspects that educators’ attraction to the idea of learning styles partly reflects their (correctly) noticing how often one student may achieve enlightenment from an approach that seems useless for another student. There is, however, a great gap from such heterogeneous responses to instructional manipulations—whose reality we do not dispute—to the notion that presently available taxonomies of student types offer any valid help in deciding what kind of instruction to offer each individual. Perhaps future research may demonstrate such linkages, but at present, we find no evidence for it. (p. 116)

As a consultant in the workplace learning-and-performance field, I will likely do my clients harm if I advised for the use of a learning-style learning design. I will continue to advise clients against designing their learning based on learning styles. At the same time, I will encourage them to be watchful for specific learning needs of individual learners. For example, when a learner is confused, he or she probably needs feedback and guidance.

I recommend that you read the article and Mayer's introduction. Both provide wisdom about how to think about research and how to avoid being fooled.

Article Note: The date in the article and on the database PsycINFO says the article is from 2008. However, the copyright is from 2009 and the article includes citations from 2009 and the article appears as the "current article" on the APS (Association for Psychological Science) website, and news reports just started surfacing in December 2009 and January 2010. The evidence suggests the article just recently came out.

Training to Prevent Back Injuries. Is it Effective?

A review of research on training to prevent back injuries offers disappointing results.

The authors conclude:

In this systematic review we found no evidence that training with or without lifting equipment is effective in the prevention of back pain or consequent disability. Either the advocated techniques did not reduce the risk of back injury or training did not lead to adequate change in lifting and handling techniques. (p. 5)

As the authors say, it could be the design of the training interventions. What I wonder is whether this is a motivational or information issue. Or if the design is just not effective in generating spontaneous remembering in the workplace.

Research Citation:

Martimo, K., Verbeek, J., Karppinen, J., Furlan, A. D., Takala, E., & Kuijer, P. P. F. M. (2008). Effect of training and lifting equipment for preventing back pain in lifting and handling: Systematic review. BMJ: British Medical Journal, 336. 1-6. doi: 10.1136/bmj.39463.418380.BE

Article may be available by clicking here.

Deadly Fire Truck Accident Blamed on Poor Training. What We Can Learn From the Tragedy.

It is not always enough to know something. Often people must respond immediately to circumstances. Often they must respond under stress and distraction. To be able to do this, they need to develop a cognitive link between situational cues and action.

On January 9^th 2009, firefighter Robert O’Neill was at the wheel of a 22-ton fire truck as it headed down a steep street in Boston. When O’Neill attempted to apply the brakes, nothing happened. As the truck gained speed heading straight for a large brick wall, O’Neill made frantic attempts to pump the brakes and shift into another gear—ultimately shifting into neutral. His efforts went to no avail and the truck crashed through the brick wall into an apartment building injuring several children in an afterschool program and killing his colleague, firefighter Lieutenant Kevin M. Kelley who was riding beside O’Neill in the truck’s passenger seat.

Photo from Boston Globe Story

After an 11-month investigation, District Attorney Daniel Conley reported that O’Neill received “limited classroom instruction and no driver training in the proper use of air brakes in downhill and emergency circumstances.” What’s really tragic is that O’Neill’s actions in that time of panic may have actually made things worse. As the Boston Globe reported:

The driver did not know to check brake pressure before he got behind the wheel that day and then, when the brakes failed, did not know how to engage secondary braking systems. Instead, he pumped the brakes, releasing any remaining air pressure from the brake system, and put the truck into neutral, preventing the secondary brakes from engaging.

Many things might have prevented this tragedy. The truck could have been better maintained. A job aid that forced drivers to do routine safety check might have been used to ensure brake pressure. Management oversight might have prompted the drivers to actually engage safety routines. Training that helped drivers understand how air brakes worked might have helped—just before the accident the driver turned the truck around in a parking lot releasing air pressure in the braking system. Finally, the driver could have been trained to spontaneously remember what do when facing such a situation.

Ideally, when someone is in an emergency situation, the cues from that situation ought to remind them of what to do. As learning professionals we want to help our learners engage in spontaneous remembering. To do this, we need to help our learners make links between situational cues and actions. It can help to teach these links, but it is even better to have learners practice these links.

Research on general context-alignment effects shows the benefits of making such links (for reviews see Bjork & Richardson-Klavehn, 1989; Smith, 1988; Smith & Vela, 2001; Eich, 1980; Roediger & Guynn, 1996; Davies, 1986). Research on retrieval practice shows us the benefits of retrieval practice in automating such responding (for reviews on retrieval practice see Roediger & Karpicke, 2006a; Pashler, Rohrer, Cepeda, & Carpenter, 2007; Bjork, 1988; Crooks, 1988). Finally, recent research on implementation intentions shows how powerful it can be to help learners link situational cues to action (for review see Gollwitzer & Sheeran, 2006).

If firefighter O’Neill had been properly trained, when he found himself careening down the street with inadequate brakes, the situation would have reminded him to apply steady pressure on the brakes and engage the secondary braking system.

What makes this situation even sadder is that firefighters typically have lots of time between emergencies to engage in training. Even if a high-fidelity simulation was too expensive, a simple e-learning program that simulated driving emergencies might have worked to create cognitive links sufficient to create spontaneous remembering.

 

Research

Bjork, R. A. (1988). Retrieval practice and the maintenance of knowledge. In M. M. Gruneberg, P. E. Morris, R. N. Sykes (Eds.), Practical Aspects of Memory: Current Research and Issues, Vol. 1., Memory in Everyday Life (pp. 396-401). NY: Wiley.

Bjork, R. A., & Richardson-Klavehn, A. (1989). On the puzzling relationship between environmental context and human memory. In C. Izawa (Ed.) Current Issues in Cognitive Processes: The Tulane Floweree Symposium on Cognition (pp. 313-344). Hillsdale, NJ: Erlbaum.

Crooks, T. J. (1988). The impact of classroom evaluation practices on students. Review of Educational Research, 58, 438-481.

Davies, G. (1986). Context effects in episodic memory: A review. Cahiers de Psychologie Cognitive, 6, 157-174.

Eich, J. E. (1980). The cue dependent nature of state dependent retrieval. Memory and Cognition, 8, 157-173.

Gollwitzer, P. M., & Sheeran, P. (2006). Implementation intentions and goal achievement: A meta-analysis of effects and processes. Advances in Experimental Social Psychology, 38, 69-119.

Pashler, H., Rohrer, D., Cepeda, N. J., & Carpenter, S. K. (2007). Enhancing learning and retarding forgetting: Choices and consequences. Psychonomic Bulletin & Review, 14, 187-193.

Roediger, H. L., III, & Guynn, M. J. (1996). Retrieval processes. In E. L. Bjork & R. A. Bjork (eds.), Memory (pp. 197-236). San Diego, CA: Academic Press.

Roediger, H.L. & Karpicke, J.D. (2006a). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181-210.

Smith, S. M. (1988). Environmental context-dependent memory. In G. M. Davies & D. M. Thomson (eds.) Memory in Context: Context in Memory (pp. 13-34), Chichester, UK: Wiley.

Smith, S. M., & Vela, E. (2001). Environmental context-dependent memory: A review and meta-analysis. Psychonomic Bulletin & Review, 8, 203-220.

Story of the accident: http://www.boston.com/news/local/breaking_news/2009/01/fire_engine_cra.html

Story of the accident investigation:

http://www.boston.com/news/local/massachusetts/articles/2009/12/11/poor_training_cited_in_crash/

Photo from Massachusetts AFL-CIO website

Lieutenant Kevin M. Kelley
(Killed in Fire Truck Accident)

Aging. Can We Enhance People's Cognitive Outcomes?

If you work in the workplace learning-and-performance field, one of your jobs is to ensure that employees are maximizing their cognitive performance, their decision making, and their overall work output. If people's cognitive abilities decreased with age, that would be a problem. More importantly, if we can improve our employee's cognitive abilities, we have a responsibility to do just that. The benefits will accrue to our organizations and to our employees too (and probably then to their families and society at large).

This begs the following questions then:

• "Is there research in refereed scientific journals that provides evidence for cognitive decline as people age?"
• "Is there research in refereed scientific journals that provides evidence that we can help improve people's cognitive abilities as they age?"

I've created a short 4-item quiz for you to test your knowledge in this area. Take the quiz. When you are done it will return you directly to this blog post (is that cool or what)?

Take the Quiz. Test your Knowledge of Aging's Effect on Cognitive Ability.

Click here to take the quiz

The quiz is based on an article by Christopher Hertzog, Arthur F. Kramer, Robert S. Wilson, and Ulman Lindenberger.

HEY, what are you doing? Go take the quiz first. There's research to show that the sort of questions I ask in the quiz will actually help you remember this topic. Doh!

The article by Hertzog, Kramer, Wilson, and Lindenberger is in Volume 9—Number 1 in the refereed scientific journal Psychological Science in the Public Interest that was just published in 2009. The title of the article is: Enrichment Effects on Adult Cognitive Development Can the Functional Capacity of Older Adults Be Preserved and Enhanced?

HEY, really. Go take the quiz first!!

Both of my parents (75+) are doing everything right according to the article.


Findings:

Cognitive ability does tend to decline with age. See graph from the article:



But notice that though AVERAGE cognitive ability declines there are wide ranges. And since I'm 51 years old as I write this, I'd like you to note that maximum cognitive performance seems highest near 50 years of age.

Can Cognitive Ability be Improved?

Yes, these researchers conclude that it can. Although they admit that more research is needed.

What Can Improve Cognitive Ability?

Well, they didn't look at everything that might impact cognitive ability, so we don't have a clear picture yet.

They highlighted the strongest findings in their conclusion:

"The literature is far from definitive, which is no surprise given the inherent difficulties in empirically testing the enrichment hypothesis. However, we believe there is a strong and sound empirical basis for arguing that a variety of factors, including engaging in intellectually and mentally stimulating activities, both (a) slow rates of cognitive aging and (b) enhance levels of cognitive functioning in later life." p. 41

"What is most impressive to us is the evidence demonstrating benefits of aerobic physical exercise on cognitive functioning in older adults. Such a conclusion would have been controversial in the not-too-distant past, but the evidence that has accumulated since 2000 from both human and animal studies argues overwhelmingly that aerobic exercise enhances cognitive function in older adults. The hypothesis of exercise-induced cognitive-enrichment effects is supported by longitudinal studies of predictors of cognitive decline and incidence of dementia, but also by short-term intervention studies in human and animal populations. The exercise-intervention work suggest relatively general cognitive benefits of aerobic exercise but indicates that cognitive tasks that require executive functioning, working memory, and attentional control are most likely to benefit." p. 41

They also noted some other more-tentative findings:

"...these data support the idea that a higher level of social engagement is related to a reduced risk of cognitive decline and dementia in old age. The basis of the association is not well understood, however." p. 33

"...these data suggest that chronic psychological distress may contribute to late-life loss of cognition by causing neurodeteriorative changes in portions of the limbic system that help regulate affect and cognition, changes that do not leave a pathologic footprint (e.g., dendritic atrophy) or whose pathology is not recognizable with currently available methods. These changes, when extreme, might actually be sufficient to cause dementia, but it is more likely that they contribute to cognitive impairment and thereby increase the likelihood that other common age-related neuropathologies are clinically expressed as dementia" p. 36

"...in observational studies that examine more than one lifestyle factor, cognitive activities appear to be the strongest predictor of cognitive change. However, this could be the result of several factors, including the following: (a) Rarely are physical activities characterized in terms of intensity, frequency, and duration; (b) the period across which activities are assessed has been different for cognitive and physical activities; (c) with one exception, activities have been treated as unidimensional in nature. Clearly, these issues require additional consideration in future studies." p. 39

They also offer a word of caution about software programs that are marketed as ways to improve cognitive ability:

"The majority of software programs marketed as enhancing cognition or brain function lack supporting empirical evidence for training and transfer effects. Clearly, there is a need to introduce standards of good practice in this area. Software developers should be urged to report the reliability and validity of the trained tasks, the magnitude of training effects, the scope and maintenance of transfer to untrained tasks, and the population to which effects are likely to generalize. Arriving at thisinformation requires experiments with random assignment to treatment and control groups, and an adequate sample description. Just as the pharmaceutical industry is required to show benefit and provide evidence regarding potential side effects, companies marketing cognitive-enhancement products should be required to provide empirical evidence of product effectiveness." p.48

So, to answer the quiz questions:

1. What happens to most people's cognitive abilities as they age from 50 years onward? Answer: Declines with age.

2. Is there valid research evidence from scientific refereed journals that suggests that people can improve their cognitive outcomes by engaging in certain activities? Answer: Solid evidence, but still some controversy.

3. Which of the following have been shown to improve cognitive ability as people age. Answer: The article didn't cover all the territory, but the strongest evidence is for (1) mentally and intellectually challenging activities and (2) aerobic physical activity.

4. Imagine that you work for a company that consists of a substantial number of workers over the age of 50. If you had a set budget to spend to improve their cognitive functioning, which of the following investments would garner the greatest results? Answer: Well, the research review does NOT compare the differences between (1) mentally challenging activities, (2) aerobic exercise, and (3) social engagement. However, see their overall conclusion below, which suggests that intellectual engagement and physically activity are key.

Their overall conclusion:

"We conclude that, on balance, the available evidence favors the hypothesis that maintaining an intellectually engaged and physically active lifestyle promotes successful cognitive aging." p.1

More research on benefits of exercise:

"Unlike the literature on an active lifestyle, there is already an impressive array of work with humans and animal populations showing that exercise interventions have substantial benefits
for cognitive function, particularly for aspects of fluid intelligence and executive function. Recent neuroscience research on this topic indicates that exercise has substantial effects on brain morphology and function, representing a plausible brain substrate for the observed effects of aerobic exercise and other activities on cognition." p. 1

They cite the potential for training interventions:

"...cognitive-training studies have demonstrated that older adults can improve cognitive functioning when provided with intensive training in strategies that promote thinking and remembering. The early training literature suggested little transfer of function from specifically trained skills to new cognitive tasks; learning was highly specific to the cognitive processes targeted by training. Recently, however, a new generation of studies suggests that providing structured experience in situations demanding executive coordination of skills—such as complex video games, task-switching paradigms, and divided attention tasks—train strategic control over cognition that does show transfer to different task environments. These studies suggest that there is considerable reserve potential in older adults’ cognition that can be enhanced through training." p. 1

But they offer a warning against one-shot interventions:

"There is no magic pill or no one-shot vaccine that inoculates the individual against the possibility of cognitive decline in old age. As noted earlier, participation in intervention programs is unlikely to affect long-term outcomes unless the relevant behaviors are continued over time." p. 47

What do we have to do?

Well, if we take our job seriously, we ought to heed the research. We can improve our fellow employees cognitive abilities as they age, so we ought to figure out how we might support that.

I certainly haven't got this nailed but if your company is interested, I think it would be fascinating to see what we might do.

Eye Movement Studies. Should we be using them for learning design?

Google has a nice blog post out on its use of eye movement research.

I remember getting a tour of Fidelity a few years ago and learning that their eye movement studies on web browsing showed that people were beginning to ignore big dark chunks of graphics because they thought they were advertisements.

My dissertation advisor, Ernie Rothkopf did a classic study (with Billington) in 1979 using eye movement data to test whether learners actually paid more attention (had more and higher-quality eye movements) toward information in the learning material that was targeted by learning objectives than to information that was not so targeted. It turned out that learning objectives worked to boost learning because they prompted learners to pay more attention to the objective-relevant material and less attention to the rest of the information.

See: Rothkopf, E. Z., & Billington, M. J. (1979). Goal-guided learning from text: Inferring a descriptive processing model from inspection times and eye movements. Journal of Educational Psychology, 71, 310-327.

To answer the question I posed above. Yes, more of us should be using eye-movement research to support us as we do e-learning design.

And by the way, as web pages change their strategies to gain our attention, our learners may change their strategies to avoid things deemed irrelevant. Moreover, as our learners see more and more of our company's e-learning, their eyes may learn where to go...In fact, a lot of them already have a well-learned capacity to find the NEXT key through a swarm of bees.

Personality Instruments and Workplace Performance

The learning-and-performance industry is deluged with instruments purported to help people (1) work better in teams, (2) manage more effectively, (3) hire the right people, (4) promote the best people, (5) etcetera. Unfortunately, many of these instruments have validity, reliability, and magnitude-of-effect issues, despite being well-received by respondents and by learning-and-performance professionals. For example, I will note problems with the MBTI Myers-Briggs below.

Such instruments include multi-rater 360-degree instruments, job-skills tests, knowledge tests, and personality inventories. This blog post is related specifically to personality inventories.

Personality instruments include the wildly-popular MBTI Myers-Briggs Type Indicator and the DISC, plus all sorts of other tests indexed with colors, shapes, and other personality dimensions.

The thinking is that people's personalities influence their actions and their actions determine their workplace effectiveness. This makes sense intuitively, but in practice it has not always been easy to show that personality affects behavior. Early excitement about this possibility in the mid 1900's (i.e., 1930 to 1960) gave way to skepticism, only rebounding into favor in the 1990's as new research found evidence that personality tests could be used in relationship to job performance. For a good historic overview see John and Srivastava (1999, link in reference section below).

Recent research has generally found that personality inventories are related to job performance, though the relationships may be modest and not always consistent. Barrick and Mount (1991) did a meta-analysis looking at many aspects of job performance and found personality to be a factor. Zhao and Seibert (2006) found that the Five-Factor Personality types were related to entrepreneurial skills. Clarke and Robertson (2005) found that personality was related to workplace and non-workplace accidents. Barrick, Mount, and Judge (2001) examined 15 different meta-analyses and concluded that personality and performance were linked.

But this research needs to be understood with some perspective. As Hurtz and Donovan (2000) and others have pointed out, the relationship between the five-factor personality inventories and job performance can be somewhat limited. In other words, just because a person scores a certain way doesn't necessarily mean that they will act a certain way; while there is a slight tendency in the predicted direction, it often is only a slight tendency. Hurtz and Donovan worry further that when other indicators are used (e.g., previous job experience, interviews, etc.), personality measures may provide very little additional information. Moreover, they cite the worry that respondents can fake their responses on personality inventories (see also, Birkeland, Manson, Kisamore, Brannick, & Smith, 2006).

It is particularly important to note that personality research is now almost all tied to the "Big-Five" or "Five-Factor" personality taxonomy. This taxonomy measures personality along five distinct scales, including Openness, Conscientiousness, Extraversion, Agreeableness, and Emotional Stability. The “Big-Five” or “Five-Factor” Personality taxonomy has been validated in many scientific studies (Digman, 1990; Hogan, Hogan, & Roberts, 1996) and is the most widely-regarded of the many personality models, especially as it relates to workplace behaviors. For example, Barrick, Mount, and Judge in 2001 looked at 15 meta-analyses that investigated the relationship between the five personality factors and job performance.

Other personality taxonomies have not fared as well. For example, the MBTI (Myers-Briggs) has been widely discredited by researchers. It is considered neither reliable nor valid. For example, see Pittenger's (2005) caution about using the MBTI. The DISC has not been studied enough to be scientifically validated.

Years ago, I used the MBTI in leadership training to make the point that people are different and may bring different skills and needs to the table. While using such a diagnostic seemed helpful in making that point, today I would use other ways to get that message across or use instruments that are scientifically validated.

To Learn More about Five-Factor Model of Personality

http://en.wikipedia.org/wiki/Big_five_personality_traits

http://www.centacs.com/quickstart.htm

http://www.outofservice.com/bigfive/

http://www.uoregon.edu/~sanjay/bigfive.html

http://en.wikipedia.org/wiki/Neuroticism_Extraversion_Openness_Personality_Inventory

To Purchase/Use Instruments based on the Five-Factor Model

Most Trusted Instrument is the NEO: http://www3.parinc.com/products/product.aspx?Productid=NEO-PI-R, http://www.sigmaassessmentsystems.com/assessments/neopir.asp

Shorter Version of the NEO: http://www3.parinc.com/products/product.aspx?Productid=NEO_FFI, http://www.sigmaassessmentsystems.com/assessments/neoffi.asp

The BFI Instrument: http://www.ocf.berkeley.edu/~johnlab/bfi.htm

Public Domain Instrument (the IPAP): http://ipip.ori.org/, but be sure to read their article before using it: http://ipip.ori.org/Goldberg_etal_2006_IPIP_JRP.pdf

Research Citations

Barrick, M. R., & Mount, M. K. (1991). The Big Five personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44, 1-26.

Barrick, M. R., Mount, M. K., & Judge, T. A. (2001). Personality and performance at the beginning of the new millennium: What do we know and where do we go next?. International Journal of Selection and Assessment, 9, 9-30.

Birkeland, S. A., Manson, T. M., Kisamore, J. L., Brannick, M. T., & Smith, M. A. (2006). A Meta-Analytic Investigation of Job Applicant Faking on Personality Measures. International Journal of Selection and Assessment, 14, 317-335.

Clarke, S., & Robertson, I. T. (2005). A meta​-​analytic review of the Big Five personality factors and accident involvement in occupational and non​-​occupational settings. Journal of Occupational and Organizational Psychology, 78(3), 355-376.

Costa, P., & McCrae, R. (1992). NEO-PI-R and NEO-FFI professional manual. Odessa, FL: Psychological Assessment Resources.

Digman, J. M. (1990). Personality structure: Emergence of the five-factor model. Annual Review of Psychology, 41, 417-440.

Hogan, R., Hogan, J., & Roberts, B. W. (1996). Personality measurement and employment decisions: Questions and answers. American Psychologist, 51, 469-477.

John, O. P., & Srivastava, S.  (1999). The Big Five Trait Taxonomy:  History, measurement, and theoretical perspectives.  In L. A. Pervin & O. P. John (Eds.), Handbook of Personality:  Theory and Research (2nd ed., pp. 102-138), New York:  Guilford Press. Available at: http://www.ocf.berkeley.edu/~johnlab/pdfs/john&srivastava,1999.pdf or http://www.uoregon.edu/~sanjay/pubs/bigfive.pdf.

Pittenger, D. J. (2005). Cautionary Comments Regarding the Myers-Briggs Type Indicator. Consulting Psychology Journal: Practice and Research, 57, 210-221.

Zhao, H., & Seibert, S. E. (2006). The Big Five personality dimensions and entrepreneurial status: A meta-analytical review. Journal of Applied Psychology, 91, 259-271.

Some Interesting Articles on Personality and the Workplace (and their abstracts)

Personality and Team Performance: A Meta​-​Analysis.

By Peeters, Miranda A. G.; Van Tuijl, Harrie F. J. M.; Rutte, Christel G.; Reymen, Isabelle M. M. J.

European Journal of Personality. Vol 20(5), Aug 2006, 377-396.

Using a meta-analytical procedure, the relationship between team composition in terms of the Big-Five personality traits (trait elevation and variability) and team performance were researched. The number of teams upon which analyses were performed ranged from 106 to 527. For the total sample, significant effects were found for elevation in agreeableness (p = 0.24) and conscientiousness (p = 0.20), and for variability in agreeableness (p = -0.12) and conscientiousness (p = -0.24). Moderation by type of team was tested for professional teams versus student teams. Moderation results for agreeableness and conscientiousness were in line with the total sample results. However, student and professional teams differed in effects for emotional stability and openness to experience. Based on these results, suggestions for future team composition research are presented.

An examination of the role of personality in work accidents using meta​-​analysis.

By Clarke, Sharon; Roberston, Ivan

Applied Psychology: An International Review. Vol 57(1), Jan 2008, 94-108.

Personality has been studied as a predictor variable in a range of occupational settings. The study reported is based on a systematic search and meta-analysis of the literature, using the "Big Five" personality framework. The results indicated that there was substantial variability in the effect of personality on workplace accidents, with evidence of situational moderators operating in most cases. However, one aspect of personality, low agreeableness, was found to be a valid and generalisable predictor of involvement in work accidents. The implications of the findings for future research are discussed. Although meta-analysis can be used to provide definite estimates of effect sizes, the limitations of such an approach are also considered.

Personality and Transformational and Transactional Leadership: A Meta​-​Analysis.

By Bono, Joyce E.; Judge, Timothy A.

Journal of Applied Psychology. Vol 89(5), Oct 2004, 901-910.

This study was a meta-analysis of the relationship between personality and ratings of transformational and transactional leadership behaviors. Using the 5-factor model of personality as an organizing framework, the authors accumulated 384 correlations from 26 independent studies. Personality traits were related to 3 dimensions of transformational leadership--idealized influence-inspirational motivation (charisma), intellectual stimulation, and individualized consideration--and 3 dimensions of transactional leadership--contingent reward, management by exception-active, and passive leadership. Extraversion was the strongest and most consistent correlate of transformational leadership. Although results provided some support for the dispositional basis of transformational leadership--especially with respect to the charisma dimension--generally, weak associations suggested the importance of future research to focus on both narrower personality traits and nondispositional determinants of transformational and transactional leadership.

The Big Five personality dimensions and entrepreneurial status: A meta​-​analytical review.

By Zhao, Hao; Seibert, Scott E.

Journal of Applied Psychology. Vol 91(2), Mar 2006, 259-271.

In this study, the authors used meta-analytical techniques to examine the relationship between personality and entrepreneurial status. Personality variables used in previous studies were categorized according to the five-factor model of personality. Results indicate significant differences between entrepreneurs and managers on 4 personality dimensions such that entrepreneurs scored higher on Conscientiousness and Openness to Experience and lower on Neuroticism and Agreeableness. No difference was found for Extraversion. Effect sizes for each personality dimension were small, although the multivariate relationship for the full set of personality variables was moderate (R = .37). Considerable heterogeneity existed for all of the personality variables except Agreeableness, suggesting that future research should explore possible moderators of the personality-entrepreneurial status relationship.

Predicting job performance using FFM and non​-​FFM personality measures.

By Salgado, Jesús F.

Journal of Occupational and Organizational Psychology. Vol 76(3), Sep 2003, 323-346.

This study compares the criterion validity of the Big Five personality dimensions when assessed using Five-Factor Model (FFM)-based inventories and non-FFM-based inventories. A large database consisting of American as well as European validity studies was meta-analysed. The results showed that for conscientiousness and emotional stability, the FFM-based inventories had greater criterion validity than the non FFM-based inventories. Conscientiousness showed an operational validity of .28 (N=19,460, 90% CV=.07) for FFM-based inventories and .18 (N=5,874, 90% CV=-.04) for non-FFM inventories. Emotional stability showed an operational validity of .16 (N=10,786, 90% CV=.04) versus .05 (N=4,54I, 90% CV=-.05) for FFM and non-FFM-based inventories, respectively. No relevant differences emerged for extraversion, openness, and agreeableness. From a practical point of view, these findings suggest that practitioners should use inventories based on the FFM in order to make personnel selection decisions.

A Meta​-​Analytic Investigation of Job Applicant Faking on Personality Measures.

By Birkeland, Scott A.; Manson, Todd M.; Kisamore, Jennifer L.; Brannick, Michael T.; Smith, Mark A.

International Journal of Selection and Assessment. Vol 14(4), Dec 2006, 317-335.

This study investigates the extent to which job applicants fake their responses on personality tests. Thirty-three studies that compared job applicant and non-applicant personality scale scores were meta-analyzed. Across all job types, applicants scored significantly higher than non-applicants on extraversion (d = .11), emotional stability (d = .44), conscientiousness (d = .45), and openness (d = .13). For certain jobs (e.g., sales), however, the rank ordering of mean differences changed substantially suggesting that job applicants distort responses on personality dimensions that are viewed as particularly job relevant. Smaller mean differences were found in this study than those reported by Viswesvaran and Ones (Educational and Psychological Measurement, 59(2), 197-210), who compared scores for induced 'fake-good' vs. honest response conditions. Also, direct Big Five measures produced substantially larger differences than did indirect Big Five measures.

A meta​-​analytic review of the Big Five personality factors and accident involvement in occupational and non​-​occupational settings.

By Clarke, Sharon; Robertson, Ivan T.

Journal of Occupational and Organizational Psychology. Vol 78(3), Sep 2005, 355-376.

Although a number of studies have examined individual personality traits and their influence on accident involvement, consistent evidence of a predictive relationship is lacking due to contradictory findings. The current study reports a meta-analysis of the relationship between accident involvement and the Big Five personality dimensions (extraversion, neuroticism, conscientiousness, agreeableness, and openness). Low conscientiousness and low agreeableness were found to be valid and generalizable predictors of accident involvement, with corrected mean validities of .27 and .26, respectively. The context of the accident acts as a moderator in the personality-accident relationship, with different personality dimensions associated with occupational and non-occupational accidents. Extraversion was found to be a valid and generalizable predictor of traffic accidents, but not occupational accidents. Avenues for further research are highlighted and discussed.

Big Five personality predictors of post​-​secondary academic performance.

By O'Connor, Melissa C.; Paunonen, Sampo V.

Personality and Individual Differences. Vol 43(5), Oct 2007, 971-990.

We reviewed the recent empirical literature on the relations between the Big Five personality dimensions and post-secondary academic achievement, and found some consistent results. A meta-analysis showed Conscientiousness, in particular, to be most strongly and consistently associated with academic success. In addition, Openness to Experience was sometimes positively associated with scholastic achievement, whereas Extraversion was sometimes negatively related to the same criterion, although the empirical evidence regarding these latter two dimensions was somewhat mixed. Importantly, the literature indicates that the narrow personality traits or facets presumed to underlie the broad Big Five personality factors are generally stronger predictors of academic performance than are the Big Five personality factors themselves. Furthermore, personality predictors can account for variance in academic performance beyond that accounted for by measures of cognitive ability. A template for future research on this topic is proposed, which aims to improve the prediction of scholastic achievement by overcoming identifiable and easily correctable limitations of past studies.

Gender differences in personality traits across cultures: Robust and surprising findings.

By Costa Jr., Paul; Terracciano, Antonio; McCrae, Robert R.

Journal of Personality and Social Psychology. Vol 81(2), Aug 2001, 322-331.

Secondary analyses of Revised NEO Personality inventory data from 26 cultures (N =23,031) suggest that gender differences are small relative to individual variation within genders; differences are replicated across cultures for both college-age and adult samples, and differences are broadly consistent with gender stereotypes: Women reported themselves to be higher in Neuroticism, Agreeableness, Warmth, and Openness to Feelings, whereas men were higher in Assertiveness and Openness to Ideas. Contrary to predictions from evolutionary theory, the magnitude of gender differences varied across cultures. Contrary to predictions from the social role model, gender differences were most pronounced in European and American cultures in which traditional sex roles are minimized. Possible explanations for this surprising finding are discussed, including the attribution of masculine and feminine behaviors to roles rather than traits in traditional cultures.

Five​-​factor model of personality and job satisfaction: A meta​-​analysis.

By Judge, Timothy A.; Heller, Daniel; Mount, Michael K.

Journal of Applied Psychology. Vol 87(3), Jun 2002, 530-541.

This study reports results of a meta-analysis linking traits from the 5-factor model of personality to overall job satisfaction. Using the model as an organizing framework, 334 correlations from 163 independent samples were classified according to the model. The estimated true score correlations with job satisfaction were -.29 for Neuroticism, .25 for Extraversion, .02 for Openness to Experience, .17 for Agreeableness, and .26 for Conscientiousness. Results further indicated that only the relations of Neuroticism and Extraversion with job satisfaction generalized across studies. As a set, the Big Five traits had a multiple correlation of .41 with job satisfaction, indicating support for the validity of the dispositional source of job satisfaction when traits are organized according to the 5-factor model.

Relationship of personality to performance motivation: A meta​-​analytic review.

By Judge, Timothy A.; Ilies, Remus

Journal of Applied Psychology. Vol 87(4), Aug 2002, 797-807.

This article provides a meta-analysis of the relationship between the 5-factor model of personality and 3 central theories of performance motivation (goal-setting, expectancy, and self-efficacy motivation). The quantitative review includes 150 correlations from 65 studies. Traits were organized according to the 5-factor model of personality. Results indicated that Neuroticism (average validity=-.31) and Conscientiousness (average validity=.24) were the strongest and most consistent correlates of performance motivation across the 3 theoretical perspectives. Results further indicated that the validity of 3 of the Big Five traits--Neuroticism, Extraversion, and Conscientiousness--generalized across studies. As a set, the Big 5 traits had an average multiple correlation of .49 with the motivational criteria, suggesting that the Big 5 traits are an important source of performance motivation.

Temperament and personality in dogs (Canis familiaris): A review and evaluation of past research.

By Jones, Amanda C.; Gosling, Samuel D.

Applied Animal Behaviour Science. Vol 95(1-2), Nov 2005, 1-53.

Spurred by theoretical and applied goals, the study of dog temperament has begun to garner considerable research attention. The researchers studying temperament in dogs come from varied backgrounds, bringing with them diverse perspectives, and publishing in a broad range of journals. This paper reviews and evaluates the disparate work on canine temperament. We begin by summarizing general trends in research on canine temperament. To identify specific patterns, we propose several frameworks for organizing the literature based on the methods of assessment, the breeds examined, the purpose of the studies, the age at which the dogs were tested, the breeding and rearing environment, and the sexual status of the dogs. Next, an expert-sorting study shows that the enormous number of temperament traits examined can be usefully classified into seven broad dimensions. Meta-analyses of the findings pertaining to inter-rater agreement, test-retest reliability, internal consistency, and convergent validity generally support the reliability and validity of canine temperament tests but more studies are needed to support these preliminary findings. Studies examining discriminant validity are needed, as preliminary findings on discriminant validity are mixed. We close by drawing 18 conclusions about the field, identifying the major theoretical and empirical questions that remain to be addressed.

Will's Note: I included this last one because it amused me that searching for "personality" one might find a research review on dog personality---and to keep all this research stuff in perspective.