Will at Work Learning

Are you being fooled by claims of brain-based learning?

What happens when people add scientific-sounding words to their arguments--even if the scientific-sounding words have nothing to do with those arguments?

Research now has the answer. Here's the research abstract from a recent publication:

Explanations of psychological phenomena seem to generate more public interest when they contain neuroscientific information. Even irrelevant neuroscience information in an explanation of a psychological phenomenon may interfere with people's abilities to critically consider the underlying logic of this explanation.

We tested this hypothesis by giving naïve adults, students in a neuroscience course, and neuroscience experts brief descriptions of psychological phenomena followed by one of four types of explanation, according to a 2 (good explanation vs. bad explanation) × 2 (without neuroscience vs. with neuroscience) design.

Crucially, the neuroscience information was irrelevant to the logic of the explanation, as confirmed by the expert subjects.

Subjects in all three groups judged good explanations as more satisfying than bad ones. But subjects in the two nonexpert groups additionally judged that explanations with logically irrelevant neuroscience information were more satisfying than explanations without.

The neuroscience information had a particularly striking effect on nonexperts' judgments of bad explanations, masking otherwise salient problems in these explanations.

So, if you're swayed by "brain-based learning" and other such scientifically-sounding arguments, you might ask yourself, was the argument really persuasive or was I just fooled into thinking it was.

Weisberg, Deena Skolnick; Keil, Frank C.; Goodstein, Joshua; Rawson, Elizabeth; Gray, Jeremy R. (2008). The seductive allure of neuroscience explanations. Weisberg, D. S., Keil, F. C., Goodstein, J., Rawson, E., & Gray, J. R. (2008). The seductive allure of neuroscience explanations. Journal of Cognitive Neuroscience, 20(3), 470-477.

The Displacement Hypothesis -- a tangentially-related research study

The Displacement Hypothesis says that one activity can displace another activity.

In the following research by Robert Weis and Brittany Cerankosky, boys given video games (1) did worse in school, (2) spent less time in other after-school activities, (3) had more behavioral problems, and (4) had lower reading and writing scores.

Abstract of the Research Article:

Young boys who did not own video games were promised a video-game system and child-appropriate games in exchange for participating in an “ongoing study of child development.” After baseline assessment of boys’ academic achievement and parent- and teacher-reported behavior, boys were randomly assigned to receive the video-game system immediately or to receive the video-game system after follow-up assessment, 4 months later. Boys who received the system immediately spent more time playing video games and less time engaged in after-school academic activities than comparison children. Boys who received the system immediately also had lower reading and writing scores and greater teacher-reported academic problems at follow-up than comparison children. Amount of video-game play mediated the relationship between video-game ownership and academic outcomes. Results provide experimental evidence that video games may displace after-school activities that have educational value and may interfere with the development of reading and writing skills in some children.

Analogs in Adult Learning

Are there analogs in adult learning? In a quick database review I couldn't uncover any research on the displacement hypothesis with adults, but here are some learning events that may displace other learning events:

1. Twitter
2. Social Networking
3. Web Surfing
4. Blogs
5. RSS following
6. Gaming

Postscript:

When I asked the lead author if he knew of any studies on adults regarding the displacement hypothesis, he said "no," but he pointed me to this article on college students.

Learning Styles Reviewed by Association for Psychological Science AND FOUND WANTING.

The Association of Psychological Science commissioned a review of the evidence for the benefits of using learning styles, and the report is clear.

We conclude therefore, that at present, there is no adequate evidence base to justify incorporating learning-styles assessments into general educational practice. Thus, limited education resources would better be devoted to adopting other educational practices that have a strong evidence base, of which there are an increasing number. However, given the lack of methodologically sound studies of learning styles, it would be an error to conclude that all possible versions of learning styles have been tested and found wanting; many have simply not been tested at all. (p. 105)

Research Citation:

Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008). Learning styles: Concepts and evidence. Psychological Science in the Public Interest, 9, 105-119.

You can access the article by clicking here.

You can access Richard Mayer's nice intro to the article---which stresses the benefits of research---by clicking here.

My $1,000 Learning Styles Challenge

Three and one half years ago I offered $1,000 to any person or group who could demonstrate the benefits of learning styles in a real-world practical training program. No one has collected the money yet.

Here was the challenge:

Can an e-learning program that utilizes learning-style information outperform an e-learning program that doesn't utilize such information by 10% or more on a realistic test of learning, even it is allowed to cost up to twice as much to build?

You can access my original Learning Styles Challenge by clicking here. 

You can access my three-year update on the challenge by clicking here.

Final Nail in the Coffin of Learning Styles?

Is this excellent research review by some of the most highly-respected researchers in the learning-research field a final nail in the coffin of learning styles?

Well, as a researcher I must always maintain openness to new information. Perhaps someday more research will demonstrate some specific benefits to learning styles. As the authors of the review say themselves:

Although we have argued that the extant data do not provide support for the learning-styles hypothesis, it should be emphasized that we do not claim that the same kind of instruction is most useful in all contexts and with all learners. An obvious point is that the optimal instructional method is likely to vary across disciplines. For instance, the optimal curriculum for a writing course probably includes a heavy verbal emphasis, whereas the most efficient and effective method of teaching geometry obviously requires visual–spatial materials. Of course, identifying the optimal approach for each discipline is an empirical question, and we espouse research using strong research methods to identify the optimal approach for each kind of subject matter.

Furthermore, it is undoubtedly the case that a particular student will sometimes benefit from having a particular kind of course content presented in one way versus another. One suspects that educators’ attraction to the idea of learning styles partly reflects their (correctly) noticing how often one student may achieve enlightenment from an approach that seems useless for another student. There is, however, a great gap from such heterogeneous responses to instructional manipulations—whose reality we do not dispute—to the notion that presently available taxonomies of student types offer any valid help in deciding what kind of instruction to offer each individual. Perhaps future research may demonstrate such linkages, but at present, we find no evidence for it. (p. 116)

As a consultant in the workplace learning-and-performance field, I will likely do my clients harm if I advised for the use of a learning-style learning design. I will continue to advise clients against designing their learning based on learning styles. At the same time, I will encourage them to be watchful for specific learning needs of individual learners. For example, when a learner is confused, he or she probably needs feedback and guidance.

I recommend that you read the article and Mayer's introduction. Both provide wisdom about how to think about research and how to avoid being fooled.

Article Note: The date in the article and on the database PsycINFO says the article is from 2008. However, the copyright is from 2009 and the article includes citations from 2009 and the article appears as the "current article" on the APS (Association for Psychological Science) website, and news reports just started surfacing in December 2009 and January 2010. The evidence suggests the article just recently came out.

Training to Prevent Back Injuries. Is it Effective?

A review of research on training to prevent back injuries offers disappointing results.

The authors conclude:

In this systematic review we found no evidence that training with or without lifting equipment is effective in the prevention of back pain or consequent disability. Either the advocated techniques did not reduce the risk of back injury or training did not lead to adequate change in lifting and handling techniques. (p. 5)

As the authors say, it could be the design of the training interventions. What I wonder is whether this is a motivational or information issue. Or if the design is just not effective in generating spontaneous remembering in the workplace.

Research Citation:

Martimo, K., Verbeek, J., Karppinen, J., Furlan, A. D., Takala, E., & Kuijer, P. P. F. M. (2008). Effect of training and lifting equipment for preventing back pain in lifting and handling: Systematic review. BMJ: British Medical Journal, 336. 1-6. doi: 10.1136/bmj.39463.418380.BE

Article may be available by clicking here.

Deadly Fire Truck Accident Blamed on Poor Training. What We Can Learn From the Tragedy.

It is not always enough to know something. Often people must respond immediately to circumstances. Often they must respond under stress and distraction. To be able to do this, they need to develop a cognitive link between situational cues and action.

On January 9^th 2009, firefighter Robert O’Neill was at the wheel of a 22-ton fire truck as it headed down a steep street in Boston. When O’Neill attempted to apply the brakes, nothing happened. As the truck gained speed heading straight for a large brick wall, O’Neill made frantic attempts to pump the brakes and shift into another gear—ultimately shifting into neutral. His efforts went to no avail and the truck crashed through the brick wall into an apartment building injuring several children in an afterschool program and killing his colleague, firefighter Lieutenant Kevin M. Kelley who was riding beside O’Neill in the truck’s passenger seat.

Photo from Boston Globe Story

After an 11-month investigation, District Attorney Daniel Conley reported that O’Neill received “limited classroom instruction and no driver training in the proper use of air brakes in downhill and emergency circumstances.” What’s really tragic is that O’Neill’s actions in that time of panic may have actually made things worse. As the Boston Globe reported:

The driver did not know to check brake pressure before he got behind the wheel that day and then, when the brakes failed, did not know how to engage secondary braking systems. Instead, he pumped the brakes, releasing any remaining air pressure from the brake system, and put the truck into neutral, preventing the secondary brakes from engaging.

Many things might have prevented this tragedy. The truck could have been better maintained. A job aid that forced drivers to do routine safety check might have been used to ensure brake pressure. Management oversight might have prompted the drivers to actually engage safety routines. Training that helped drivers understand how air brakes worked might have helped—just before the accident the driver turned the truck around in a parking lot releasing air pressure in the braking system. Finally, the driver could have been trained to spontaneously remember what do when facing such a situation.

Ideally, when someone is in an emergency situation, the cues from that situation ought to remind them of what to do. As learning professionals we want to help our learners engage in spontaneous remembering. To do this, we need to help our learners make links between situational cues and actions. It can help to teach these links, but it is even better to have learners practice these links.

Research on general context-alignment effects shows the benefits of making such links (for reviews see Bjork & Richardson-Klavehn, 1989; Smith, 1988; Smith & Vela, 2001; Eich, 1980; Roediger & Guynn, 1996; Davies, 1986). Research on retrieval practice shows us the benefits of retrieval practice in automating such responding (for reviews on retrieval practice see Roediger & Karpicke, 2006a; Pashler, Rohrer, Cepeda, & Carpenter, 2007; Bjork, 1988; Crooks, 1988). Finally, recent research on implementation intentions shows how powerful it can be to help learners link situational cues to action (for review see Gollwitzer & Sheeran, 2006).

If firefighter O’Neill had been properly trained, when he found himself careening down the street with inadequate brakes, the situation would have reminded him to apply steady pressure on the brakes and engage the secondary braking system.

What makes this situation even sadder is that firefighters typically have lots of time between emergencies to engage in training. Even if a high-fidelity simulation was too expensive, a simple e-learning program that simulated driving emergencies might have worked to create cognitive links sufficient to create spontaneous remembering.

 

Research

Bjork, R. A. (1988). Retrieval practice and the maintenance of knowledge. In M. M. Gruneberg, P. E. Morris, R. N. Sykes (Eds.), Practical Aspects of Memory: Current Research and Issues, Vol. 1., Memory in Everyday Life (pp. 396-401). NY: Wiley.

Bjork, R. A., & Richardson-Klavehn, A. (1989). On the puzzling relationship between environmental context and human memory. In C. Izawa (Ed.) Current Issues in Cognitive Processes: The Tulane Floweree Symposium on Cognition (pp. 313-344). Hillsdale, NJ: Erlbaum.

Crooks, T. J. (1988). The impact of classroom evaluation practices on students. Review of Educational Research, 58, 438-481.

Davies, G. (1986). Context effects in episodic memory: A review. Cahiers de Psychologie Cognitive, 6, 157-174.

Eich, J. E. (1980). The cue dependent nature of state dependent retrieval. Memory and Cognition, 8, 157-173.

Gollwitzer, P. M., & Sheeran, P. (2006). Implementation intentions and goal achievement: A meta-analysis of effects and processes. Advances in Experimental Social Psychology, 38, 69-119.

Pashler, H., Rohrer, D., Cepeda, N. J., & Carpenter, S. K. (2007). Enhancing learning and retarding forgetting: Choices and consequences. Psychonomic Bulletin & Review, 14, 187-193.

Roediger, H. L., III, & Guynn, M. J. (1996). Retrieval processes. In E. L. Bjork & R. A. Bjork (eds.), Memory (pp. 197-236). San Diego, CA: Academic Press.

Roediger, H.L. & Karpicke, J.D. (2006a). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181-210.

Smith, S. M. (1988). Environmental context-dependent memory. In G. M. Davies & D. M. Thomson (eds.) Memory in Context: Context in Memory (pp. 13-34), Chichester, UK: Wiley.

Smith, S. M., & Vela, E. (2001). Environmental context-dependent memory: A review and meta-analysis. Psychonomic Bulletin & Review, 8, 203-220.

Story of the accident: http://www.boston.com/news/local/breaking_news/2009/01/fire_engine_cra.html

Story of the accident investigation:

http://www.boston.com/news/local/massachusetts/articles/2009/12/11/poor_training_cited_in_crash/

Photo from Massachusetts AFL-CIO website

Lieutenant Kevin M. Kelley
(Killed in Fire Truck Accident)

Aging. Can We Enhance People's Cognitive Outcomes?

If you work in the workplace learning-and-performance field, one of your jobs is to ensure that employees are maximizing their cognitive performance, their decision making, and their overall work output. If people's cognitive abilities decreased with age, that would be a problem. More importantly, if we can improve our employee's cognitive abilities, we have a responsibility to do just that. The benefits will accrue to our organizations and to our employees too (and probably then to their families and society at large).

This begs the following questions then:

• "Is there research in refereed scientific journals that provides evidence for cognitive decline as people age?"
• "Is there research in refereed scientific journals that provides evidence that we can help improve people's cognitive abilities as they age?"

I've created a short 4-item quiz for you to test your knowledge in this area. Take the quiz. When you are done it will return you directly to this blog post (is that cool or what)?

Take the Quiz. Test your Knowledge of Aging's Effect on Cognitive Ability.

Click here to take the quiz

The quiz is based on an article by Christopher Hertzog, Arthur F. Kramer, Robert S. Wilson, and Ulman Lindenberger.

HEY, what are you doing? Go take the quiz first. There's research to show that the sort of questions I ask in the quiz will actually help you remember this topic. Doh!

The article by Hertzog, Kramer, Wilson, and Lindenberger is in Volume 9—Number 1 in the refereed scientific journal Psychological Science in the Public Interest that was just published in 2009. The title of the article is: Enrichment Effects on Adult Cognitive Development Can the Functional Capacity of Older Adults Be Preserved and Enhanced?

HEY, really. Go take the quiz first!!

Both of my parents (75+) are doing everything right according to the article.


Findings:

Cognitive ability does tend to decline with age. See graph from the article:



But notice that though AVERAGE cognitive ability declines there are wide ranges. And since I'm 51 years old as I write this, I'd like you to note that maximum cognitive performance seems highest near 50 years of age.

Can Cognitive Ability be Improved?

Yes, these researchers conclude that it can. Although they admit that more research is needed.

What Can Improve Cognitive Ability?

Well, they didn't look at everything that might impact cognitive ability, so we don't have a clear picture yet.

They highlighted the strongest findings in their conclusion:

"The literature is far from definitive, which is no surprise given the inherent difficulties in empirically testing the enrichment hypothesis. However, we believe there is a strong and sound empirical basis for arguing that a variety of factors, including engaging in intellectually and mentally stimulating activities, both (a) slow rates of cognitive aging and (b) enhance levels of cognitive functioning in later life." p. 41

"What is most impressive to us is the evidence demonstrating benefits of aerobic physical exercise on cognitive functioning in older adults. Such a conclusion would have been controversial in the not-too-distant past, but the evidence that has accumulated since 2000 from both human and animal studies argues overwhelmingly that aerobic exercise enhances cognitive function in older adults. The hypothesis of exercise-induced cognitive-enrichment effects is supported by longitudinal studies of predictors of cognitive decline and incidence of dementia, but also by short-term intervention studies in human and animal populations. The exercise-intervention work suggest relatively general cognitive benefits of aerobic exercise but indicates that cognitive tasks that require executive functioning, working memory, and attentional control are most likely to benefit." p. 41

They also noted some other more-tentative findings:

"...these data support the idea that a higher level of social engagement is related to a reduced risk of cognitive decline and dementia in old age. The basis of the association is not well understood, however." p. 33

"...these data suggest that chronic psychological distress may contribute to late-life loss of cognition by causing neurodeteriorative changes in portions of the limbic system that help regulate affect and cognition, changes that do not leave a pathologic footprint (e.g., dendritic atrophy) or whose pathology is not recognizable with currently available methods. These changes, when extreme, might actually be sufficient to cause dementia, but it is more likely that they contribute to cognitive impairment and thereby increase the likelihood that other common age-related neuropathologies are clinically expressed as dementia" p. 36

"...in observational studies that examine more than one lifestyle factor, cognitive activities appear to be the strongest predictor of cognitive change. However, this could be the result of several factors, including the following: (a) Rarely are physical activities characterized in terms of intensity, frequency, and duration; (b) the period across which activities are assessed has been different for cognitive and physical activities; (c) with one exception, activities have been treated as unidimensional in nature. Clearly, these issues require additional consideration in future studies." p. 39

They also offer a word of caution about software programs that are marketed as ways to improve cognitive ability:

"The majority of software programs marketed as enhancing cognition or brain function lack supporting empirical evidence for training and transfer effects. Clearly, there is a need to introduce standards of good practice in this area. Software developers should be urged to report the reliability and validity of the trained tasks, the magnitude of training effects, the scope and maintenance of transfer to untrained tasks, and the population to which effects are likely to generalize. Arriving at thisinformation requires experiments with random assignment to treatment and control groups, and an adequate sample description. Just as the pharmaceutical industry is required to show benefit and provide evidence regarding potential side effects, companies marketing cognitive-enhancement products should be required to provide empirical evidence of product effectiveness." p.48

So, to answer the quiz questions:

1. What happens to most people's cognitive abilities as they age from 50 years onward? Answer: Declines with age.

2. Is there valid research evidence from scientific refereed journals that suggests that people can improve their cognitive outcomes by engaging in certain activities? Answer: Solid evidence, but still some controversy.

3. Which of the following have been shown to improve cognitive ability as people age. Answer: The article didn't cover all the territory, but the strongest evidence is for (1) mentally and intellectually challenging activities and (2) aerobic physical activity.

4. Imagine that you work for a company that consists of a substantial number of workers over the age of 50. If you had a set budget to spend to improve their cognitive functioning, which of the following investments would garner the greatest results? Answer: Well, the research review does NOT compare the differences between (1) mentally challenging activities, (2) aerobic exercise, and (3) social engagement. However, see their overall conclusion below, which suggests that intellectual engagement and physically activity are key.

Their overall conclusion:

"We conclude that, on balance, the available evidence favors the hypothesis that maintaining an intellectually engaged and physically active lifestyle promotes successful cognitive aging." p.1

More research on benefits of exercise:

"Unlike the literature on an active lifestyle, there is already an impressive array of work with humans and animal populations showing that exercise interventions have substantial benefits
for cognitive function, particularly for aspects of fluid intelligence and executive function. Recent neuroscience research on this topic indicates that exercise has substantial effects on brain morphology and function, representing a plausible brain substrate for the observed effects of aerobic exercise and other activities on cognition." p. 1

They cite the potential for training interventions:

"...cognitive-training studies have demonstrated that older adults can improve cognitive functioning when provided with intensive training in strategies that promote thinking and remembering. The early training literature suggested little transfer of function from specifically trained skills to new cognitive tasks; learning was highly specific to the cognitive processes targeted by training. Recently, however, a new generation of studies suggests that providing structured experience in situations demanding executive coordination of skills—such as complex video games, task-switching paradigms, and divided attention tasks—train strategic control over cognition that does show transfer to different task environments. These studies suggest that there is considerable reserve potential in older adults’ cognition that can be enhanced through training." p. 1

But they offer a warning against one-shot interventions:

"There is no magic pill or no one-shot vaccine that inoculates the individual against the possibility of cognitive decline in old age. As noted earlier, participation in intervention programs is unlikely to affect long-term outcomes unless the relevant behaviors are continued over time." p. 47

What do we have to do?

Well, if we take our job seriously, we ought to heed the research. We can improve our fellow employees cognitive abilities as they age, so we ought to figure out how we might support that.

I certainly haven't got this nailed but if your company is interested, I think it would be fascinating to see what we might do.

Eye Movement Studies. Should we be using them for learning design?

Google has a nice blog post out on its use of eye movement research.

I remember getting a tour of Fidelity a few years ago and learning that their eye movement studies on web browsing showed that people were beginning to ignore big dark chunks of graphics because they thought they were advertisements.

My dissertation advisor, Ernie Rothkopf did a classic study (with Billington) in 1979 using eye movement data to test whether learners actually paid more attention (had more and higher-quality eye movements) toward information in the learning material that was targeted by learning objectives than to information that was not so targeted. It turned out that learning objectives worked to boost learning because they prompted learners to pay more attention to the objective-relevant material and less attention to the rest of the information.

See: Rothkopf, E. Z., & Billington, M. J. (1979). Goal-guided learning from text: Inferring a descriptive processing model from inspection times and eye movements. Journal of Educational Psychology, 71, 310-327.

To answer the question I posed above. Yes, more of us should be using eye-movement research to support us as we do e-learning design.

And by the way, as web pages change their strategies to gain our attention, our learners may change their strategies to avoid things deemed irrelevant. Moreover, as our learners see more and more of our company's e-learning, their eyes may learn where to go...In fact, a lot of them already have a well-learned capacity to find the NEXT key through a swarm of bees.

Personality Instruments and Workplace Performance

The learning-and-performance industry is deluged with instruments purported to help people (1) work better in teams, (2) manage more effectively, (3) hire the right people, (4) promote the best people, (5) etcetera. Unfortunately, many of these instruments have validity, reliability, and magnitude-of-effect issues, despite being well-received by respondents and by learning-and-performance professionals. For example, I will note problems with the MBTI Myers-Briggs below.

Such instruments include multi-rater 360-degree instruments, job-skills tests, knowledge tests, and personality inventories. This blog post is related specifically to personality inventories.

Personality instruments include the wildly-popular MBTI Myers-Briggs Type Indicator and the DISC, plus all sorts of other tests indexed with colors, shapes, and other personality dimensions.

The thinking is that people's personalities influence their actions and their actions determine their workplace effectiveness. This makes sense intuitively, but in practice it has not always been easy to show that personality affects behavior. Early excitement about this possibility in the mid 1900's (i.e., 1930 to 1960) gave way to skepticism, only rebounding into favor in the 1990's as new research found evidence that personality tests could be used in relationship to job performance. For a good historic overview see John and Srivastava (1999, link in reference section below).

Recent research has generally found that personality inventories are related to job performance, though the relationships may be modest and not always consistent. Barrick and Mount (1991) did a meta-analysis looking at many aspects of job performance and found personality to be a factor. Zhao and Seibert (2006) found that the Five-Factor Personality types were related to entrepreneurial skills. Clarke and Robertson (2005) found that personality was related to workplace and non-workplace accidents. Barrick, Mount, and Judge (2001) examined 15 different meta-analyses and concluded that personality and performance were linked.

But this research needs to be understood with some perspective. As Hurtz and Donovan (2000) and others have pointed out, the relationship between the five-factor personality inventories and job performance can be somewhat limited. In other words, just because a person scores a certain way doesn't necessarily mean that they will act a certain way; while there is a slight tendency in the predicted direction, it often is only a slight tendency. Hurtz and Donovan worry further that when other indicators are used (e.g., previous job experience, interviews, etc.), personality measures may provide very little additional information. Moreover, they cite the worry that respondents can fake their responses on personality inventories (see also, Birkeland, Manson, Kisamore, Brannick, & Smith, 2006).

It is particularly important to note that personality research is now almost all tied to the "Big-Five" or "Five-Factor" personality taxonomy. This taxonomy measures personality along five distinct scales, including Openness, Conscientiousness, Extraversion, Agreeableness, and Emotional Stability. The “Big-Five” or “Five-Factor” Personality taxonomy has been validated in many scientific studies (Digman, 1990; Hogan, Hogan, & Roberts, 1996) and is the most widely-regarded of the many personality models, especially as it relates to workplace behaviors. For example, Barrick, Mount, and Judge in 2001 looked at 15 meta-analyses that investigated the relationship between the five personality factors and job performance.

Other personality taxonomies have not fared as well. For example, the MBTI (Myers-Briggs) has been widely discredited by researchers. It is considered neither reliable nor valid. For example, see Pittenger's (2005) caution about using the MBTI. The DISC has not been studied enough to be scientifically validated.

Years ago, I used the MBTI in leadership training to make the point that people are different and may bring different skills and needs to the table. While using such a diagnostic seemed helpful in making that point, today I would use other ways to get that message across or use instruments that are scientifically validated.

To Learn More about Five-Factor Model of Personality

http://en.wikipedia.org/wiki/Big_five_personality_traits

http://www.centacs.com/quickstart.htm

http://www.outofservice.com/bigfive/

http://www.uoregon.edu/~sanjay/bigfive.html

http://en.wikipedia.org/wiki/Neuroticism_Extraversion_Openness_Personality_Inventory

To Purchase/Use Instruments based on the Five-Factor Model

Most Trusted Instrument is the NEO: http://www3.parinc.com/products/product.aspx?Productid=NEO-PI-R, http://www.sigmaassessmentsystems.com/assessments/neopir.asp

Shorter Version of the NEO: http://www3.parinc.com/products/product.aspx?Productid=NEO_FFI, http://www.sigmaassessmentsystems.com/assessments/neoffi.asp

The BFI Instrument: http://www.ocf.berkeley.edu/~johnlab/bfi.htm

Public Domain Instrument (the IPAP): http://ipip.ori.org/, but be sure to read their article before using it: http://ipip.ori.org/Goldberg_etal_2006_IPIP_JRP.pdf

Research Citations

Barrick, M. R., & Mount, M. K. (1991). The Big Five personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44, 1-26.

Barrick, M. R., Mount, M. K., & Judge, T. A. (2001). Personality and performance at the beginning of the new millennium: What do we know and where do we go next?. International Journal of Selection and Assessment, 9, 9-30.

Birkeland, S. A., Manson, T. M., Kisamore, J. L., Brannick, M. T., & Smith, M. A. (2006). A Meta-Analytic Investigation of Job Applicant Faking on Personality Measures. International Journal of Selection and Assessment, 14, 317-335.

Clarke, S., & Robertson, I. T. (2005). A meta​-​analytic review of the Big Five personality factors and accident involvement in occupational and non​-​occupational settings. Journal of Occupational and Organizational Psychology, 78(3), 355-376.

Costa, P., & McCrae, R. (1992). NEO-PI-R and NEO-FFI professional manual. Odessa, FL: Psychological Assessment Resources.

Digman, J. M. (1990). Personality structure: Emergence of the five-factor model. Annual Review of Psychology, 41, 417-440.

Hogan, R., Hogan, J., & Roberts, B. W. (1996). Personality measurement and employment decisions: Questions and answers. American Psychologist, 51, 469-477.

John, O. P., & Srivastava, S.  (1999). The Big Five Trait Taxonomy:  History, measurement, and theoretical perspectives.  In L. A. Pervin & O. P. John (Eds.), Handbook of Personality:  Theory and Research (2nd ed., pp. 102-138), New York:  Guilford Press. Available at: http://www.ocf.berkeley.edu/~johnlab/pdfs/john&srivastava,1999.pdf or http://www.uoregon.edu/~sanjay/pubs/bigfive.pdf.

Pittenger, D. J. (2005). Cautionary Comments Regarding the Myers-Briggs Type Indicator. Consulting Psychology Journal: Practice and Research, 57, 210-221.

Zhao, H., & Seibert, S. E. (2006). The Big Five personality dimensions and entrepreneurial status: A meta-analytical review. Journal of Applied Psychology, 91, 259-271.

Some Interesting Articles on Personality and the Workplace (and their abstracts)

Personality and Team Performance: A Meta​-​Analysis.

By Peeters, Miranda A. G.; Van Tuijl, Harrie F. J. M.; Rutte, Christel G.; Reymen, Isabelle M. M. J.

European Journal of Personality. Vol 20(5), Aug 2006, 377-396.

Using a meta-analytical procedure, the relationship between team composition in terms of the Big-Five personality traits (trait elevation and variability) and team performance were researched. The number of teams upon which analyses were performed ranged from 106 to 527. For the total sample, significant effects were found for elevation in agreeableness (p = 0.24) and conscientiousness (p = 0.20), and for variability in agreeableness (p = -0.12) and conscientiousness (p = -0.24). Moderation by type of team was tested for professional teams versus student teams. Moderation results for agreeableness and conscientiousness were in line with the total sample results. However, student and professional teams differed in effects for emotional stability and openness to experience. Based on these results, suggestions for future team composition research are presented.

An examination of the role of personality in work accidents using meta​-​analysis.

By Clarke, Sharon; Roberston, Ivan

Applied Psychology: An International Review. Vol 57(1), Jan 2008, 94-108.

Personality has been studied as a predictor variable in a range of occupational settings. The study reported is based on a systematic search and meta-analysis of the literature, using the "Big Five" personality framework. The results indicated that there was substantial variability in the effect of personality on workplace accidents, with evidence of situational moderators operating in most cases. However, one aspect of personality, low agreeableness, was found to be a valid and generalisable predictor of involvement in work accidents. The implications of the findings for future research are discussed. Although meta-analysis can be used to provide definite estimates of effect sizes, the limitations of such an approach are also considered.

Personality and Transformational and Transactional Leadership: A Meta​-​Analysis.

By Bono, Joyce E.; Judge, Timothy A.

Journal of Applied Psychology. Vol 89(5), Oct 2004, 901-910.

This study was a meta-analysis of the relationship between personality and ratings of transformational and transactional leadership behaviors. Using the 5-factor model of personality as an organizing framework, the authors accumulated 384 correlations from 26 independent studies. Personality traits were related to 3 dimensions of transformational leadership--idealized influence-inspirational motivation (charisma), intellectual stimulation, and individualized consideration--and 3 dimensions of transactional leadership--contingent reward, management by exception-active, and passive leadership. Extraversion was the strongest and most consistent correlate of transformational leadership. Although results provided some support for the dispositional basis of transformational leadership--especially with respect to the charisma dimension--generally, weak associations suggested the importance of future research to focus on both narrower personality traits and nondispositional determinants of transformational and transactional leadership.

The Big Five personality dimensions and entrepreneurial status: A meta​-​analytical review.

By Zhao, Hao; Seibert, Scott E.

Journal of Applied Psychology. Vol 91(2), Mar 2006, 259-271.

In this study, the authors used meta-analytical techniques to examine the relationship between personality and entrepreneurial status. Personality variables used in previous studies were categorized according to the five-factor model of personality. Results indicate significant differences between entrepreneurs and managers on 4 personality dimensions such that entrepreneurs scored higher on Conscientiousness and Openness to Experience and lower on Neuroticism and Agreeableness. No difference was found for Extraversion. Effect sizes for each personality dimension were small, although the multivariate relationship for the full set of personality variables was moderate (R = .37). Considerable heterogeneity existed for all of the personality variables except Agreeableness, suggesting that future research should explore possible moderators of the personality-entrepreneurial status relationship.

Predicting job performance using FFM and non​-​FFM personality measures.

By Salgado, Jesús F.

Journal of Occupational and Organizational Psychology. Vol 76(3), Sep 2003, 323-346.

This study compares the criterion validity of the Big Five personality dimensions when assessed using Five-Factor Model (FFM)-based inventories and non-FFM-based inventories. A large database consisting of American as well as European validity studies was meta-analysed. The results showed that for conscientiousness and emotional stability, the FFM-based inventories had greater criterion validity than the non FFM-based inventories. Conscientiousness showed an operational validity of .28 (N=19,460, 90% CV=.07) for FFM-based inventories and .18 (N=5,874, 90% CV=-.04) for non-FFM inventories. Emotional stability showed an operational validity of .16 (N=10,786, 90% CV=.04) versus .05 (N=4,54I, 90% CV=-.05) for FFM and non-FFM-based inventories, respectively. No relevant differences emerged for extraversion, openness, and agreeableness. From a practical point of view, these findings suggest that practitioners should use inventories based on the FFM in order to make personnel selection decisions.

A Meta​-​Analytic Investigation of Job Applicant Faking on Personality Measures.

By Birkeland, Scott A.; Manson, Todd M.; Kisamore, Jennifer L.; Brannick, Michael T.; Smith, Mark A.

International Journal of Selection and Assessment. Vol 14(4), Dec 2006, 317-335.

This study investigates the extent to which job applicants fake their responses on personality tests. Thirty-three studies that compared job applicant and non-applicant personality scale scores were meta-analyzed. Across all job types, applicants scored significantly higher than non-applicants on extraversion (d = .11), emotional stability (d = .44), conscientiousness (d = .45), and openness (d = .13). For certain jobs (e.g., sales), however, the rank ordering of mean differences changed substantially suggesting that job applicants distort responses on personality dimensions that are viewed as particularly job relevant. Smaller mean differences were found in this study than those reported by Viswesvaran and Ones (Educational and Psychological Measurement, 59(2), 197-210), who compared scores for induced 'fake-good' vs. honest response conditions. Also, direct Big Five measures produced substantially larger differences than did indirect Big Five measures.

A meta​-​analytic review of the Big Five personality factors and accident involvement in occupational and non​-​occupational settings.

By Clarke, Sharon; Robertson, Ivan T.

Journal of Occupational and Organizational Psychology. Vol 78(3), Sep 2005, 355-376.

Although a number of studies have examined individual personality traits and their influence on accident involvement, consistent evidence of a predictive relationship is lacking due to contradictory findings. The current study reports a meta-analysis of the relationship between accident involvement and the Big Five personality dimensions (extraversion, neuroticism, conscientiousness, agreeableness, and openness). Low conscientiousness and low agreeableness were found to be valid and generalizable predictors of accident involvement, with corrected mean validities of .27 and .26, respectively. The context of the accident acts as a moderator in the personality-accident relationship, with different personality dimensions associated with occupational and non-occupational accidents. Extraversion was found to be a valid and generalizable predictor of traffic accidents, but not occupational accidents. Avenues for further research are highlighted and discussed.

Big Five personality predictors of post​-​secondary academic performance.

By O'Connor, Melissa C.; Paunonen, Sampo V.

Personality and Individual Differences. Vol 43(5), Oct 2007, 971-990.

We reviewed the recent empirical literature on the relations between the Big Five personality dimensions and post-secondary academic achievement, and found some consistent results. A meta-analysis showed Conscientiousness, in particular, to be most strongly and consistently associated with academic success. In addition, Openness to Experience was sometimes positively associated with scholastic achievement, whereas Extraversion was sometimes negatively related to the same criterion, although the empirical evidence regarding these latter two dimensions was somewhat mixed. Importantly, the literature indicates that the narrow personality traits or facets presumed to underlie the broad Big Five personality factors are generally stronger predictors of academic performance than are the Big Five personality factors themselves. Furthermore, personality predictors can account for variance in academic performance beyond that accounted for by measures of cognitive ability. A template for future research on this topic is proposed, which aims to improve the prediction of scholastic achievement by overcoming identifiable and easily correctable limitations of past studies.

Gender differences in personality traits across cultures: Robust and surprising findings.

By Costa Jr., Paul; Terracciano, Antonio; McCrae, Robert R.

Journal of Personality and Social Psychology. Vol 81(2), Aug 2001, 322-331.

Secondary analyses of Revised NEO Personality inventory data from 26 cultures (N =23,031) suggest that gender differences are small relative to individual variation within genders; differences are replicated across cultures for both college-age and adult samples, and differences are broadly consistent with gender stereotypes: Women reported themselves to be higher in Neuroticism, Agreeableness, Warmth, and Openness to Feelings, whereas men were higher in Assertiveness and Openness to Ideas. Contrary to predictions from evolutionary theory, the magnitude of gender differences varied across cultures. Contrary to predictions from the social role model, gender differences were most pronounced in European and American cultures in which traditional sex roles are minimized. Possible explanations for this surprising finding are discussed, including the attribution of masculine and feminine behaviors to roles rather than traits in traditional cultures.

Five​-​factor model of personality and job satisfaction: A meta​-​analysis.

By Judge, Timothy A.; Heller, Daniel; Mount, Michael K.

Journal of Applied Psychology. Vol 87(3), Jun 2002, 530-541.

This study reports results of a meta-analysis linking traits from the 5-factor model of personality to overall job satisfaction. Using the model as an organizing framework, 334 correlations from 163 independent samples were classified according to the model. The estimated true score correlations with job satisfaction were -.29 for Neuroticism, .25 for Extraversion, .02 for Openness to Experience, .17 for Agreeableness, and .26 for Conscientiousness. Results further indicated that only the relations of Neuroticism and Extraversion with job satisfaction generalized across studies. As a set, the Big Five traits had a multiple correlation of .41 with job satisfaction, indicating support for the validity of the dispositional source of job satisfaction when traits are organized according to the 5-factor model.

Relationship of personality to performance motivation: A meta​-​analytic review.

By Judge, Timothy A.; Ilies, Remus

Journal of Applied Psychology. Vol 87(4), Aug 2002, 797-807.

This article provides a meta-analysis of the relationship between the 5-factor model of personality and 3 central theories of performance motivation (goal-setting, expectancy, and self-efficacy motivation). The quantitative review includes 150 correlations from 65 studies. Traits were organized according to the 5-factor model of personality. Results indicated that Neuroticism (average validity=-.31) and Conscientiousness (average validity=.24) were the strongest and most consistent correlates of performance motivation across the 3 theoretical perspectives. Results further indicated that the validity of 3 of the Big Five traits--Neuroticism, Extraversion, and Conscientiousness--generalized across studies. As a set, the Big 5 traits had an average multiple correlation of .49 with the motivational criteria, suggesting that the Big 5 traits are an important source of performance motivation.

Temperament and personality in dogs (Canis familiaris): A review and evaluation of past research.

By Jones, Amanda C.; Gosling, Samuel D.

Applied Animal Behaviour Science. Vol 95(1-2), Nov 2005, 1-53.

Spurred by theoretical and applied goals, the study of dog temperament has begun to garner considerable research attention. The researchers studying temperament in dogs come from varied backgrounds, bringing with them diverse perspectives, and publishing in a broad range of journals. This paper reviews and evaluates the disparate work on canine temperament. We begin by summarizing general trends in research on canine temperament. To identify specific patterns, we propose several frameworks for organizing the literature based on the methods of assessment, the breeds examined, the purpose of the studies, the age at which the dogs were tested, the breeding and rearing environment, and the sexual status of the dogs. Next, an expert-sorting study shows that the enormous number of temperament traits examined can be usefully classified into seven broad dimensions. Meta-analyses of the findings pertaining to inter-rater agreement, test-retest reliability, internal consistency, and convergent validity generally support the reliability and validity of canine temperament tests but more studies are needed to support these preliminary findings. Studies examining discriminant validity are needed, as preliminary findings on discriminant validity are mixed. We close by drawing 18 conclusions about the field, identifying the major theoretical and empirical questions that remain to be addressed.

Will's Note: I included this last one because it amused me that searching for "personality" one might find a research review on dog personality---and to keep all this research stuff in perspective.

FREE Research Report on Feedback

Call me crazy, but I think it's important to invest in the research base for our field. I've spent a good chunk of the last year reviewing research from the world's preeminent refereed journals in regard to how to give learners feedback. I've created, I'd like to think, the seminal research review on how to give learners feedback, written in a way that puts feedback in perspective, that goes deep into the fundamentals to give readers clear mental models for how feedback works. It's this kind of in-depth exploration that allows you as a learning professional to use your wisdom to make the difficult design tradeoffs that you have to make. Recipes are for short-order cooks. Research-based wisdom for learning professionals is much more useful in the gritty day-to-day of our learning shops.

And now, instead of selling this document, I'm going to try an experiment, and give it away. Call me crazy, but internet users (hey that's us) just don't like to pay. I've been swimming upstream against the movement toward free information, knowing that the information I'm compiling is the best information out there, and that it takes an incredibly exhaustive effort to sift through refereed research, make sense of it, and repackage it in a way that resonates and is practical. But maybe research karma will work. It's worth a try, right?

You can help me by reading the report, AND if you think it's good, sending the link to it to everyone you know in your organization, in every learning-development organization you know, to your mom, your kids, your elected officials, to Elliott Masie and Tony Bingham (CEO of ASTD), to the New York Times.

Special thanks go out to my friends at Questionmark who agreed in advance of me finishing the report to license it for their clients and learning community. Questionmark is providing a great service by making it possible to dessimate world-class research-based information that is both valid and useful, including their support of the aforementioned research report on feedback.

Here are some of the insights from the two-part 88-page research report:

1. The most important thing to remember about feedback is that it is generally beneficial for learners.
2. The second most important thing to remember about feedback is that it should be corrective. Typically, this means that feedback ought to specify what the correct answer is. When learners are still building understanding, however, this could also mean that learners might benefit from additional statements describing the “whys” and “wherefores.”
3. The third most important thing to remember about feedback is that it must be paid attention to in a manner that is conducive to learning.
4. Feedback works by correcting errors, whether those errors are detected or hidden.
5. Feedback works through two separate mechanisms: (a) supporting learners in correctly understanding concepts, and (b) supporting learners in retrieval.
6. To help learners build understanding, feedback should diagnose learners’ incorrect mental models and specifically correct those misconceptions, thereby enabling additional correct retrieval practice opportunities.
7. To prepare learners for future long-term retrieval and fluency, learners need practice in retrieving. For this purpose, retrieval practice is generally more important than feedback.
8. Elaborative feedback may be more beneficial as learners build understanding, whereas brief feedback may be more beneficial as learners practice retrieval.
9. Immediate feedback prevents subsequent confusion and limits the likelihood for continued inappropriate retrieval practice.
10. Delayed feedback creates a beneficial spacing effect.
11. When in doubt about the timing of feedback, you can (a) give immediate feedback and then a subsequent delayed retrieval opportunity, (b) delay feedback slightly, and/or (c) just be sure to give some kind of feedback.
12. Feedback should usually be provided before learners get another chance to retrieve incorrectly again.
13. Provide feedback on correct responses when:
    a. Learners experience difficulty in responding to questions or decisions.
    b. Learners respond correctly with less-than-high confidence.
    c. All the information learned is of critical importance.
    d. Learners are relatively new to the subject material.
    e. The concepts are very complex.
14. Provide feedback on incorrect responses:
    a. Almost always.
    b. Except:
    i. When feedback would disrupt the learning event.
    ii. When it would be better to wait to provide feedback.
15. When learners seek out and/or encounter relevant learning material either before or after feedback, this can modify the benefits of the feedback itself.
16. When learners are working to support retrieval or fluency, short-circuiting their retrieval practice attempts by enabling them to access feedback in advance of retrieval can seriously hurt their learning results.
17. When learners retrieve incorrectly and get subsequent well-designed feedback, they still have not retrieved successfully; so they need at least one additional opportunity to retrieve—preferably after a delay.
18. On-the-job support from managers, mentors, coaches, learning administrators, or performance-support tools can be considered a potentially powerful form of feedback.
19. Training follow-through software—that keeps track of learners’ implementation goals—provides another opportunity for feedback.
20. Feedback can affect future learning by focusing learners on certain aspects of learning material at the expense of other aspects of learning material. Learners may take the hint from the feedback to guide their attention in subsequent learning efforts.
21. Extra acknowledgements (when learners are correct) and extra handholding (when learners are wrong) are generally not effective (depending on the learners). In fact, when feedback encourages learners to think about how well they appear to be doing, future learning can suffer as learners aim to look good instead of working to build rich mental models of the learning concepts.

Some of the concepts and language in the above recommendations may not be obvious until you actually read the research report. You can do that by clicking the link below.

The link to download the feedback report

Human Cognition is More Reactive Than You Think

In today's New York Times there is a great article, Who's Minding the Mind?, by BENEDICT CAREY that sums up a large number of research studies on human cognition that show that human beings are more reactive than we might think. We tend to believe that we, as human beings, are very proactive and consciously in control of our thoughts and actions; but these studies show that much of what we do and think is due to hard-wired, often unconscious processes.

For example, the article cites how sitting near a briefcase (as opposed to a backpack) can make people more competitive. Or as Carey writes:

New studies have found that people tidy up more thoroughly when there’s a faint tang of cleaning liquid in the air; they become more competitive if there’s a briefcase in sight, or more cooperative if they glimpse words like “dependable” and “support” — all without being aware of the change, or what prompted it.

This basic fact about human behavior is relevant to the learning and performance field, of course. One of the things I've talked about for years is the notion of "spontaneous remembering." If we create learning right, we're more likely to help our learners—when they're on the job at a later time—by helping them spontaneously trigger memories of what they've learned. We can do this best by requiring our learners to utilize realistic cues in the learning context in making real-world decisions and taking real-world actions. This is why simulations are so effective (if they are well designed).

When learners process learning objectives or prequestions before encountering learning material, the learners are primed to pay attention to relevant learning material. It's not necessarily a conscious process, but it works.

There are many examples available, but here's another point: The learner-centric movement of the 1990's and 2000's has relied too heavily on the notion that the learners always know best, and that they are in conscious control of their learning and we just need to let them make the best decisions.

When we realize that our learners are more deterministically driven than the we want to believe (its about free will a little, isn't it?), we have more work to do if we really want to drive maximum performance. Even when our clients consciously want to do something, we may be able to help them reach their goals by setting up learning and performance situations that unconsciously trigger the behavior they want to achieve.