Will at Work Learning

Deadly Fire Truck Accident Blamed on Poor Training. What We Can Learn From the Tragedy.

It is not always enough to know something. Often people must respond immediately to circumstances. Often they must respond under stress and distraction. To be able to do this, they need to develop a cognitive link between situational cues and action.

On January 9^th 2009, firefighter Robert O’Neill was at the wheel of a 22-ton fire truck as it headed down a steep street in Boston. When O’Neill attempted to apply the brakes, nothing happened. As the truck gained speed heading straight for a large brick wall, O’Neill made frantic attempts to pump the brakes and shift into another gear—ultimately shifting into neutral. His efforts went to no avail and the truck crashed through the brick wall into an apartment building injuring several children in an afterschool program and killing his colleague, firefighter Lieutenant Kevin M. Kelley who was riding beside O’Neill in the truck’s passenger seat.

Photo from Boston Globe Story

After an 11-month investigation, District Attorney Daniel Conley reported that O’Neill received “limited classroom instruction and no driver training in the proper use of air brakes in downhill and emergency circumstances.” What’s really tragic is that O’Neill’s actions in that time of panic may have actually made things worse. As the Boston Globe reported:

The driver did not know to check brake pressure before he got behind the wheel that day and then, when the brakes failed, did not know how to engage secondary braking systems. Instead, he pumped the brakes, releasing any remaining air pressure from the brake system, and put the truck into neutral, preventing the secondary brakes from engaging.

Many things might have prevented this tragedy. The truck could have been better maintained. A job aid that forced drivers to do routine safety check might have been used to ensure brake pressure. Management oversight might have prompted the drivers to actually engage safety routines. Training that helped drivers understand how air brakes worked might have helped—just before the accident the driver turned the truck around in a parking lot releasing air pressure in the braking system. Finally, the driver could have been trained to spontaneously remember what do when facing such a situation.

Ideally, when someone is in an emergency situation, the cues from that situation ought to remind them of what to do. As learning professionals we want to help our learners engage in spontaneous remembering. To do this, we need to help our learners make links between situational cues and actions. It can help to teach these links, but it is even better to have learners practice these links.

Research on general context-alignment effects shows the benefits of making such links (for reviews see Bjork & Richardson-Klavehn, 1989; Smith, 1988; Smith & Vela, 2001; Eich, 1980; Roediger & Guynn, 1996; Davies, 1986). Research on retrieval practice shows us the benefits of retrieval practice in automating such responding (for reviews on retrieval practice see Roediger & Karpicke, 2006a; Pashler, Rohrer, Cepeda, & Carpenter, 2007; Bjork, 1988; Crooks, 1988). Finally, recent research on implementation intentions shows how powerful it can be to help learners link situational cues to action (for review see Gollwitzer & Sheeran, 2006).

If firefighter O’Neill had been properly trained, when he found himself careening down the street with inadequate brakes, the situation would have reminded him to apply steady pressure on the brakes and engage the secondary braking system.

What makes this situation even sadder is that firefighters typically have lots of time between emergencies to engage in training. Even if a high-fidelity simulation was too expensive, a simple e-learning program that simulated driving emergencies might have worked to create cognitive links sufficient to create spontaneous remembering.

 

Research

Bjork, R. A. (1988). Retrieval practice and the maintenance of knowledge. In M. M. Gruneberg, P. E. Morris, R. N. Sykes (Eds.), Practical Aspects of Memory: Current Research and Issues, Vol. 1., Memory in Everyday Life (pp. 396-401). NY: Wiley.

Bjork, R. A., & Richardson-Klavehn, A. (1989). On the puzzling relationship between environmental context and human memory. In C. Izawa (Ed.) Current Issues in Cognitive Processes: The Tulane Floweree Symposium on Cognition (pp. 313-344). Hillsdale, NJ: Erlbaum.

Crooks, T. J. (1988). The impact of classroom evaluation practices on students. Review of Educational Research, 58, 438-481.

Davies, G. (1986). Context effects in episodic memory: A review. Cahiers de Psychologie Cognitive, 6, 157-174.

Eich, J. E. (1980). The cue dependent nature of state dependent retrieval. Memory and Cognition, 8, 157-173.

Gollwitzer, P. M., & Sheeran, P. (2006). Implementation intentions and goal achievement: A meta-analysis of effects and processes. Advances in Experimental Social Psychology, 38, 69-119.

Pashler, H., Rohrer, D., Cepeda, N. J., & Carpenter, S. K. (2007). Enhancing learning and retarding forgetting: Choices and consequences. Psychonomic Bulletin & Review, 14, 187-193.

Roediger, H. L., III, & Guynn, M. J. (1996). Retrieval processes. In E. L. Bjork & R. A. Bjork (eds.), Memory (pp. 197-236). San Diego, CA: Academic Press.

Roediger, H.L. & Karpicke, J.D. (2006a). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181-210.

Smith, S. M. (1988). Environmental context-dependent memory. In G. M. Davies & D. M. Thomson (eds.) Memory in Context: Context in Memory (pp. 13-34), Chichester, UK: Wiley.

Smith, S. M., & Vela, E. (2001). Environmental context-dependent memory: A review and meta-analysis. Psychonomic Bulletin & Review, 8, 203-220.

Story of the accident: http://www.boston.com/news/local/breaking_news/2009/01/fire_engine_cra.html

Story of the accident investigation:

http://www.boston.com/news/local/massachusetts/articles/2009/12/11/poor_training_cited_in_crash/

Photo from Massachusetts AFL-CIO website

Lieutenant Kevin M. Kelley
(Killed in Fire Truck Accident)

New Research Report on Using Culturally, Linguistically, and Situationally Relevant Scenarios

For years I've been compiling research from preeminent refereed journals that shows, time and time again, that aligning the learning and on-the-job performance context is key in supporting long-term remembering.

Now, I continue by focusing on cultural and linguistic context.

 

Read the research report.

Here are the major recommendations:

1. Utilize decision-making scenarios. Consider using them not just in a minor role—for example at the end of a section—but integrated into the main narrative of your learning design.
2. Figure out what the salient cues will be in the workplace situations that your learners will face in utilizing the content you are conveying. As much as possible, simulate those cues in your decision scenarios. Consider using multimedia to augment this effect, relying on excellent acting, directing, and set design to enable the context effects that will trigger remembering.
3. In simulating workplace cues, consider the range of cues that your learners will pay attention to in their work, including background objects, people and their facial expressions, language cues, and cultural referents.
4. Determine the most important points you want to get across AND the most important situations in which these points are critical. Then, provide extra repetitions spaced over time on these key points and situations.
5. Utilize culturally-appropriate objects, backgrounds, actors, and narrators in creating your scenarios. Consider not just ethnicity, but the many aspects of culture, including such things as socio-economics, education, international experience, immersion in popular culture, age, etc.
6. Pilot test new designs using valid evaluation methods to determine the most effective designs for your learners, your workplace situations, and your learning points.

Generating Interest at the Beginning of Learning

Robert Gagne's 1st event of instruction was "Gain Attention." Michael Allen's company, Allen Interactions, has been saying for years, "No More Boring e-Learning." We've all heard the stories of how often e-learning turns learners off. And yet, there is still a whole lot of boring e-learning out there.

An article from the eLearning Guild helps us avoid the trap, specifically by helping us start our learning interventions in ways that grab attention. Paul Clothier interviews Carmen Taran, author of the book Better Beginnings.

Dan Balzer and Susan Manning offer an excellent Podcast on the topic. You can find the link to the Guild article at their web page as well.

Eye Movement Studies. Should we be using them for learning design?

Google has a nice blog post out on its use of eye movement research.

I remember getting a tour of Fidelity a few years ago and learning that their eye movement studies on web browsing showed that people were beginning to ignore big dark chunks of graphics because they thought they were advertisements.

My dissertation advisor, Ernie Rothkopf did a classic study (with Billington) in 1979 using eye movement data to test whether learners actually paid more attention (had more and higher-quality eye movements) toward information in the learning material that was targeted by learning objectives than to information that was not so targeted. It turned out that learning objectives worked to boost learning because they prompted learners to pay more attention to the objective-relevant material and less attention to the rest of the information.

See: Rothkopf, E. Z., & Billington, M. J. (1979). Goal-guided learning from text: Inferring a descriptive processing model from inspection times and eye movements. Journal of Educational Psychology, 71, 310-327.

To answer the question I posed above. Yes, more of us should be using eye-movement research to support us as we do e-learning design.

And by the way, as web pages change their strategies to gain our attention, our learners may change their strategies to avoid things deemed irrelevant. Moreover, as our learners see more and more of our company's e-learning, their eyes may learn where to go...In fact, a lot of them already have a well-learned capacity to find the NEXT key through a swarm of bees.

Present Like Steve Jobs? Yes, BUT...

Here's a nice presentation about what makes Steve Jobs a great presenter.

Check it out, then read my comment below:

Here's my comment on this:

Okay, Steve Jobs is great at giving a product-sales presentation. No doubt about that. But let's not generalize this too far. In my field, the learning-and-performance field, many of the recommendations made here are spot on (for example, keep slides simple and relatively undecorated), BUT some are not relevant (for example, "and one more thing") and some important things are not mentioned (for example, provide people with practice opportunities, etc.).

Jobs also has a big advantage that most of the rest of us don't have. He's a celebrity. For some reason, deep in human evolution, this gives him our loving attention.

Presentation characteristics depend on the audience, purpose, etc. If you acted like Steve Jobs at a scientific convention, you would not be trusted. If you acted like Steve Jobs in training people, you would not create long-term remembering of key learning points.

Again, I'm not criticizing Job's presentation skills. He's perfect for his audience and purpose. I've even used him as an example for some of my training-and-development clients. It's just that we have to be a little discerning in deciding what we can use of Jobs' repertoire for our particular purposes.

Learning Landscape Model Annotated

Last June, I blogged about the Learning Landscape Model, a visual model that shows how learning becomes performance and creates results. It's based on years of research and I have found it very helpful in (1) clarifying my own thoughts, in (2) helping other learning-and-performance professionals fully understand their roles/goals, and in (3) explaining training-and-development to business (non-learning) stakeholders.

I have now created an annotated slide deck to introduce the Learning Landscape Model. It also includes insights and new opportunities for how we can measure learning.

Check it out, and let me know what you think.

In Memory of H.M., cognitive science's most unforgettable experimental subject.

H.M. died on Tuesday. He was a severe amnesiac. At the age of 27 he underwent surgery to correct severe and debilitating seizures. When he awoke, he was unable to remember much of anything ever again--at least not anything in the declarative memory system.

He lived life as the most famous experimental subject in the history of cognitive psychology and neuroscience. I remember reading about him when I was a graduate student in the late 1980's and 1990's. What researchers learned by studying him was that there was more than one memory system. This information led to a revolution in our understanding of human cognition and learning.

After years being known only as H.M., to protect his identity, in death we learn that his name was Henry Gustav Molaison, and he lived his life in Connecticut, on the east coast of the United States.

The New York Times tells his story better than I can. It is well worth the read.

And NPR has a previous story, that you can hear. It is well worth the listen.

And here's H.M.'s wikipedia entry.

Can we utilize the "Elevation" Emotion in Learning?

Read this intriguing article in Slate.

It talks about how some people can elicit an emotional response in others that enables optimism and moral inspiration.

Quoted from the article:

University of Virginia moral psychologist Jonathan Haidt, who coined the term elevation, writes, "Powerful moments of elevation sometimes seem to push a mental 'reset button,' wiping out feelings of cynicism and replacing them with feelings of hope, love, and optimism, and a sense of moral inspiration."

I don't think most training situations would benefit from such elevation, but some might. There's little likelihood that someone learning how to use a spreadsheet could be elevated, for example.

On the other hand, I can see particular opportunities for socially-responsible organizations or initiatives, especially those that are led by elevation-enabling leaders. Perhaps some soft-skill training may benefit, for example, where a management-training facilitator tells stories of others' efforts to help develop the people they work with.

There can be downsides to elevation as well, not least of which is that those who don't feel the elevation think that those who do feel it are either ridiculous or brainwashed. And, elevation by itself doesn't generate changes in behavior.

Haidt's research shows that elevation is good at provoking a desire to make a difference but not so good at motivating real action.

Still, I think it's worth thinking about how to connect with our learners at a deeper level. It doesn't have to be super-deep, just a little bit deeper may help.

Let's remember that we as learning professionals have a responsibility not just to inform, prepare, and provide practice. We can also increase the likelihood that our learners will actually utilize what we teach by enabling their "motivation to apply" what they've learned. Sure, we can utilize our learners' management to promote application, but we can play a role in enabling the learners to want to apply what they've learned. Elevation may be another tool we can use in our work.

FREE Research Report on Feedback

Call me crazy, but I think it's important to invest in the research base for our field. I've spent a good chunk of the last year reviewing research from the world's preeminent refereed journals in regard to how to give learners feedback. I've created, I'd like to think, the seminal research review on how to give learners feedback, written in a way that puts feedback in perspective, that goes deep into the fundamentals to give readers clear mental models for how feedback works. It's this kind of in-depth exploration that allows you as a learning professional to use your wisdom to make the difficult design tradeoffs that you have to make. Recipes are for short-order cooks. Research-based wisdom for learning professionals is much more useful in the gritty day-to-day of our learning shops.

And now, instead of selling this document, I'm going to try an experiment, and give it away. Call me crazy, but internet users (hey that's us) just don't like to pay. I've been swimming upstream against the movement toward free information, knowing that the information I'm compiling is the best information out there, and that it takes an incredibly exhaustive effort to sift through refereed research, make sense of it, and repackage it in a way that resonates and is practical. But maybe research karma will work. It's worth a try, right?

You can help me by reading the report, AND if you think it's good, sending the link to it to everyone you know in your organization, in every learning-development organization you know, to your mom, your kids, your elected officials, to Elliott Masie and Tony Bingham (CEO of ASTD), to the New York Times.

Special thanks go out to my friends at Questionmark who agreed in advance of me finishing the report to license it for their clients and learning community. Questionmark is providing a great service by making it possible to dessimate world-class research-based information that is both valid and useful, including their support of the aforementioned research report on feedback.

Here are some of the insights from the two-part 88-page research report:

1. The most important thing to remember about feedback is that it is generally beneficial for learners.
2. The second most important thing to remember about feedback is that it should be corrective. Typically, this means that feedback ought to specify what the correct answer is. When learners are still building understanding, however, this could also mean that learners might benefit from additional statements describing the “whys” and “wherefores.”
3. The third most important thing to remember about feedback is that it must be paid attention to in a manner that is conducive to learning.
4. Feedback works by correcting errors, whether those errors are detected or hidden.
5. Feedback works through two separate mechanisms: (a) supporting learners in correctly understanding concepts, and (b) supporting learners in retrieval.
6. To help learners build understanding, feedback should diagnose learners’ incorrect mental models and specifically correct those misconceptions, thereby enabling additional correct retrieval practice opportunities.
7. To prepare learners for future long-term retrieval and fluency, learners need practice in retrieving. For this purpose, retrieval practice is generally more important than feedback.
8. Elaborative feedback may be more beneficial as learners build understanding, whereas brief feedback may be more beneficial as learners practice retrieval.
9. Immediate feedback prevents subsequent confusion and limits the likelihood for continued inappropriate retrieval practice.
10. Delayed feedback creates a beneficial spacing effect.
11. When in doubt about the timing of feedback, you can (a) give immediate feedback and then a subsequent delayed retrieval opportunity, (b) delay feedback slightly, and/or (c) just be sure to give some kind of feedback.
12. Feedback should usually be provided before learners get another chance to retrieve incorrectly again.
13. Provide feedback on correct responses when:
    a. Learners experience difficulty in responding to questions or decisions.
    b. Learners respond correctly with less-than-high confidence.
    c. All the information learned is of critical importance.
    d. Learners are relatively new to the subject material.
    e. The concepts are very complex.
14. Provide feedback on incorrect responses:
    a. Almost always.
    b. Except:
    i. When feedback would disrupt the learning event.
    ii. When it would be better to wait to provide feedback.
15. When learners seek out and/or encounter relevant learning material either before or after feedback, this can modify the benefits of the feedback itself.
16. When learners are working to support retrieval or fluency, short-circuiting their retrieval practice attempts by enabling them to access feedback in advance of retrieval can seriously hurt their learning results.
17. When learners retrieve incorrectly and get subsequent well-designed feedback, they still have not retrieved successfully; so they need at least one additional opportunity to retrieve—preferably after a delay.
18. On-the-job support from managers, mentors, coaches, learning administrators, or performance-support tools can be considered a potentially powerful form of feedback.
19. Training follow-through software—that keeps track of learners’ implementation goals—provides another opportunity for feedback.
20. Feedback can affect future learning by focusing learners on certain aspects of learning material at the expense of other aspects of learning material. Learners may take the hint from the feedback to guide their attention in subsequent learning efforts.
21. Extra acknowledgements (when learners are correct) and extra handholding (when learners are wrong) are generally not effective (depending on the learners). In fact, when feedback encourages learners to think about how well they appear to be doing, future learning can suffer as learners aim to look good instead of working to build rich mental models of the learning concepts.

Some of the concepts and language in the above recommendations may not be obvious until you actually read the research report. You can do that by clicking the link below.

The link to download the feedback report

Human Cognition is More Reactive Than You Think

In today's New York Times there is a great article, Who's Minding the Mind?, by BENEDICT CAREY that sums up a large number of research studies on human cognition that show that human beings are more reactive than we might think. We tend to believe that we, as human beings, are very proactive and consciously in control of our thoughts and actions; but these studies show that much of what we do and think is due to hard-wired, often unconscious processes.

For example, the article cites how sitting near a briefcase (as opposed to a backpack) can make people more competitive. Or as Carey writes:

New studies have found that people tidy up more thoroughly when there’s a faint tang of cleaning liquid in the air; they become more competitive if there’s a briefcase in sight, or more cooperative if they glimpse words like “dependable” and “support” — all without being aware of the change, or what prompted it.

This basic fact about human behavior is relevant to the learning and performance field, of course. One of the things I've talked about for years is the notion of "spontaneous remembering." If we create learning right, we're more likely to help our learners—when they're on the job at a later time—by helping them spontaneously trigger memories of what they've learned. We can do this best by requiring our learners to utilize realistic cues in the learning context in making real-world decisions and taking real-world actions. This is why simulations are so effective (if they are well designed).

When learners process learning objectives or prequestions before encountering learning material, the learners are primed to pay attention to relevant learning material. It's not necessarily a conscious process, but it works.

There are many examples available, but here's another point: The learner-centric movement of the 1990's and 2000's has relied too heavily on the notion that the learners always know best, and that they are in conscious control of their learning and we just need to let them make the best decisions.

When we realize that our learners are more deterministically driven than the we want to believe (its about free will a little, isn't it?), we have more work to do if we really want to drive maximum performance. Even when our clients consciously want to do something, we may be able to help them reach their goals by setting up learning and performance situations that unconsciously trigger the behavior they want to achieve.