Will at Work Learning

New Research Report on Using Culturally, Linguistically, and Situationally Relevant Scenarios

For years I've been compiling research from preeminent refereed journals that shows, time and time again, that aligning the learning and on-the-job performance context is key in supporting long-term remembering.

Now, I continue by focusing on cultural and linguistic context.

 

Read the research report.

Here are the major recommendations:

1. Utilize decision-making scenarios. Consider using them not just in a minor role—for example at the end of a section—but integrated into the main narrative of your learning design.
2. Figure out what the salient cues will be in the workplace situations that your learners will face in utilizing the content you are conveying. As much as possible, simulate those cues in your decision scenarios. Consider using multimedia to augment this effect, relying on excellent acting, directing, and set design to enable the context effects that will trigger remembering.
3. In simulating workplace cues, consider the range of cues that your learners will pay attention to in their work, including background objects, people and their facial expressions, language cues, and cultural referents.
4. Determine the most important points you want to get across AND the most important situations in which these points are critical. Then, provide extra repetitions spaced over time on these key points and situations.
5. Utilize culturally-appropriate objects, backgrounds, actors, and narrators in creating your scenarios. Consider not just ethnicity, but the many aspects of culture, including such things as socio-economics, education, international experience, immersion in popular culture, age, etc.
6. Pilot test new designs using valid evaluation methods to determine the most effective designs for your learners, your workplace situations, and your learning points.

Generating Interest at the Beginning of Learning

Robert Gagne's 1st event of instruction was "Gain Attention." Michael Allen's company, Allen Interactions, has been saying for years, "No More Boring e-Learning." We've all heard the stories of how often e-learning turns learners off. And yet, there is still a whole lot of boring e-learning out there.

An article from the eLearning Guild helps us avoid the trap, specifically by helping us start our learning interventions in ways that grab attention. Paul Clothier interviews Carmen Taran, author of the book Better Beginnings.

Dan Balzer and Susan Manning offer an excellent Podcast on the topic. You can find the link to the Guild article at their web page as well.

Eye Movement Studies. Should we be using them for learning design?

Google has a nice blog post out on its use of eye movement research.

I remember getting a tour of Fidelity a few years ago and learning that their eye movement studies on web browsing showed that people were beginning to ignore big dark chunks of graphics because they thought they were advertisements.

My dissertation advisor, Ernie Rothkopf did a classic study (with Billington) in 1979 using eye movement data to test whether learners actually paid more attention (had more and higher-quality eye movements) toward information in the learning material that was targeted by learning objectives than to information that was not so targeted. It turned out that learning objectives worked to boost learning because they prompted learners to pay more attention to the objective-relevant material and less attention to the rest of the information.

See: Rothkopf, E. Z., & Billington, M. J. (1979). Goal-guided learning from text: Inferring a descriptive processing model from inspection times and eye movements. Journal of Educational Psychology, 71, 310-327.

To answer the question I posed above. Yes, more of us should be using eye-movement research to support us as we do e-learning design.

And by the way, as web pages change their strategies to gain our attention, our learners may change their strategies to avoid things deemed irrelevant. Moreover, as our learners see more and more of our company's e-learning, their eyes may learn where to go...In fact, a lot of them already have a well-learned capacity to find the NEXT key through a swarm of bees.

Present Like Steve Jobs? Yes, BUT...

Here's a nice presentation about what makes Steve Jobs a great presenter.

Check it out, then read my comment below:

Here's my comment on this:

Okay, Steve Jobs is great at giving a product-sales presentation. No doubt about that. But let's not generalize this too far. In my field, the learning-and-performance field, many of the recommendations made here are spot on (for example, keep slides simple and relatively undecorated), BUT some are not relevant (for example, "and one more thing") and some important things are not mentioned (for example, provide people with practice opportunities, etc.).

Jobs also has a big advantage that most of the rest of us don't have. He's a celebrity. For some reason, deep in human evolution, this gives him our loving attention.

Presentation characteristics depend on the audience, purpose, etc. If you acted like Steve Jobs at a scientific convention, you would not be trusted. If you acted like Steve Jobs in training people, you would not create long-term remembering of key learning points.

Again, I'm not criticizing Job's presentation skills. He's perfect for his audience and purpose. I've even used him as an example for some of my training-and-development clients. It's just that we have to be a little discerning in deciding what we can use of Jobs' repertoire for our particular purposes.

Learning Landscape Model Annotated

Last June, I blogged about the Learning Landscape Model, a visual model that shows how learning becomes performance and creates results. It's based on years of research and I have found it very helpful in (1) clarifying my own thoughts, in (2) helping other learning-and-performance professionals fully understand their roles/goals, and in (3) explaining training-and-development to business (non-learning) stakeholders.

I have now created an annotated slide deck to introduce the Learning Landscape Model. It also includes insights and new opportunities for how we can measure learning.

Check it out, and let me know what you think.

In Memory of H.M., cognitive science's most unforgettable experimental subject.

H.M. died on Tuesday. He was a severe amnesiac. At the age of 27 he underwent surgery to correct severe and debilitating seizures. When he awoke, he was unable to remember much of anything ever again--at least not anything in the declarative memory system.

He lived life as the most famous experimental subject in the history of cognitive psychology and neuroscience. I remember reading about him when I was a graduate student in the late 1980's and 1990's. What researchers learned by studying him was that there was more than one memory system. This information led to a revolution in our understanding of human cognition and learning.

After years being known only as H.M., to protect his identity, in death we learn that his name was Henry Gustav Molaison, and he lived his life in Connecticut, on the east coast of the United States.

The New York Times tells his story better than I can. It is well worth the read.

And NPR has a previous story, that you can hear. It is well worth the listen.

And here's H.M.'s wikipedia entry.

Can we utilize the "Elevation" Emotion in Learning?

Read this intriguing article in Slate.

It talks about how some people can elicit an emotional response in others that enables optimism and moral inspiration.

Quoted from the article:

University of Virginia moral psychologist Jonathan Haidt, who coined the term elevation, writes, "Powerful moments of elevation sometimes seem to push a mental 'reset button,' wiping out feelings of cynicism and replacing them with feelings of hope, love, and optimism, and a sense of moral inspiration."

I don't think most training situations would benefit from such elevation, but some might. There's little likelihood that someone learning how to use a spreadsheet could be elevated, for example.

On the other hand, I can see particular opportunities for socially-responsible organizations or initiatives, especially those that are led by elevation-enabling leaders. Perhaps some soft-skill training may benefit, for example, where a management-training facilitator tells stories of others' efforts to help develop the people they work with.

There can be downsides to elevation as well, not least of which is that those who don't feel the elevation think that those who do feel it are either ridiculous or brainwashed. And, elevation by itself doesn't generate changes in behavior.

Haidt's research shows that elevation is good at provoking a desire to make a difference but not so good at motivating real action.

Still, I think it's worth thinking about how to connect with our learners at a deeper level. It doesn't have to be super-deep, just a little bit deeper may help.

Let's remember that we as learning professionals have a responsibility not just to inform, prepare, and provide practice. We can also increase the likelihood that our learners will actually utilize what we teach by enabling their "motivation to apply" what they've learned. Sure, we can utilize our learners' management to promote application, but we can play a role in enabling the learners to want to apply what they've learned. Elevation may be another tool we can use in our work.

FREE Research Report on Feedback

Call me crazy, but I think it's important to invest in the research base for our field. I've spent a good chunk of the last year reviewing research from the world's preeminent refereed journals in regard to how to give learners feedback. I've created, I'd like to think, the seminal research review on how to give learners feedback, written in a way that puts feedback in perspective, that goes deep into the fundamentals to give readers clear mental models for how feedback works. It's this kind of in-depth exploration that allows you as a learning professional to use your wisdom to make the difficult design tradeoffs that you have to make. Recipes are for short-order cooks. Research-based wisdom for learning professionals is much more useful in the gritty day-to-day of our learning shops.

And now, instead of selling this document, I'm going to try an experiment, and give it away. Call me crazy, but internet users (hey that's us) just don't like to pay. I've been swimming upstream against the movement toward free information, knowing that the information I'm compiling is the best information out there, and that it takes an incredibly exhaustive effort to sift through refereed research, make sense of it, and repackage it in a way that resonates and is practical. But maybe research karma will work. It's worth a try, right?

You can help me by reading the report, AND if you think it's good, sending the link to it to everyone you know in your organization, in every learning-development organization you know, to your mom, your kids, your elected officials, to Elliott Masie and Tony Bingham (CEO of ASTD), to the New York Times.

Special thanks go out to my friends at Questionmark who agreed in advance of me finishing the report to license it for their clients and learning community. Questionmark is providing a great service by making it possible to dessimate world-class research-based information that is both valid and useful, including their support of the aforementioned research report on feedback.

Here are some of the insights from the two-part 88-page research report:

1. The most important thing to remember about feedback is that it is generally beneficial for learners.
2. The second most important thing to remember about feedback is that it should be corrective. Typically, this means that feedback ought to specify what the correct answer is. When learners are still building understanding, however, this could also mean that learners might benefit from additional statements describing the “whys” and “wherefores.”
3. The third most important thing to remember about feedback is that it must be paid attention to in a manner that is conducive to learning.
4. Feedback works by correcting errors, whether those errors are detected or hidden.
5. Feedback works through two separate mechanisms: (a) supporting learners in correctly understanding concepts, and (b) supporting learners in retrieval.
6. To help learners build understanding, feedback should diagnose learners’ incorrect mental models and specifically correct those misconceptions, thereby enabling additional correct retrieval practice opportunities.
7. To prepare learners for future long-term retrieval and fluency, learners need practice in retrieving. For this purpose, retrieval practice is generally more important than feedback.
8. Elaborative feedback may be more beneficial as learners build understanding, whereas brief feedback may be more beneficial as learners practice retrieval.
9. Immediate feedback prevents subsequent confusion and limits the likelihood for continued inappropriate retrieval practice.
10. Delayed feedback creates a beneficial spacing effect.
11. When in doubt about the timing of feedback, you can (a) give immediate feedback and then a subsequent delayed retrieval opportunity, (b) delay feedback slightly, and/or (c) just be sure to give some kind of feedback.
12. Feedback should usually be provided before learners get another chance to retrieve incorrectly again.
13. Provide feedback on correct responses when:
    a. Learners experience difficulty in responding to questions or decisions.
    b. Learners respond correctly with less-than-high confidence.
    c. All the information learned is of critical importance.
    d. Learners are relatively new to the subject material.
    e. The concepts are very complex.
14. Provide feedback on incorrect responses:
    a. Almost always.
    b. Except:
    i. When feedback would disrupt the learning event.
    ii. When it would be better to wait to provide feedback.
15. When learners seek out and/or encounter relevant learning material either before or after feedback, this can modify the benefits of the feedback itself.
16. When learners are working to support retrieval or fluency, short-circuiting their retrieval practice attempts by enabling them to access feedback in advance of retrieval can seriously hurt their learning results.
17. When learners retrieve incorrectly and get subsequent well-designed feedback, they still have not retrieved successfully; so they need at least one additional opportunity to retrieve—preferably after a delay.
18. On-the-job support from managers, mentors, coaches, learning administrators, or performance-support tools can be considered a potentially powerful form of feedback.
19. Training follow-through software—that keeps track of learners’ implementation goals—provides another opportunity for feedback.
20. Feedback can affect future learning by focusing learners on certain aspects of learning material at the expense of other aspects of learning material. Learners may take the hint from the feedback to guide their attention in subsequent learning efforts.
21. Extra acknowledgements (when learners are correct) and extra handholding (when learners are wrong) are generally not effective (depending on the learners). In fact, when feedback encourages learners to think about how well they appear to be doing, future learning can suffer as learners aim to look good instead of working to build rich mental models of the learning concepts.

Some of the concepts and language in the above recommendations may not be obvious until you actually read the research report. You can do that by clicking the link below.

The link to download the feedback report

Human Cognition is More Reactive Than You Think

In today's New York Times there is a great article, Who's Minding the Mind?, by BENEDICT CAREY that sums up a large number of research studies on human cognition that show that human beings are more reactive than we might think. We tend to believe that we, as human beings, are very proactive and consciously in control of our thoughts and actions; but these studies show that much of what we do and think is due to hard-wired, often unconscious processes.

For example, the article cites how sitting near a briefcase (as opposed to a backpack) can make people more competitive. Or as Carey writes:

New studies have found that people tidy up more thoroughly when there’s a faint tang of cleaning liquid in the air; they become more competitive if there’s a briefcase in sight, or more cooperative if they glimpse words like “dependable” and “support” — all without being aware of the change, or what prompted it.

This basic fact about human behavior is relevant to the learning and performance field, of course. One of the things I've talked about for years is the notion of "spontaneous remembering." If we create learning right, we're more likely to help our learners—when they're on the job at a later time—by helping them spontaneously trigger memories of what they've learned. We can do this best by requiring our learners to utilize realistic cues in the learning context in making real-world decisions and taking real-world actions. This is why simulations are so effective (if they are well designed).

When learners process learning objectives or prequestions before encountering learning material, the learners are primed to pay attention to relevant learning material. It's not necessarily a conscious process, but it works.

There are many examples available, but here's another point: The learner-centric movement of the 1990's and 2000's has relied too heavily on the notion that the learners always know best, and that they are in conscious control of their learning and we just need to let them make the best decisions.

When we realize that our learners are more deterministically driven than the we want to believe (its about free will a little, isn't it?), we have more work to do if we really want to drive maximum performance. Even when our clients consciously want to do something, we may be able to help them reach their goals by setting up learning and performance situations that unconsciously trigger the behavior they want to achieve.

The Magic Question for Learning and Instructional Design

The most important question that instructional designers can ask is:

“What do learners need to be able to do, and in what situations do they need to do those things?”

While we might discount such a simple question as insignificant, the question brilliantly forces us to focus on our ultimate goals and helps us to align our learning interventions with the human learning system.

Too many of us design with a focus on topics, content, knowledge. This tendency pushes us, almost unconsciously, to create learning that is too boring, filled with too much information, and bereft of practice in realistic situations.

The Magic Question requires us to be relevant. For workplace learning, it focuses our thinking toward learners' future job situations. For education learning, it focuses our thinking toward real-world relevance of our academic topics.

The Magic Question in Practice

In practice, the Magic Question forces us to begin our instructional-design efforts by not only creating a list of instructional objectives, but also by creating a list of performance situations. For example, if we're creating leadership training, we not only need to compile objectives like, "For most decisions, it can be helpful to bring your direct reports into decision-making, so as to increase the likelihood that they will bring energy and passion in implementing decisions." We also need to compile a list of situations were this objective is relevant, for example in weekly staff meetings, project meetings, in one-on-one face-to-face conversations, in phone conversations, etc. Also, for general decision making, but not in situations where time is urgent, where safety is an issue, where legal ramifications are evident, etc.

By framing our instructional-design projects in this way, we get to think about our learning designs in ways that are much more action-oriented, relevant, and practical. The framing makes it more likely that we will align our learning and performance contexts, making it more likely that our learners, in their future situations, will spontaneously remember what we've taught them. The framing makes it more likely that we will focus on practice instead of overloading our learners with information. The framing also makes it more likely that we will utilize relevant scenarios that more fully engage our learners. Finally, using the Magic Question forces our SME's (subject-matter experts) to reformulate their expertise into potent practical packages of relevant material. It's not always easy to bend SME's to this discipline, but after the pain, they'll thank you profusely as together you push their content to a much higher level.

Obviously, there is more to be said about how the Magic Question can be integrated into learning-design efforts. On the other hand, as my clients have reported, the Magic Question has within it a simple power to (1) change the way we think about instructional design, and (2) transform the learning interventions we build.