Will at Work Learning

Is Your Training Course Likely to Boost Performance? Free Course Review Template.

Despite decades of advocacy by our best trade associations, our wisest gurus, and our most practical researchers, most organizations today still rely on training courses that have little impact in promoting on-the-job performance.

As I mentioned in a recent article, we as learning professionals continue to fail in five major ways. You can access that article by clicking here.

I used to think that this was just a failure of knowledge, but in most of the organizations in which I've consulted, there are at least a few learning-and-performance professionals who understand that training alone is not enough. Part of the problem is the dead weight of tradition--the "old normal" continues to blind us to new possibilities. The enlightened few have a hard time pushing back against the gravitational pull of this mass hypnosis.

I recently had a new insight--a way of looking at this problem that I think might enable organizations to break out of their bad habits. The solution is that we have to gain control of the leverage points we have to push for change. We have to change the levers that warp and control our thinking. The big lever is learning measurement. I've been pushing this for years as our most important leverage point. If we measured better, we'd get better feedback, which would push us to create better learning interventions.

But learning measurement isn't our only lever and changing your learning measurement practices is not always easy politically. Beside learning measurement, I've compiled a whole list of other leverage points that  really matter. In fact, it was only recently that I had this incredible insight (one I maybe should have had 10 years ago), that we ought to figure out all the levers we have at our disposal and change them to help push our organizations toward a performance orientation. I'd like to reveal one of those levers today.

One of the things we do in our organizations is review our training courses from time to time--either intentionally or by osmosis and feeling. Well, instead of using the wrong metrics, why not use methods that we know--based on our understanding of learning-and-performance--are likely to be good indicators of whether our training course will support actual on-the-job performance.

The Course Review Template is something that can be used on any training course--classroom training or e-learning.It includes a set of questions that are indicators of how performance-based your training course is. Each rubric in this tool is inspired by research or proven practices which I've learned in my 25+ years in the workplace learning field. 

I should give you a warning. You're unlikely to be happy with what you find. If I bet each of you one dollar for each training course of yours that doesn't support performance, I'd be a millionaire overnight.

But to be fair, I'm going to let you try out the tool yourself. It's free. Use it. And, let me know how your training courses rate. Are they likely to improve on-the-job performance or not?

Click to Download the Course Review Template

After you review a course, post your results at the following link, and when we get enough responses, we'll let you compare your results to others.

Click to Post Your Course Review Results

Maybe I'm having a momentary bout of delusional cognition, but I'm thinking right now that this simple Course Review Template might just revolutionize our ability to simply review our courses to see how performance focused they are.

Such a grandiose statement will provoke eye rolls in some, so let me stipulate a few things. First, this is a first draft, so the Course Review Template is going to be imminently improveable. Second, the Course Review Template is NOT a precision instrument. It is not psychometrically derived, the numbers it assigns to each rubric are best guesses, and there was no super-committee here--just me. Third, the rubrics themselves are subject to interpretation. Instead of overcomplicating the form and making it unusable, I decided to keep it simple and make it less precise. Finally, course reviews are just one of the levers you'll need to completely transition from a course-focus to a performance-focus.

The bottom line is that we have to try some innovate new things to push our organizations to a performance focus. The old ways have not worked. The Course Review Template--or something like it--is worth a try. And seriously, I think it could revolutionize the way your organization views its training courses.

 

The Five Failures of Workplace Learning Professionals

To view this as a PDF, click here.

-------------------------

To improve, we must know our biggest failings.

In the training and development field, our five biggest failures are as follows:

1. We forget to minimize forgetting and improve remembering.
2. We don’t provide training follow-through.
3. We don’t fully utilize the power of prompting mechanisms.
4. We don’t fully leverage on-the-job learning.
5. We measure so poorly that we don’t get good feedback to enable improvement.

1. Minimizing Forgetting, Improving Remembering

It is not enough to help people understand new concepts or even to motivate them to utilize those concepts. If they don’t remember concepts when they encounter situations in which those concepts would be useful, then previous understanding and motivation is for naught.

There are three powerful mechanisms that support long-term remembering, (a) aligning the learning and performance contexts, (b) providing retrieval practice, and (c) utilizing spaced repetitions. Most of our learning interventions do a poor job of providing these mechanisms—resulting in training that may create awareness but doesn’t support remembering or performance improvement.

We need to give our learners more realistic practice using scenarios and simulations. We also need to space repetitions of learning over time—much more than we do now. Instead of trying to teach everything at a basic awareness level, we need to cover less content—but not just present it—instead giving our learners opportunities for deliberate practice.

2. Training Follow-Through

Providing training but no effort to ensure that learners will apply what they’ve learned is the height of professional malpractice. If we assume that learners remember what they’ve learned (which as we just saw is not a given), learners still must (a) remain motivated to apply what they’ve learned, (b) feel that there is some benefit to applying the learning, (c) have the resources and time to put their learning into practice, (d) get feedback and guidance to improve their performance, and (e) be prepared to overcome obstacles and frustrations in applying the learning.

Note how the first two failures create an additive effect—both significantly lessen the likelihood of on-the-job application of the learning. If learners don’t remember, they’re not going to apply what they’ve learned. If learners don’t receive after-training follow-through support, they are unlikely to provide the continuous and persistent focus needed to apply the learning in a way that creates sustainable success.

To reach a credible level of training follow-through we need to (a) engage our learners managers to enlist their support, (b) provide reminders to apply the learning, (c) provide relearning opportunities for that which has been forgotten, (d) enable additional learning to improve and elaborate on the performance, (e) ensure our learners have the resources and time they need to apply the learning and integrate it into their behavioral repertoire, (f) provide coaching support to guide the learning-and-performance process, (g) ensure the learners are incentivized either tangibly with money or perks or intrinsically by aligning efforts with personal values and sense-of-identity, and (h) encourage persistence even in the face of obstacles and frustration.

3. Prompting Mechanisms

Prompting mechanisms rely on one particularly powerful foible of the human cognitive architecture—that our working memories are triggered easily by environmental stimuli. Prompting mechanisms include things like job aids, performance support tools, signage, intuitive cues in our tools and equipment, and some forms of management oversight. They work because they prompt certain strands of thinking, and thus performance. For example, a job aid that lists 5 key interview goals, 10 key interview questions and their rationales automatically triggers in the interviewer a certain way of thinking about interviewing. For example, an interview template might remind its user that interviews are more telling if interviewees are asked to perform a work task or describe how they would perform a work task. Without such a prompt, the interviewer might focus only on how well they think the person would fit into the work culture, etc.

While we are aware of these prompting mechanisms, we are not aggressive enough in their use. If we utilized prompting mechanisms more often with our training and more often as a replacement for training, we’d create better outcomes. If we went looking for grassroots prompting mechanisms already being used and helped spread their use, we’d be more effective. If we evaluated learning facilitators on their use of prompting mechanisms, we’d be more likely to encourage the use of prompting mechanisms. If we asked learners in training to practice with prompting mechanisms, we’d see more being used on the job—and our learners would remember more of what they learned.

4. On-the-Job Learning

We as learning professionals tend to focus almost exclusively on the creation and delivery of training interventions even when we know that our learners are doing a great deal of their learning on the job without any training. Employees learn through trial-and-practice, getting help from others, through social media, by reading task instructions, by using help systems, and so forth. While we have much less direct influence on on-the-job learning than on training, we do have some influence and we ought to use it if we are serious about getting results.

Often the biggest impact we can have is by accessing managers and encouraging them to actively promote learning. Managers can improve learning in their direct reports by (a) making it a point to monitor their employees’ competencies and guide them toward learning opportunities, (b) being approachable and available for questions and advice, (c) creating a culture of learning and information sharing, (d) encouraging data-driven decision-making instead of opinion-driven decision-making, (e) utilizing an experimental mindset, for example by encourage pilot-testing and rapid prototyping, and (f) giving direct reports time for learning and exploration.

We can also have an influence on on-the-job learning by creating and maintaining social-media mechanisms that can be tailored to particular needs. For example, wikis can be used by project teams to get input from various parties and blogs can be used by senior folks to lay out a compelling vision.
We can encourage better on-the-job learning by improving people’s ability to coach their fellow employees. Too often people asked to coach others do a poor job because they just don’t know what good coaching looks like.

We can utilize diagnostic tools to help people in the organization see things about themselves—or about the organization—that they might not otherwise see. For example, if the organization engages in an effort to improve coaching ability, those being coached can be asked to take a short diagnostic survey on how well their coach is doing in coaching them. If an organization wants to change its culture to one that is more flexible and creative, we can utilize a diagnostic to track progress. We can also use a diagnostic to get the organization talking about specifics—so that employees know what behaviors represent the past culture and which represent the new culture.

There are, of course, other things we can do to directly influence on-the-job learning. In addition, we can change our brand by stopping our tendency to be order takers for training. By changing the way we define our role, we can encourage the business side to be fuller partners in organizational learning.

5. Measurement and Feedback to Spur Improvement

We as learning professionals suck at measurement, creating a vacuum of information that pushes us to make poor decision after poor decision in our learning designs. By only seeking learner opinions about the learning, we encourage a bias toward entertainment and engagement and away from content validity, remembering, and application. By measuring only when the learners are in the training context, we don’t learn whether the learning intervention would generate remembering in a work context that is not like the training situation. By measuring only during the learning event, we measure the learning intervention’s ability to create understanding, but we do not measure the learning intervention’s ability to support long-term remembering. We also fail to examine whether any training follow-through is utilized. By utilizing only low-level questions in our tests of learning, we fail to measure the ability of our learners to make decisions that relate to workplace performance. In short, we don’t get the feedback we need to make good learning decisions.

Maintaining ourselves in a state of permanent darkness, we continue to make terrible decisions in regard to learning design, development, and deployment. We design primarily for engagement and understanding, while ignoring remembering, motivation, and application. We hire and promote trainers and training companies who get great ratings but who don’t help learners remember or apply what they’ve learned. Because our measurement is focused only on training, we fail to engage our business partners to ensure that they are adequately supporting learning application—we also never learn what obstacles and leverage points face our learners when they go to apply the learning in their jobs. We build e-learning programs that encourage learners to focus on low-level trivia instead of focusing on the main points. By abstaining from diagnostics, we leave employees blind to conditions from which they might benefit. Poor measurement enables the first four failures.

The bottom line on measurement is that measurement should provide us with valid feedback. Unfortunately, because we haven’t taken the human learning system into account in our measurement designs—and in our measurement models—we are getting biased information and drawing inappropriate conclusions from poor data.

The Five Failures are Fixable

We as learning professionals—as a whole—though working honorably and with good intentions, are too often failing to maximize our impact. Our job is work-performance improvement. We can start by improving our own work performance.

But instead of focusing on everything—which will certainly overwhelm us—we should focus on the things that really matter. We should focus on our five failures. Instead of following willy-nilly prescriptions that pop like fads from a popcorn popper—we should focus on five things that are fundamental—and inspired by the learning research. We should focus on the five failures.

In this brief article, I have provided strong hints about how to rethink and redirect each of the five failures. While such a brief synopsis is certainly not sufficient to enable you to completely redesign your learning efforts, it should, I hope, motivate you to get started.

-------------------------

To view this as a PDF, click here.

 

Learning and Forgetting Curves -- In Depth On Video

I've been sharing the ideas behind the learning and forgetting curves for years.

In this new video, I go into depth about the powerful implications of these curves.

• On Vimeo
• On YouTube

 

 

The Information Explosion

Great article in the Economist on the Information Explosion.

This has huge implications for human learning and performance.

Here's what Bob Cialdini wrote in his masterful book, "Influence: Science and Practice."

More and more frequently, we will find ourselves in the position of the lower animals---with a mental apparatus that is unequipped to deal thoroughly with the intricacy and richness of the outside environment...The consequence of our new deficiency is the same as that of the animals' long-standing one: when making a decision, we will less frequently engage in a fully considered analysis of the total situation. In response to this "paralysis of analysis," we will revert increasingly to a focus on a single, usually reliable feature of the situation...The problem comes when something causes the normally trustworthy cues to counsel us poorly, to lead us to erroneous actions and wrongheaded decisions. (p. 232)

As learning professionals, our clients---our fellow workers---will be more and more confused and duped by information overload. To be successful, we'll have to figure out ways to help them fight their way through the accelerating storm of information.

Again, read the Economist article.

Great Article on How to Create Great Teachers...

Great article on How to Create Great Teachers. It's focused on K-12 education primarily, but there is wisdom in the discussion relevant to workplace learning.

Here's the major points I take away:

1. Great teachers need deep content knowledge.
2. Great teachers need good classroom-management verbalization skills.
3. Great teachers need their content knowledge to be fluently available to them in the context of typical classroom situations. To get this fluency, they need to practice in such situations---and practice linking actions (especially their verbal utterances) to specific classroom situations.

Cammy Bean Interviews Me -- On E-learning Design Flaws

Cammy Bean interviews me in regard to the three most important e-learning design flaws in today's e-learning. I discussed three---and then two more!! Five design flaws in all.

How's your e-learning?

Check out the interview here.

You can also download the segments as podcasts.

The Displacement Hypothesis -- a tangentially-related research study

The Displacement Hypothesis says that one activity can displace another activity.

In the following research by Robert Weis and Brittany Cerankosky, boys given video games (1) did worse in school, (2) spent less time in other after-school activities, (3) had more behavioral problems, and (4) had lower reading and writing scores.

Abstract of the Research Article:

Young boys who did not own video games were promised a video-game system and child-appropriate games in exchange for participating in an “ongoing study of child development.” After baseline assessment of boys’ academic achievement and parent- and teacher-reported behavior, boys were randomly assigned to receive the video-game system immediately or to receive the video-game system after follow-up assessment, 4 months later. Boys who received the system immediately spent more time playing video games and less time engaged in after-school academic activities than comparison children. Boys who received the system immediately also had lower reading and writing scores and greater teacher-reported academic problems at follow-up than comparison children. Amount of video-game play mediated the relationship between video-game ownership and academic outcomes. Results provide experimental evidence that video games may displace after-school activities that have educational value and may interfere with the development of reading and writing skills in some children.

Analogs in Adult Learning

Are there analogs in adult learning? In a quick database review I couldn't uncover any research on the displacement hypothesis with adults, but here are some learning events that may displace other learning events:

1. Twitter
2. Social Networking
3. Web Surfing
4. Blogs
5. RSS following
6. Gaming

Postscript:

When I asked the lead author if he knew of any studies on adults regarding the displacement hypothesis, he said "no," but he pointed me to this article on college students.

Deadly Fire Truck Accident Blamed on Poor Training. What We Can Learn From the Tragedy.

It is not always enough to know something. Often people must respond immediately to circumstances. Often they must respond under stress and distraction. To be able to do this, they need to develop a cognitive link between situational cues and action.

On January 9^th 2009, firefighter Robert O’Neill was at the wheel of a 22-ton fire truck as it headed down a steep street in Boston. When O’Neill attempted to apply the brakes, nothing happened. As the truck gained speed heading straight for a large brick wall, O’Neill made frantic attempts to pump the brakes and shift into another gear—ultimately shifting into neutral. His efforts went to no avail and the truck crashed through the brick wall into an apartment building injuring several children in an afterschool program and killing his colleague, firefighter Lieutenant Kevin M. Kelley who was riding beside O’Neill in the truck’s passenger seat.

Photo from Boston Globe Story

After an 11-month investigation, District Attorney Daniel Conley reported that O’Neill received “limited classroom instruction and no driver training in the proper use of air brakes in downhill and emergency circumstances.” What’s really tragic is that O’Neill’s actions in that time of panic may have actually made things worse. As the Boston Globe reported:

The driver did not know to check brake pressure before he got behind the wheel that day and then, when the brakes failed, did not know how to engage secondary braking systems. Instead, he pumped the brakes, releasing any remaining air pressure from the brake system, and put the truck into neutral, preventing the secondary brakes from engaging.

Many things might have prevented this tragedy. The truck could have been better maintained. A job aid that forced drivers to do routine safety check might have been used to ensure brake pressure. Management oversight might have prompted the drivers to actually engage safety routines. Training that helped drivers understand how air brakes worked might have helped—just before the accident the driver turned the truck around in a parking lot releasing air pressure in the braking system. Finally, the driver could have been trained to spontaneously remember what do when facing such a situation.

Ideally, when someone is in an emergency situation, the cues from that situation ought to remind them of what to do. As learning professionals we want to help our learners engage in spontaneous remembering. To do this, we need to help our learners make links between situational cues and actions. It can help to teach these links, but it is even better to have learners practice these links.

Research on general context-alignment effects shows the benefits of making such links (for reviews see Bjork & Richardson-Klavehn, 1989; Smith, 1988; Smith & Vela, 2001; Eich, 1980; Roediger & Guynn, 1996; Davies, 1986). Research on retrieval practice shows us the benefits of retrieval practice in automating such responding (for reviews on retrieval practice see Roediger & Karpicke, 2006a; Pashler, Rohrer, Cepeda, & Carpenter, 2007; Bjork, 1988; Crooks, 1988). Finally, recent research on implementation intentions shows how powerful it can be to help learners link situational cues to action (for review see Gollwitzer & Sheeran, 2006).

If firefighter O’Neill had been properly trained, when he found himself careening down the street with inadequate brakes, the situation would have reminded him to apply steady pressure on the brakes and engage the secondary braking system.

What makes this situation even sadder is that firefighters typically have lots of time between emergencies to engage in training. Even if a high-fidelity simulation was too expensive, a simple e-learning program that simulated driving emergencies might have worked to create cognitive links sufficient to create spontaneous remembering.

 

Research

Bjork, R. A. (1988). Retrieval practice and the maintenance of knowledge. In M. M. Gruneberg, P. E. Morris, R. N. Sykes (Eds.), Practical Aspects of Memory: Current Research and Issues, Vol. 1., Memory in Everyday Life (pp. 396-401). NY: Wiley.

Bjork, R. A., & Richardson-Klavehn, A. (1989). On the puzzling relationship between environmental context and human memory. In C. Izawa (Ed.) Current Issues in Cognitive Processes: The Tulane Floweree Symposium on Cognition (pp. 313-344). Hillsdale, NJ: Erlbaum.

Crooks, T. J. (1988). The impact of classroom evaluation practices on students. Review of Educational Research, 58, 438-481.

Davies, G. (1986). Context effects in episodic memory: A review. Cahiers de Psychologie Cognitive, 6, 157-174.

Eich, J. E. (1980). The cue dependent nature of state dependent retrieval. Memory and Cognition, 8, 157-173.

Gollwitzer, P. M., & Sheeran, P. (2006). Implementation intentions and goal achievement: A meta-analysis of effects and processes. Advances in Experimental Social Psychology, 38, 69-119.

Pashler, H., Rohrer, D., Cepeda, N. J., & Carpenter, S. K. (2007). Enhancing learning and retarding forgetting: Choices and consequences. Psychonomic Bulletin & Review, 14, 187-193.

Roediger, H. L., III, & Guynn, M. J. (1996). Retrieval processes. In E. L. Bjork & R. A. Bjork (eds.), Memory (pp. 197-236). San Diego, CA: Academic Press.

Roediger, H.L. & Karpicke, J.D. (2006a). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181-210.

Smith, S. M. (1988). Environmental context-dependent memory. In G. M. Davies & D. M. Thomson (eds.) Memory in Context: Context in Memory (pp. 13-34), Chichester, UK: Wiley.

Smith, S. M., & Vela, E. (2001). Environmental context-dependent memory: A review and meta-analysis. Psychonomic Bulletin & Review, 8, 203-220.

Story of the accident: http://www.boston.com/news/local/breaking_news/2009/01/fire_engine_cra.html

Story of the accident investigation:

http://www.boston.com/news/local/massachusetts/articles/2009/12/11/poor_training_cited_in_crash/

Photo from Massachusetts AFL-CIO website

Lieutenant Kevin M. Kelley
(Killed in Fire Truck Accident)

Will's Video: Job Aids -- Not Just Child's Play

Video Overview:

The following video provides an entertaining and, I hope, enlightening look at the humble job aid.

Featuring:

• This is only the second video that I shot and edited. See how I did.
    
• Allison Rossett, co-author of the book, Job Aids and Performance Support (with Lisa Schafer) is interviewed.
    
• Worldwide public introduction to incredible new talent, the incomparable Alena.
    
• Brewer the dog has cameo role.
    

Video Notes:

Because of YouTube size restrictions, it is divided into 2 parts.

Enjoy in HD (if your computer can handle it) by:

1. Starting the Video
2. Clicking on HD at Lower Right, AND
3. Clicking on the full-screen display (the box in a box) at Lower Right
4. IF the audio doesn't track, your computer can't handle HD.

Part 1

Part 2

Purchasing (or learning more about) Allison's Book:

Is this a Better Video Release Form?

Yikes!! I did a quick search online for video release forms and found that they were poorly written, mired in legalese, and too one-sided. So, I did what any good instructional designer would do, I wrote my own.

I am NOT a lawyer, NOR do I assume that identity on my blog, SO YOU SHOULD check with a lawyer first before borrowing from my attempt (or using it as is).

Still, I'd like to know what you think. Especially if you're a lawyer who specializes in this sort of thing. If you're not a lawyer, but know one, send it to them to get their input.

Here's the document. Download BetterReleaseForm_w1

What do you think?