Will at Work Learning

Rethinking Instructional Objectives

Clark Quinn (blog, website, Twitter)  recently cited some of my thinking about instructional objectives in the instructional technology forum of AECT (ITFORUM). I wrote a long email to Clark in response, thanking him, and going into more detail. I am reprising my response to Clark here:

In a recent post to this list, Clark Quinn rightly notes that objectives for learners and objectives for instructional designers need not be identical. Indeed, as both Clark and I have previously noted, the probably shouldn’t be identical.

Here’s the thinking: Objectives are designed to guide behavior. So, how can it be that identically-worded objectives can adequately guide the behavior of two disparate groups of individuals (learners and instructional designers)? It just doesn’t make any sense!!

And indeed, Hamilton (1985) found that presenting learners with learning objectives in the way Mager suggested, PRODUCES NO BENEFITS AND MAY BE HARMFUL. Here’s what Hamilton wrote:

“[An instructional] objective that generally identifies the information to be learned in the text will produce robust effects. Including other information (per Mager’s, 1962, definition) will not significantly help and it may hinder the effects of the objectives”

(Hamilton, 1985, p. 78).

Objectives are not only designed to change behavior for a particular set of individuals, but they are also designed with particular purposes in mind—or they should be.

So, when we talk of instructional objectives, we also need to think about what purpose we have for them.

The quote above from Hamilton is focused on how well learning objectives focus the attention of learners. Interestingly, this is the only area in which extensive research has been done on learning objectives. You might be surprised to know that learning objectives help learners focus on the information targeted by learning objectives, but actually diminish their attention on information in the learning materials not targeted by learning objectives. For example, in two experiments using specific objectives, Rothkopf and Billington (1979) found that when focusing objectives were provided to learners, performance on material related to the objectives improved by 49% and 47% over situations when focusing objectives were not used. However, the material not related to the learning objectives was learned 39% and 33% WORSE than it would have been if no learning objectives were used!

These types of instructional objectives—presented to learners prior to subsequent learning—I call “focusing objectives” because they are designed for the purpose of focusing learner attention on critical learning material. As the Hamilton (1985) review pointed out, it does NOT help to add Mager’s criterion information to focusing objectives, because it doesn’t help learners focus on the critical material.

NOW, here’s an important point (I say to focus your attention):  We don’t necessarily need to use focusing objectives with learners if we have other means to focus their attention!! We can use a relevant, gripping story. We can do a shout-out (example, “Here’s an important point…”). We can have them attempt to answer a relevant scenario-based question and struggle with it. Etcetera.

Here’s another important point: Focusing objectives are only one type of objective we might want to utilize. I have a whole list, and I’m sure you can think of more of them.

Instructional Objectives for Learners:

1. Table-of-Contents Objectives
   To give learners a big picture sense of what will be taught.
2. Performance Objectives
   To let learners know what performance will be expected of them.
3. Motivation Objective
   To ensure learners know why they might be motivated to engage the learning or application of the learning.
4. Focusing Objective
   To guide learner attention to the most critical information in the learning material.

Instructional Objectives for Developers:

1. Instructional-Design Objective
   To guide developers toward the ultimate goal of the learning intervention.
2. Evaluation Objective
   To guide developers (and other stakeholders) to the ultimate measurable outcomes that the learning intervention will be measured by.
3. Situation Objectives
   To guide developers to the situations that learners must be prepared for.
4. Organization Objective
   To guide developers to the organizational effects targeted by the instruction.

Questions:

So, here’s some questions for you:

Is it okay to use the word understand in an “instructional-design objective”?

How about in a “focusing objective”?

Answer: It’s okay to use the word understand in a focusing objective—because it does not hurt the learner in setting them up to focus attention on critical concepts. But it is NOT okay to use the word understand in an instructional-design objective—because the word “understand” doesn’t have enough specificity to guide instructional design.

My point in asking these questions is to show that over-simplistic notions about instructional objectives are likely to be harmful to your instructional designs.

As usual, the research helps us see things we wouldn’t otherwise have seen.

Hope this helps!!

= Will

 

References

Hamilton, R. J. (1985). A framework for the evaluation of the effectiveness of adjunct questions and objectives. Review of Educational Research, 55, 47-85.

Mager, R. (1962). Preparing Instructional Objectives. Palo Alto, CA: Fearon Publishers.

Rothkopf, E. Z., & Billington, M. J. (1979). Goal-guided learning from text: Inferring a descriptive processing model from inspection times and eye movements. Journal of Educational Psychology, 71(3), 310-327.

CEO's Do NOT Successfully Transfer Their Skills to Other Companies!

In a recent, NY Time Article, GRETCHEN MORGENSON cited a paper (Executive Superstars, Peer Groups and Over-­‐Compensation –Cause, Effect and Solution) by Charles M. Elson and Craig K. Ferrere who compile research from others (see citations below) that basically shows that CEO's do NOT successfully transfer their skills from one company to another. CEO's who are hired from within tend to do a much better job than CEO's brought in from the outside.

We in the learning-and-performance field should take note. The reason this is true is because it takes real-world expertise, gained through long bouts of experience in a particular set of situations, to give people the tools they need to be experts.

This we should all know. If you want to be good in calculus as it applies to astronomy, you ought to get practice in applying calculus to astronomy. It is not enough to just learn calculus and learn astromony.

And certainly it is not helpful to learn Latin to learn math, or even to learn other languages. Anybody who has been to graduate school in learning should have learned about how unlikely it is for learning to transfer from one domain to the other. Thorndike anyone?

Donald Clark has a nice blog post on Thorndike and the uselessness of learning latin.

So, from both a training and on-the-job learning perspective, we should be giving our learners extensive opportunities to learn their areas of expertise.

This doesn't mean they should never dabble in other domains. In fact, creativity research shows that "domain-spanners" are more likely to be creative than those who only learn with a narrow focus. HOWEVER, people who are only domain-spanners (and don't know a field really well) tend to just be good at coming up with ideas--but not generally good ones. Expertise matters!

Those of you in the United States who are thinking about our election in terms of this lack of transfer from one job situation to another could have a field day...or a depression.

 

Some of Elson and Ferrere's citations showing that CEO's skills don't transfer well:

Richard A. Cazier and John M. McInnis, Do Firms Contract Efficiently on Past Performance When Hiring External CEOs? (Working Paper, 2010) (studying 192 external successions from 1993-2005 they find a negative correlation between excess compensation and future performance;; paradoxically the superstars underperformed).

Gregory L. Nagel, William G. Hardin III, The Transferability of CEO Skills, (Working Paper, 2007)

Mark R. Huson, Paul H. Malatesta and Robert Parrino, Managerial succession and firm performance, 74 J. FIN. ECON. 237, 237 (2004) (finding that “the appointment of outside successors [is] not significantly related to post-turnover performance changes in [their] regression analysis.”)

Chuck Lucier, Steven Wheeler, and Rolf Habbel, The Era of the Inclusive Leader, 47 STRATEGY+BUSINESS (2007) (finding that “experienced CEOs”, though hypothesized to bring experience in dealing with stakeholders and shareholders actually underperformed).

James S. Ang and Gregory L. Nagel, The Financial Outcome of Hiring a CEO from Outside the Firm, (Working Paper, 18, 34, 2011) (using a structural self-selection model to estimate counterfactual performance that would have been obtained if the firm hired an insider rather than an outsider and vice-versa, at all target levels of performance internal hires stochastically dominate outsiders).

Stop Using Slide Templates

Too many organizations insist on using slide templates (slide decorations) in their training slides and presentations. This is a bad idea, and I've created the following narrated slide deck to make a research-based case against these bedeviling adornments:

• On YouTube
• On Vimeo

Is Your Training Course Likely to Boost Performance? Free Course Review Template.

Despite decades of advocacy by our best trade associations, our wisest gurus, and our most practical researchers, most organizations today still rely on training courses that have little impact in promoting on-the-job performance.

As I mentioned in a recent article, we as learning professionals continue to fail in five major ways. You can access that article by clicking here.

I used to think that this was just a failure of knowledge, but in most of the organizations in which I've consulted, there are at least a few learning-and-performance professionals who understand that training alone is not enough. Part of the problem is the dead weight of tradition--the "old normal" continues to blind us to new possibilities. The enlightened few have a hard time pushing back against the gravitational pull of this mass hypnosis.

I recently had a new insight--a way of looking at this problem that I think might enable organizations to break out of their bad habits. The solution is that we have to gain control of the leverage points we have to push for change. We have to change the levers that warp and control our thinking. The big lever is learning measurement. I've been pushing this for years as our most important leverage point. If we measured better, we'd get better feedback, which would push us to create better learning interventions.

But learning measurement isn't our only lever and changing your learning measurement practices is not always easy politically. Beside learning measurement, I've compiled a whole list of other leverage points that  really matter. In fact, it was only recently that I had this incredible insight (one I maybe should have had 10 years ago), that we ought to figure out all the levers we have at our disposal and change them to help push our organizations toward a performance orientation. I'd like to reveal one of those levers today.

One of the things we do in our organizations is review our training courses from time to time--either intentionally or by osmosis and feeling. Well, instead of using the wrong metrics, why not use methods that we know--based on our understanding of learning-and-performance--are likely to be good indicators of whether our training course will support actual on-the-job performance.

The Course Review Template is something that can be used on any training course--classroom training or e-learning.It includes a set of questions that are indicators of how performance-based your training course is. Each rubric in this tool is inspired by research or proven practices which I've learned in my 25+ years in the workplace learning field. 

I should give you a warning. You're unlikely to be happy with what you find. If I bet each of you one dollar for each training course of yours that doesn't support performance, I'd be a millionaire overnight.

But to be fair, I'm going to let you try out the tool yourself. It's free. Use it. And, let me know how your training courses rate. Are they likely to improve on-the-job performance or not?

Click to Download the Course Review Template

After you review a course, post your results at the following link, and when we get enough responses, we'll let you compare your results to others.

Click to Post Your Course Review Results

Maybe I'm having a momentary bout of delusional cognition, but I'm thinking right now that this simple Course Review Template might just revolutionize our ability to simply review our courses to see how performance focused they are.

Such a grandiose statement will provoke eye rolls in some, so let me stipulate a few things. First, this is a first draft, so the Course Review Template is going to be imminently improveable. Second, the Course Review Template is NOT a precision instrument. It is not psychometrically derived, the numbers it assigns to each rubric are best guesses, and there was no super-committee here--just me. Third, the rubrics themselves are subject to interpretation. Instead of overcomplicating the form and making it unusable, I decided to keep it simple and make it less precise. Finally, course reviews are just one of the levers you'll need to completely transition from a course-focus to a performance-focus.

The bottom line is that we have to try some innovate new things to push our organizations to a performance focus. The old ways have not worked. The Course Review Template--or something like it--is worth a try. And seriously, I think it could revolutionize the way your organization views its training courses.

 

The Five Failures of Workplace Learning Professionals

To view this as a PDF, click here.

-------------------------

To improve, we must know our biggest failings.

In the training and development field, our five biggest failures are as follows:

1. We forget to minimize forgetting and improve remembering.
2. We don’t provide training follow-through.
3. We don’t fully utilize the power of prompting mechanisms.
4. We don’t fully leverage on-the-job learning.
5. We measure so poorly that we don’t get good feedback to enable improvement.

1. Minimizing Forgetting, Improving Remembering

It is not enough to help people understand new concepts or even to motivate them to utilize those concepts. If they don’t remember concepts when they encounter situations in which those concepts would be useful, then previous understanding and motivation is for naught.

There are three powerful mechanisms that support long-term remembering, (a) aligning the learning and performance contexts, (b) providing retrieval practice, and (c) utilizing spaced repetitions. Most of our learning interventions do a poor job of providing these mechanisms—resulting in training that may create awareness but doesn’t support remembering or performance improvement.

We need to give our learners more realistic practice using scenarios and simulations. We also need to space repetitions of learning over time—much more than we do now. Instead of trying to teach everything at a basic awareness level, we need to cover less content—but not just present it—instead giving our learners opportunities for deliberate practice.

2. Training Follow-Through

Providing training but no effort to ensure that learners will apply what they’ve learned is the height of professional malpractice. If we assume that learners remember what they’ve learned (which as we just saw is not a given), learners still must (a) remain motivated to apply what they’ve learned, (b) feel that there is some benefit to applying the learning, (c) have the resources and time to put their learning into practice, (d) get feedback and guidance to improve their performance, and (e) be prepared to overcome obstacles and frustrations in applying the learning.

Note how the first two failures create an additive effect—both significantly lessen the likelihood of on-the-job application of the learning. If learners don’t remember, they’re not going to apply what they’ve learned. If learners don’t receive after-training follow-through support, they are unlikely to provide the continuous and persistent focus needed to apply the learning in a way that creates sustainable success.

To reach a credible level of training follow-through we need to (a) engage our learners managers to enlist their support, (b) provide reminders to apply the learning, (c) provide relearning opportunities for that which has been forgotten, (d) enable additional learning to improve and elaborate on the performance, (e) ensure our learners have the resources and time they need to apply the learning and integrate it into their behavioral repertoire, (f) provide coaching support to guide the learning-and-performance process, (g) ensure the learners are incentivized either tangibly with money or perks or intrinsically by aligning efforts with personal values and sense-of-identity, and (h) encourage persistence even in the face of obstacles and frustration.

3. Prompting Mechanisms

Prompting mechanisms rely on one particularly powerful foible of the human cognitive architecture—that our working memories are triggered easily by environmental stimuli. Prompting mechanisms include things like job aids, performance support tools, signage, intuitive cues in our tools and equipment, and some forms of management oversight. They work because they prompt certain strands of thinking, and thus performance. For example, a job aid that lists 5 key interview goals, 10 key interview questions and their rationales automatically triggers in the interviewer a certain way of thinking about interviewing. For example, an interview template might remind its user that interviews are more telling if interviewees are asked to perform a work task or describe how they would perform a work task. Without such a prompt, the interviewer might focus only on how well they think the person would fit into the work culture, etc.

While we are aware of these prompting mechanisms, we are not aggressive enough in their use. If we utilized prompting mechanisms more often with our training and more often as a replacement for training, we’d create better outcomes. If we went looking for grassroots prompting mechanisms already being used and helped spread their use, we’d be more effective. If we evaluated learning facilitators on their use of prompting mechanisms, we’d be more likely to encourage the use of prompting mechanisms. If we asked learners in training to practice with prompting mechanisms, we’d see more being used on the job—and our learners would remember more of what they learned.

4. On-the-Job Learning

We as learning professionals tend to focus almost exclusively on the creation and delivery of training interventions even when we know that our learners are doing a great deal of their learning on the job without any training. Employees learn through trial-and-practice, getting help from others, through social media, by reading task instructions, by using help systems, and so forth. While we have much less direct influence on on-the-job learning than on training, we do have some influence and we ought to use it if we are serious about getting results.

Often the biggest impact we can have is by accessing managers and encouraging them to actively promote learning. Managers can improve learning in their direct reports by (a) making it a point to monitor their employees’ competencies and guide them toward learning opportunities, (b) being approachable and available for questions and advice, (c) creating a culture of learning and information sharing, (d) encouraging data-driven decision-making instead of opinion-driven decision-making, (e) utilizing an experimental mindset, for example by encourage pilot-testing and rapid prototyping, and (f) giving direct reports time for learning and exploration.

We can also have an influence on on-the-job learning by creating and maintaining social-media mechanisms that can be tailored to particular needs. For example, wikis can be used by project teams to get input from various parties and blogs can be used by senior folks to lay out a compelling vision.
We can encourage better on-the-job learning by improving people’s ability to coach their fellow employees. Too often people asked to coach others do a poor job because they just don’t know what good coaching looks like.

We can utilize diagnostic tools to help people in the organization see things about themselves—or about the organization—that they might not otherwise see. For example, if the organization engages in an effort to improve coaching ability, those being coached can be asked to take a short diagnostic survey on how well their coach is doing in coaching them. If an organization wants to change its culture to one that is more flexible and creative, we can utilize a diagnostic to track progress. We can also use a diagnostic to get the organization talking about specifics—so that employees know what behaviors represent the past culture and which represent the new culture.

There are, of course, other things we can do to directly influence on-the-job learning. In addition, we can change our brand by stopping our tendency to be order takers for training. By changing the way we define our role, we can encourage the business side to be fuller partners in organizational learning.

5. Measurement and Feedback to Spur Improvement

We as learning professionals suck at measurement, creating a vacuum of information that pushes us to make poor decision after poor decision in our learning designs. By only seeking learner opinions about the learning, we encourage a bias toward entertainment and engagement and away from content validity, remembering, and application. By measuring only when the learners are in the training context, we don’t learn whether the learning intervention would generate remembering in a work context that is not like the training situation. By measuring only during the learning event, we measure the learning intervention’s ability to create understanding, but we do not measure the learning intervention’s ability to support long-term remembering. We also fail to examine whether any training follow-through is utilized. By utilizing only low-level questions in our tests of learning, we fail to measure the ability of our learners to make decisions that relate to workplace performance. In short, we don’t get the feedback we need to make good learning decisions.

Maintaining ourselves in a state of permanent darkness, we continue to make terrible decisions in regard to learning design, development, and deployment. We design primarily for engagement and understanding, while ignoring remembering, motivation, and application. We hire and promote trainers and training companies who get great ratings but who don’t help learners remember or apply what they’ve learned. Because our measurement is focused only on training, we fail to engage our business partners to ensure that they are adequately supporting learning application—we also never learn what obstacles and leverage points face our learners when they go to apply the learning in their jobs. We build e-learning programs that encourage learners to focus on low-level trivia instead of focusing on the main points. By abstaining from diagnostics, we leave employees blind to conditions from which they might benefit. Poor measurement enables the first four failures.

The bottom line on measurement is that measurement should provide us with valid feedback. Unfortunately, because we haven’t taken the human learning system into account in our measurement designs—and in our measurement models—we are getting biased information and drawing inappropriate conclusions from poor data.

The Five Failures are Fixable

We as learning professionals—as a whole—though working honorably and with good intentions, are too often failing to maximize our impact. Our job is work-performance improvement. We can start by improving our own work performance.

But instead of focusing on everything—which will certainly overwhelm us—we should focus on the things that really matter. We should focus on our five failures. Instead of following willy-nilly prescriptions that pop like fads from a popcorn popper—we should focus on five things that are fundamental—and inspired by the learning research. We should focus on the five failures.

In this brief article, I have provided strong hints about how to rethink and redirect each of the five failures. While such a brief synopsis is certainly not sufficient to enable you to completely redesign your learning efforts, it should, I hope, motivate you to get started.

-------------------------

To view this as a PDF, click here.

 

Learning and Forgetting Curves -- In Depth On Video

I've been sharing the ideas behind the learning and forgetting curves for years.

In this new video, I go into depth about the powerful implications of these curves.

• On Vimeo
• On YouTube

 

 

The Information Explosion

Great article in the Economist on the Information Explosion.

This has huge implications for human learning and performance.

Here's what Bob Cialdini wrote in his masterful book, "Influence: Science and Practice."

More and more frequently, we will find ourselves in the position of the lower animals---with a mental apparatus that is unequipped to deal thoroughly with the intricacy and richness of the outside environment...The consequence of our new deficiency is the same as that of the animals' long-standing one: when making a decision, we will less frequently engage in a fully considered analysis of the total situation. In response to this "paralysis of analysis," we will revert increasingly to a focus on a single, usually reliable feature of the situation...The problem comes when something causes the normally trustworthy cues to counsel us poorly, to lead us to erroneous actions and wrongheaded decisions. (p. 232)

As learning professionals, our clients---our fellow workers---will be more and more confused and duped by information overload. To be successful, we'll have to figure out ways to help them fight their way through the accelerating storm of information.

Again, read the Economist article.

Great Article on How to Create Great Teachers...

Great article on How to Create Great Teachers. It's focused on K-12 education primarily, but there is wisdom in the discussion relevant to workplace learning.

Here's the major points I take away:

1. Great teachers need deep content knowledge.
2. Great teachers need good classroom-management verbalization skills.
3. Great teachers need their content knowledge to be fluently available to them in the context of typical classroom situations. To get this fluency, they need to practice in such situations---and practice linking actions (especially their verbal utterances) to specific classroom situations.

Cammy Bean Interviews Me -- On E-learning Design Flaws

Cammy Bean interviews me in regard to the three most important e-learning design flaws in today's e-learning. I discussed three---and then two more!! Five design flaws in all.

How's your e-learning?

Check out the interview here.

You can also download the segments as podcasts.

The Displacement Hypothesis -- a tangentially-related research study

The Displacement Hypothesis says that one activity can displace another activity.

In the following research by Robert Weis and Brittany Cerankosky, boys given video games (1) did worse in school, (2) spent less time in other after-school activities, (3) had more behavioral problems, and (4) had lower reading and writing scores.

Abstract of the Research Article:

Young boys who did not own video games were promised a video-game system and child-appropriate games in exchange for participating in an “ongoing study of child development.” After baseline assessment of boys’ academic achievement and parent- and teacher-reported behavior, boys were randomly assigned to receive the video-game system immediately or to receive the video-game system after follow-up assessment, 4 months later. Boys who received the system immediately spent more time playing video games and less time engaged in after-school academic activities than comparison children. Boys who received the system immediately also had lower reading and writing scores and greater teacher-reported academic problems at follow-up than comparison children. Amount of video-game play mediated the relationship between video-game ownership and academic outcomes. Results provide experimental evidence that video games may displace after-school activities that have educational value and may interfere with the development of reading and writing skills in some children.

Analogs in Adult Learning

Are there analogs in adult learning? In a quick database review I couldn't uncover any research on the displacement hypothesis with adults, but here are some learning events that may displace other learning events:

1. Twitter
2. Social Networking
3. Web Surfing
4. Blogs
5. RSS following
6. Gaming

Postscript:

When I asked the lead author if he knew of any studies on adults regarding the displacement hypothesis, he said "no," but he pointed me to this article on college students.