Will at Work Learning

What Work-Learning Audit Reveals

I recently completed one of the most comprehensive work-learning audits I've ever been asked to do for a major U. S. retailer. The goal of the audit was to find out how their learning programs AND work-learning environment were supporting the stores in being successful. The audit involved (1) structured and unstructured interviewing with all levels of the organization, especially with store personnel, (2) focus groups held across the country with specific groups of store personnel (e.g., clerk, store managers, assistant managers, etc.), (3) task force meetings with senior line managers and representatives throughout the company, (4) learning audits of e-learning courses, (5) learning audits of a mission-critical classroom course, (6) review of company artifacts (CEO messages, publications, databases, intranet, etc.), (7) interviews with learning-and-performance professionals, (8) discussions of business strategy, (9) discussions regarding corporate information and data-gathering capabilities, (10) job shadowing, (11) store observations, etc.

Who/What Do Workers Learn From?

One of the most intriguing results came out of a relatively simple exercise I did with focus-group participants. The following is a rough approximation of those results.

What I did was ask focus-group participants who they learned from. I would hold up a large 6 x 8 index card with a position label on it, for example, "District Manager," "Clerks," or "Corporate." The group would shout out where they thought that card should go on a large diagram I had created on the wall. I would place it on the wall in a particular category based on the verbal responses and then we would negotiate as a group to determine it's final positioning. So for example, participants could say that they learned the following amounts from that person/position, and we often compromised using in-between placement:

• Learned Most
• Learned a Lot
• Learned Some
• Learned a Little
• Learned Least
• Had Little/No Contact with

See the diagram below for a rough example. This one is actually a composite based on several focus groups and more than one position. It gives a fair representation for how frontline retail clerks responded. Note that the orange boxes represent fellow employees, while the blue boxes represent other groups of people or things that they learned from.

There are several key insights from these results:

• People learn the most from those who they work closely with.
• People learn the most from their experience doing the job.
• People learn the most from their self-initiated efforts at learning.
• The more contact, the more learning (for the most part), however there are benefits from learning from experts (e.g., store managers, head clerks), though the worker has to have at least some signicant contact with them to create this benefit. You'll notice that district staff have only a little impact and regional and corporate staff have none.
• E-learning is seen as somewhat facilitative but not a place where workers learn the most. This result may be organization specific as different e-learning designs and implementations might easily move this result higher or lower.

Frontline clerks didn't get much from company magazines and the like, but managers (not represented in the results above) did find value in these. Store managers also reported that networking with other store managers was on of the "Learn Most" entries for them. For this company, this network was even more important than learning from their district managers (their direct bosses). This makes sense because their network is more accessible throughout the heat of the daily grind.

These results were eye-opening for my client, and they are still wrangling with the implications. For example, district managers and district training staff seemed to produce very little learning benefits. So, should their roles for learning be de-emphasized or re-emphasized?

These types of result have to be understood in the larger data-gathering effort of course. Analyzed alone, they suffer from the problem of de-contextualized self-report data. Combined with multiple other data sources, they paint a really robust picture of an organization's learning environment.

Informal Learning, Social Networks, etc.

Vendors are out and about in our field now selling the benefits of complicated and expensive analysis tools for looking at how people learn through so-called informal on-the-job mechanisms. The example above shows that if you don't have the big bucks, there are simpler ways to get good data as well. 

Elliott Masie's LMS Wish List

Elliott Masie came up with a great and very insightful wish list for LMS's. Click here to access it. He even added a few suggestions in the past few days, probably based on feedback from his loyal audience.

I really like the richness that Elliott's suggestions might create for a typical LMS. Most LMS implementations are just a list of course offerings.

On the other hand, I worry about overly complicating options for users. Most workers just don't have extra time to waste. Maybe the suggestion to let users rate the courses comes into play here.

I also worry about user-generated content. It can be great, could be better than what the training folks can create, could engender more engagement, could be bottom line more effective. But we should all recognize that it is a double-edge sword. User generated content could be incorrect, could be a huge waste of time, could cause the organization to leave itself vulnerable to legal liability.

Doesn't Fix the Biggest Problem with the LMS Mentality

The biggest problem with LMS's can't be fixed with Elliott's suggestions. The biggest problem is that the whole LMS face sends a powerful hidden message that "learning" is about taking courses or accessing other learning events. This "Learning Means Sitting" LMS mentality infiltrates whole organizations.

I've seen this recently with one of my clients, a huge retailer, where their LMS has encouraged store managers and other store leaders to focus learning time on taking courses, in lieu of coaching, learning from each other, trying things out and getting feedback, encouraging store employees to take responsibility for particular areas, etc. It's not that they completely ignore these other learning opportunities; it's that the LMS focuses everyones' time and attention on courses, creating a lot of wasted effort.

To get the most from an LMS, you ought to throw away your LMS and start over. People can learn something—develop competencies/skills—from courses or from other means. A competency-management system that offers multiple means to develop oneself is ideal, where courses/events are just one option. I still haven't seen a commercial system that does this though...Most are course first designs.

Maybe I'm too over-the-top recommending that we get rid of all LMS's. I make the statement to highlight the humongous problems that the LMS mentality is causing.

Evidence-Based Learning

The concept of evidence-based practice is bubbling up all over. I started Work-Learning Research in 1998 to bridge the gap between research and practice. Ruth Clark has been talking about evidence-based recommendations for years. The medical profession has transformed itself to focus on evidence-based medicine. The No-Child-Left-Behind initiative was based on the concept of evidence-based education. Though No Child Left Behind has been poorly implemented, the core concept is sound. Kevin Kruse, CEO of the custom e-learning company Axiom Health, has trademarked the term, "Evidence-Based Training." Management gurus, Jeffrey Pfeffer and Bob Sutton have recently written a best selling book (Pfeffer and Sutton, 2006. Hard Facts, Dangerous Half-Truths And Total Nonsense: Profiting From Evidence-Based Management). ISPI (the International Society for Performance Improvement) has been promoting evidence-based practices in their CPT certification program since the 1990's.

Evidence-Based Practice is on an upward trajectory, but what does it mean for those of us in the learning-and-performance field? I'm not going to go into depth now, but I want to highlight a few critical points.

1. Evidence-Based Learning (EBL) DOES NOT simply mean that we follow research-based prescriptions.
2. Evidence-Based Learning (EBL) requires us also to measure our own performance. In other words, we ought to be routinely gathering good EVIDENCE about how well our learning interventions are working. Only by having feedback loops can we learn from our performance.
3. Evidence-Based Learning (EBL) requires us also to build continuous cycles of improvement into our practices. After gathering and analyzing the evidence, we need to act on it. Then we need to evaluate and analyze and act again.

I spend an inordinate amount of time every year culling practical learning wisdom from the world's preeminent refereed journals on learning, memory, and instruction. It appears that I am one of the most passionate advocates for learning research living today. Yet still, I believe that following research-based recommendations can provide us with less than half, probably less than one-third, of the power of a full practice of evidence-based learning.

In fact, I've come to believe that, as a field, we have only reached a small measure of our potential because we don't utilize evidence-based practices. We don't have adequate feedback loops. We act too often on superstition, on the ideas inherent in commercial messaging, and on our learner's lowest-common denominator comfort zones.

The bottom line recommendation is this: Only with true evidence-based practices, not warmed over attempts to follow a few research-based prescriptions, can you build a maximally effective learning program.

Sorry I can't provide more specifics in this short blog format. If you want to know more, feel free to get in touch with me.

The Magic Question for Learning and Instructional Design

The most important question that instructional designers can ask is:

“What do learners need to be able to do, and in what situations do they need to do those things?”

While we might discount such a simple question as insignificant, the question brilliantly forces us to focus on our ultimate goals and helps us to align our learning interventions with the human learning system.

Too many of us design with a focus on topics, content, knowledge. This tendency pushes us, almost unconsciously, to create learning that is too boring, filled with too much information, and bereft of practice in realistic situations.

The Magic Question requires us to be relevant. For workplace learning, it focuses our thinking toward learners' future job situations. For education learning, it focuses our thinking toward real-world relevance of our academic topics.

The Magic Question in Practice

In practice, the Magic Question forces us to begin our instructional-design efforts by not only creating a list of instructional objectives, but also by creating a list of performance situations. For example, if we're creating leadership training, we not only need to compile objectives like, "For most decisions, it can be helpful to bring your direct reports into decision-making, so as to increase the likelihood that they will bring energy and passion in implementing decisions." We also need to compile a list of situations were this objective is relevant, for example in weekly staff meetings, project meetings, in one-on-one face-to-face conversations, in phone conversations, etc. Also, for general decision making, but not in situations where time is urgent, where safety is an issue, where legal ramifications are evident, etc.

By framing our instructional-design projects in this way, we get to think about our learning designs in ways that are much more action-oriented, relevant, and practical. The framing makes it more likely that we will align our learning and performance contexts, making it more likely that our learners, in their future situations, will spontaneously remember what we've taught them. The framing makes it more likely that we will focus on practice instead of overloading our learners with information. The framing also makes it more likely that we will utilize relevant scenarios that more fully engage our learners. Finally, using the Magic Question forces our SME's (subject-matter experts) to reformulate their expertise into potent practical packages of relevant material. It's not always easy to bend SME's to this discipline, but after the pain, they'll thank you profusely as together you push their content to a much higher level.

Obviously, there is more to be said about how the Magic Question can be integrated into learning-design efforts. On the other hand, as my clients have reported, the Magic Question has within it a simple power to (1) change the way we think about instructional design, and (2) transform the learning interventions we build.

Training and Carbon Offsets

The Carbon Offset idea works like this. We all pollute, but when we do so we can help limit the damaging effects by either (1) offsetting our damage by doing good in other ways (for example if we have to drive a large car we can replace all our light bulbs with energy-saving flourescents), or (2) we can donate money to projects that help support renewable energy, energy efficiency, and reforestation. For example, check out the not-for-profit organizations CarbonFund.org and The Clean Air Conservancy.

Here's some ideas for those of us in the training and development field:

1. Encourage the use of e-learning, which limits the carbon footprint of travel. And, make sure you build e-learning that is effective and engaging, so more folks will want to use e-learning.
2. When calculating the "cost" of training, calculate carbon footprint costs as well. See for example, The Carbon Fund's calculators or The Clean Air Conservancy's calculators. Make these costs evident.
3. Encourage your company to buy carbon offsets when utilizing training. It's not just a good thing to do, but it may help your company attract business and recruit highly-educated employees.
4. In your e-learning courses, provide an option for learners to calculate how many tons of carbon dioxide they would have utilized had they had to travel from their location to headquarters.

What other ideas can you think of?

Strong Ideas, Weakly Held

Thanks to George Siemens blog, I learned of a wonderful phrase, "Strong Opinions, Weakly Held," as blogged by Bob Sutton.

I'd actually like to modify it a bit to: "Strong Ideas, Weakly Held." This adds the connotation that the thoughts have been well researched (not just back-of-the-envelope opinions).

In a sense, it's the researchers' mindset--to work tirelessly to gather relevant data, make sense of it, state a conclusion, but then be willing to test that conclusion with data.

Michael Allen rants about Podcasting

Michael Allen has a nice post on Podcasting.

Serious Games

Today's New York Times has a nice article on how games are being used to help people learn about real-world issues like the Middle-East Conflict.

The article does a really nice job of reviewing the "serious games" movement, including the passion of the developers and the thin research support for effectiveness (as of yet).

I'm inclined to think these serious games can have profound learning benefits, but that  measurements of effectiveness are probably difficult to get right.

Also design difficulties include:

1. Insuring the correctness of the cause-and-effect relationships in the game.
2. Insuring that the design doesn't distract from the main points.
3. Insuring that the game itself generates attention to the most important points.
4. Being sure that other methods aren't more efficient and/or effective.

The Long Tail: How it Applies to the Learning and Performance Field.

Chris Anderson, editor of Wired Magazine, has a new book—The Long Tail—another inspired insight ready to rear up like a tsunami and sweep indiscriminately over everything.

The insight from the book and from the original article in a 2004 edition of Wired is this: The low cost of infinite shelf space and the reach of the internet enables niche products to reach an audience—and for this reach to be economically viable. The following chart from Slate Magazine captures the concept nicely.

It’s a nice insight and Anderson musters compelling evidence for the increasing power of the long tail to transform our economy and our business infrastructure.

To hear Anderson being interviewed by the incomparable Tom Ashbrook, check out the On Point archive from July 18, 2006.

To read a critique of the concept—a warning about its boundary conditions—from Slate Magazine’s Tim Wu, check out the article entitled, "The Wrong Tail: How to turn a powerful idea into a dubious theory of everything."

To read how the concept might affect the publishing industry, check this out from the NY Times Book Review.

To read how the concept might apply to the healthcare field, read Jim Walker's thoughtful analysis.

To read a critique of the concept (and the actual economic data) from the Wall Street Journal, read Lee Gomes excellent article, "It May Be a Long Time Before the Long Tail Is Wagging the Web."

To read more about the Long-Tail concept, check out The Long Tail blog.

How does the long tail relate to the learning-and-performance field? I offer some initial thoughts:

1. Course content: The long tail may enable more and more niche players to succeed in the marketplace, potentially hurting companies with large libraries of everything. Is this why Skillsoft’s stock has lost 75% of its value over the last four years (though it’s been inching up lately).
2. Conferences: The long tail may kill, or significantly weaken, the mega conference, pushing attendees into smaller, niche-driven conferences. Simultaneously, vendors (the financial life blood of most conferences), may avoid mega conferences and spend their exhibit dollars in smaller conferences, especially industry-specific conferences.
3. Industry organizations: Organizations like ASTD, ISPI, Masie Center, etc., may lose their influence to specialty organizations (like the elearningGuild) that are attracting an increasingly devoted membership.
4. Employment: Perhaps the long tail will push more and more individuals and small groups into external consulting, development, and delivery functions.
5. Publishing (including books, magazines, eBooks, and blogs): The long tail may make it easier and easier for authors and thought leaders to distribute their works online, putting pressure on book publishers like Pfeiffer and ASTD to compete, and periodicals like T+D, PI, and CLO to maintain an audience in the face of a burgeoning swell of blogs, white papers, and webinars.
6. Industry Advertising: If we think of advertising placement opportunities as products, the long tail may push vendors in our industry to seek niche placements. One opportunity for this is the Google or Yahoo! online advertisements that are already changing the advertising game, but other niche opportunities are available as well.
7. Best practices: Until a seismic event occurs in our industry, professionals have too little impetus to create effective learning-and-performance interventions. Thus, sadly, we will continue to window shop for fad-of-the-moment ideas emerging from the long tail. After emerging from the long tail, fad-of-the-moment ideas will enjoy a brief explosion into the head of the curve, before gradually fading back into the obscurity of the long tail.

Where will the long tail not apply:

1. Credentialing: Although ISPI’s CPT (Certified Performance Technologist) and ASTD’s CPLP (Certified Professional in Learning and Performance)—are both available (as are a few other credentials), multiple credentialing agencies weaken the meaningfulness of a credential, making it likely to put a ceiling on the market power of these credentials.
2. Authoring Tools: Buyers tend to gravitate toward stable tools and systems, not wanting to deal with the uncertainty of new technologies.
3. LMS’s: Again, buyers tend to gravitate toward tools that are proven and vendors that can afford to invest in interface-diverse interoperability.
4. Work-Learning Research and Will Thalheimer: Though I am likely to always to be hidden somewhere in the long tail, I have ambitions to be ubiquitous with my message of research-based practice. BIG TONGUE-IN-CHEEK SMILE.

The Internet is Boring. It's Low Costs That Create Innovation.

Mark Cuban has a nice piece on his blog about how the internet is NOT the reason for innovation, but that cost cutting is. Click to read the piece here.

It's not immediately clear to me how this relates to the learning-and-performance field--maybe the push toward rapid e-learning technologies--but I share Cuban's piece knowing that one of you bright readers may get an insight that leads to improvements in our field.