Will at Work Learning

Purchasing eLearning Tools? Consider Adobe!

I'm writing you from the eLearning Guild's annual conference. I went to a session presented by Silke Fleischer and colleagues at Adobe and was blown away by the work Adobe is doing to create products that support learning and related efforts. I then asked a number of industry thought leaders who confirmed my interpretation: Adobe is now a 500-pound Gorilla, likely to continue out-investing their competitors and thus creating better and better products for folks like us to use.

If you're considering elearning tools, you owe it to your organization to consider Adobe products. I have no financial relationship with Adobe, by the way. This is not to say that other products aren't worthy and/or do some things better than Adobe products. My thinking is this: Companies who invest in their products are often more likely to be there for you in the years to come. I've seen many clients who started using a particular tool five-to-ten years ago, and they are basically stuck with it because of their large installed base of learning courses.

Here are a few of the things that made me wake up and take notice:

1. Adobe's update cycle on Captivate seems to be shrinking, as they are aggressively moving forward in the development of Captivate 4.
2. Captivate is being used for many purposes, including the development of Podcasts, Advertising, etc.
3. You can embed a working Captivate file into Adobe connect and then have webinar or online-learning participants each interact with Captivate objects.
4. PDF files can now include fully-functional interactive images. So documents are not static anymore!!
5. Adobe is working on a new platform called AIR, which will enable the compilation of many types of objects for display and interaction.

Elliott Masie's LMS Wish List

Elliott Masie came up with a great and very insightful wish list for LMS's. Click here to access it. He even added a few suggestions in the past few days, probably based on feedback from his loyal audience.

I really like the richness that Elliott's suggestions might create for a typical LMS. Most LMS implementations are just a list of course offerings.

On the other hand, I worry about overly complicating options for users. Most workers just don't have extra time to waste. Maybe the suggestion to let users rate the courses comes into play here.

I also worry about user-generated content. It can be great, could be better than what the training folks can create, could engender more engagement, could be bottom line more effective. But we should all recognize that it is a double-edge sword. User generated content could be incorrect, could be a huge waste of time, could cause the organization to leave itself vulnerable to legal liability.

Doesn't Fix the Biggest Problem with the LMS Mentality

The biggest problem with LMS's can't be fixed with Elliott's suggestions. The biggest problem is that the whole LMS face sends a powerful hidden message that "learning" is about taking courses or accessing other learning events. This "Learning Means Sitting" LMS mentality infiltrates whole organizations.

I've seen this recently with one of my clients, a huge retailer, where their LMS has encouraged store managers and other store leaders to focus learning time on taking courses, in lieu of coaching, learning from each other, trying things out and getting feedback, encouraging store employees to take responsibility for particular areas, etc. It's not that they completely ignore these other learning opportunities; it's that the LMS focuses everyones' time and attention on courses, creating a lot of wasted effort.

To get the most from an LMS, you ought to throw away your LMS and start over. People can learn something—develop competencies/skills—from courses or from other means. A competency-management system that offers multiple means to develop oneself is ideal, where courses/events are just one option. I still haven't seen a commercial system that does this though...Most are course first designs.

Maybe I'm too over-the-top recommending that we get rid of all LMS's. I make the statement to highlight the humongous problems that the LMS mentality is causing.

Neon Elephant Award 2007 to Sharon Shrock and Bill Coscarelli

I'd like to announce the winners of the 2007 Neon Elephant Award, given this year to Sharon Shrock and Bill Coscarelli for advocating against the use of memorization-level questions in learning measurement and for the use of authentic assessment items, including scenario-based questions, simulations, and real-world skills tests.

The Neon Elephant Award is awarded to a person, team, or organization exemplifying enlightenment, integrity, and innovation in the field of workplace learning and performance. Announced on the day of the winter solstice—the day of the year when the northern hemisphere turns away from darkness toward the light and hope of warmer days to come—the Neon Elephant Award honors those who have truly changed the way we think about the practice of learning and performance improvement. Award winners are selected for demonstrated success in pushing the field forward in significant paradigm-altering ways while maintaining the highest standards of ethics and professionalism.

See the full announcement by clicking here...

Human Cognition is More Reactive Than You Think

In today's New York Times there is a great article, Who's Minding the Mind?, by BENEDICT CAREY that sums up a large number of research studies on human cognition that show that human beings are more reactive than we might think. We tend to believe that we, as human beings, are very proactive and consciously in control of our thoughts and actions; but these studies show that much of what we do and think is due to hard-wired, often unconscious processes.

For example, the article cites how sitting near a briefcase (as opposed to a backpack) can make people more competitive. Or as Carey writes:

New studies have found that people tidy up more thoroughly when there’s a faint tang of cleaning liquid in the air; they become more competitive if there’s a briefcase in sight, or more cooperative if they glimpse words like “dependable” and “support” — all without being aware of the change, or what prompted it.

This basic fact about human behavior is relevant to the learning and performance field, of course. One of the things I've talked about for years is the notion of "spontaneous remembering." If we create learning right, we're more likely to help our learners—when they're on the job at a later time—by helping them spontaneously trigger memories of what they've learned. We can do this best by requiring our learners to utilize realistic cues in the learning context in making real-world decisions and taking real-world actions. This is why simulations are so effective (if they are well designed).

When learners process learning objectives or prequestions before encountering learning material, the learners are primed to pay attention to relevant learning material. It's not necessarily a conscious process, but it works.

There are many examples available, but here's another point: The learner-centric movement of the 1990's and 2000's has relied too heavily on the notion that the learners always know best, and that they are in conscious control of their learning and we just need to let them make the best decisions.

When we realize that our learners are more deterministically driven than the we want to believe (its about free will a little, isn't it?), we have more work to do if we really want to drive maximum performance. Even when our clients consciously want to do something, we may be able to help them reach their goals by setting up learning and performance situations that unconsciously trigger the behavior they want to achieve.

Evidence-Based Learning

The concept of evidence-based practice is bubbling up all over. I started Work-Learning Research in 1998 to bridge the gap between research and practice. Ruth Clark has been talking about evidence-based recommendations for years. The medical profession has transformed itself to focus on evidence-based medicine. The No-Child-Left-Behind initiative was based on the concept of evidence-based education. Though No Child Left Behind has been poorly implemented, the core concept is sound. Kevin Kruse, CEO of the custom e-learning company Axiom Health, has trademarked the term, "Evidence-Based Training." Management gurus, Jeffrey Pfeffer and Bob Sutton have recently written a best selling book (Pfeffer and Sutton, 2006. Hard Facts, Dangerous Half-Truths And Total Nonsense: Profiting From Evidence-Based Management). ISPI (the International Society for Performance Improvement) has been promoting evidence-based practices in their CPT certification program since the 1990's.

Evidence-Based Practice is on an upward trajectory, but what does it mean for those of us in the learning-and-performance field? I'm not going to go into depth now, but I want to highlight a few critical points.

1. Evidence-Based Learning (EBL) DOES NOT simply mean that we follow research-based prescriptions.
2. Evidence-Based Learning (EBL) requires us also to measure our own performance. In other words, we ought to be routinely gathering good EVIDENCE about how well our learning interventions are working. Only by having feedback loops can we learn from our performance.
3. Evidence-Based Learning (EBL) requires us also to build continuous cycles of improvement into our practices. After gathering and analyzing the evidence, we need to act on it. Then we need to evaluate and analyze and act again.

I spend an inordinate amount of time every year culling practical learning wisdom from the world's preeminent refereed journals on learning, memory, and instruction. It appears that I am one of the most passionate advocates for learning research living today. Yet still, I believe that following research-based recommendations can provide us with less than half, probably less than one-third, of the power of a full practice of evidence-based learning.

In fact, I've come to believe that, as a field, we have only reached a small measure of our potential because we don't utilize evidence-based practices. We don't have adequate feedback loops. We act too often on superstition, on the ideas inherent in commercial messaging, and on our learner's lowest-common denominator comfort zones.

The bottom line recommendation is this: Only with true evidence-based practices, not warmed over attempts to follow a few research-based prescriptions, can you build a maximally effective learning program.

Sorry I can't provide more specifics in this short blog format. If you want to know more, feel free to get in touch with me.

Training and Carbon Offsets

The Carbon Offset idea works like this. We all pollute, but when we do so we can help limit the damaging effects by either (1) offsetting our damage by doing good in other ways (for example if we have to drive a large car we can replace all our light bulbs with energy-saving flourescents), or (2) we can donate money to projects that help support renewable energy, energy efficiency, and reforestation. For example, check out the not-for-profit organizations CarbonFund.org and The Clean Air Conservancy.

Here's some ideas for those of us in the training and development field:

1. Encourage the use of e-learning, which limits the carbon footprint of travel. And, make sure you build e-learning that is effective and engaging, so more folks will want to use e-learning.
2. When calculating the "cost" of training, calculate carbon footprint costs as well. See for example, The Carbon Fund's calculators or The Clean Air Conservancy's calculators. Make these costs evident.
3. Encourage your company to buy carbon offsets when utilizing training. It's not just a good thing to do, but it may help your company attract business and recruit highly-educated employees.
4. In your e-learning courses, provide an option for learners to calculate how many tons of carbon dioxide they would have utilized had they had to travel from their location to headquarters.

What other ideas can you think of?

American Health Falling "Short"

In today's New York Times, columnist and economist Paul Krugman details new data that shows that Americans are literally losing stature. Here's a quote from the article.

The data show that Americans, who in the words of a recent paper by the economic historian John Komlos and Benjamin Lauderdale in Social Science Quarterly, were “tallest in the world between colonial times and the middle of the 20th century,” have now “become shorter (and fatter) than Western and Northern Europeans. In fact, the U.S. population is currently at the bottom end of the height distribution in advanced industrial countries.”

This is not a trivial matter. As the paper says, “height is indicative of how well the human organism thrives in its socioeconomic environment.”

The link to the article is here, but you have to be a subscriber to read it.

How might this relate to those of us in the United State's learning-and-performance field? Well, mostly this is an interesting tidbit that we have little control over. On the other hand, it might give us pause. After all, if we create learning programs of equal effectiveness to our overseas competitors, but their learners are healthier than our learners, their learners will learn more and perform better in their work. Their companies will have a competitive advantage. We will all die penniless and alone. (Exaggeration).

Krugman reflects on the argument that American's unhealthy ways might be related the fact that we work too much, and thus don't have time to exercise and eat right. Is that a hook into our responsibility as learning professionals? Is there anything that we can do to lower the average time our workers are swimming in the ocean of work responsibilities?

Don't just think content here. Preaching and information are not likely to help that much.

Well, I'm brainstorming here (as I have no idea), we can encourage e-learning to be done on work time, maybe by utilizing more synchronous, and more social interactions. We can realize that learners will forget a large chunk of what we teach, and either cut the forgettable crap out of our courses or demand from our organization and vendors that it be put into performance support. Maybe we can design m-learning interactions that are especially appropriate to be used during exercise. I don't know how to do this, and it may not be doable, but maybe you can be the one to figure it out. Maybe we can provide truly healthy and delicious food for our training participants.

What else? I don't know. Do you?

Or do you think it's outside our influence?

Wiki Improvements

I have critiqued the wiki revolution (see posts in 2005 and in 2006) because I worried about the garbage-in garbage-out problem. That is, how can consumers of wikized information know whether the information is valid? Previously, the knowledge culture has relied on editors, peer-reviewers, and publishers to vet information prior to publication.

Wikipedia has come to recognize this problem and is taking steps to lessen it. See their WikiProject Fact and Reference Check webpage. As you browse Wikipedia today, you'll often see pages marked with the warning, "This [entry] does not adequately cite its references or sources. Please help improve this article by adding citations to reliable sources."

This is a great first step. Whether it is enough, we will see. It probably depends on the topic area discussed.

And to be balanced, we must recognize that any influence can tilt content toward one truth or another, whether that influence is an editor or an admonition to cite reliable sources. Therefore, my next recommendation is to formalize alternative interpretations within wiki technology and practice. The wiki interface sometimes formalizes and encourages a "one-truth" paradigm, so where one truth is inadequate, the technology needs to encourage a different sort of mental model, one that allows information consumers to feel the heat, sweat, and uncertainty of the truth-building process.

Learning Research Quiz Results 2002-2007

Since 2002, I have offered the Learning Research Quiz on the Work-Learning Research website. Over 1200 people have taken that quiz. 991 people answered all 15 questions. There are 15 questions on the quiz. A blind monkey randomly guessing the answers would score about 24% (there are three to seven answer choices per question).

The questions are scenario-based questions presenting relatively realistic learning-design situations. The answer choices are plausible---there are no throwaway answer choices. The questions are thus not easy to answer. They are specifically designed to measure people's understanding of fundamental learning principles like repetition, spacing, feedback, retrieval practice, instructional objectives, etc.

People who answered the questions are self-selected from visitors to the Work-Learning Research website so we can't really be sure who this sample represents. In general, it is my experience that most people who come to the website are experienced learning professionals who are passionate about learning, training, teaching, and/or instructional design.

Here are the results for the 991 people who answered all 15 questions:

If 24% is a completely random score, what should we expect from those who take the quiz? Would you expect them to do twice as well (around 50% correct?) or do better, say 60%, 70%, 80%, 90%, or 100%?

On average our 991 participants scored 32% correct, barely better than chance responding!!

Scoring 1% to 25% correct were 279 respondents.
Scoring 25% to 50% correct were 610 respondents.
Scoring 50% to 75% correct were 101 respondents.
Scoring 75% to 100% correct was 1 respondent, who scored 80% correct.

Association affilitation had little impact on average correct:

ISPI members scored 36% correct.
ASTD members scored 33% correct.
Other trade association members 28% correct.
Research association members 30% correct.
Non-affilitated respondents scored 31% correct.

Age of respondents had little impact on average correct:

Under 21 years of age scored 26%
21-25 scored 28%
26-35 scored 32%
36-45 scored 33%
46-55 scored 30%
56-65 scored 35%
66-75 scored 33%

Education degree had little impact on average correct:

"Ph.D. in psychology, learning, etc." scored 35% correct.
"Ph.D. in other discipline" scored 34% correct.
"Masters in psychology, learning, etc." scored 33% correct.
"Masters in other discipline" scored 32% correct.
"Bachelors" scored 31% correct.
"Other" scored 29% correct.

Job Title had little impact on average correct:

"Trainer" scored 32% correct.
"Instructional Designer" scored 33% correct.
"Performance Consultant" scored 33% correct.
"Human Performance Technologist" 38% correct.
"E-Learning Specialist" scored 29% correct.
"Learning Technology Developer" scored 33% correct.
"Teacher" scored 31% correct.
"Professor (with little research activity)" scored 33% correct.
"Learning Researcher (professors too)" scored 31% correct.
"Manager of Instructional Development" scored 34% correct.
"Manager of Training" scored 34% correct.
"Student" scored 28% correct.
"Other" scored 29% correct.

Date of Taking the Quiz had little impact on average correct:

2002 scored 30%
2003 scored 33%
2004 scored 31%
2005 scored 33%
2006 scored 30%
2007 scored 33%

Conclusion

Because the sample of respondents is difficult to define, any conclusions must remain speculative. Still, the results suggest a massive competence gap. The 32% average score---and the stubborn lack of improvement regardless of experience, education, and age---suggests that most people in the learning-and-performance field are unprepared for roles as designers of learning, at least as far as their ability to apply knowledge of learning research.

Links

• The Quiz
• The Quiz Questions with Feedback

m-Learning Device is Coming: Apple iPhone

Apple's new iPhone is really quite amazing. I'm not usually a gadget guy, but you ought to check this out. It really has some great potential as an m-learning device. It will have some of the same limitations of any m-learning device, but it does give us a good glimpse of the future. Check out Steve Job's introduction of the iPhone by clicking here. Job's keynote is not only interesting for the content. It's great theater as well. I clicked on it to find a quote, but I ended up watching almost the whole damn keynote. I found it inspiring. And I don't think it was inspiring because I was once the proud owner of the first line of the Apple Macintosh's---the one's with the 128K disk drives. SMILE.

Of course, there is always a downside isn't there. Check out this article from the NY Times on how Apple is restricting iPod and iTunes users in using music files. Is Apple in danger of losing it's Teflon-coated image as the good-tech angel?

You can hear an analysis of the iPhone and Apple's stock option troubles on the radio show OnPoint.

Lev Grossman wrote the following in Time Magazine.

"Everybody hates their phone," Jobs says, "and that's not a good thing. And there's an opportunity there." To Jobs's perfectionist eyes, phones are broken. Jobs likes things that are broken. It means he can make something that isn't and sell it to you for a premium price.

This brilliant business analysis ought to relate to our industry as well. What's out there in the training-and-development, e-learning, learning-and-performance field that is broken? What can we fix? What can we get people to actually pay for?

Stay tuned for my answers to these questions over the next couple years. And leave your comments about what is broken and what can be fixed.