Work-Learning Research

  • Work-Learning Research
  • Catalog of Publications
My Photo


Notable Books

Recommended Books

  • Turning Research into Results: A Guide to Selecting the Right Performance Solutions, by Richard E. Clark, Fred Estes
  • How People Learn: Brain, Mind, Experience, and School: Expanded Edition, by National Research Council, edited by John Bransford, Ann L. Brown, Rodney R. Cocking
  • Criterion-Referenced Test Development 2nd Edition, by Sharon Shrock, William Coscarelli, Patricia Eyres
  • Michael Allen's Guide to E-Learning, by Michael Allen
  • e-Learning and the Science of Instruction, by Ruth Colvin Clark, Richard E. Mayer
  • Efficiency in E-Learning by Ruth Colvin Clark, Frank Nguyen, John Sweller (2006)

Google Advertisements

Thursday, 16 March 2006

March Madness and Productivity Loss

March Madness, three weeks of college basketball tournaments in the month of March, has been estimated to cost U.S. companies 3.8 billion dollars of lost productivity.

Various estimates and commentary:


There can be upsides to such an energizing event as well, of course. What I wonder is whether any learning or organizational development initiatives might be wrapped around such events. Imagine the following:

  1. Company intranet portal offers March Madness links, and also includes corporate advertising of key business initiatives, strategic messages, or even training opportunities.
  2. Maybe company even makes links unavailable except through this central march-madness portal.
  3. Managers initiate business-critical conversations in staff meetings after highlighting the latest results for the office pool.
  4. Managers utilize March Madness frenzy for reward and recognition.

Anyway, those are just some ideas off the top of my head. I'd love to get your more thoughtful ideas in the comments. Better yet, have you seen any real-world implementations? Have they been successful?

Monday, 13 March 2006

What Prevents the Use of Research

What prevents people in the learning-and-performance field from utilizing proven instructional-design knowledge?

This is an update to an old newsletter post I wrote about in 2002. Most of it is still relevant, but I've learned a thing or two in the last few years.

Back in 2002, I spoke with several very experienced learning-and-performance consultants who have each---in their own way---asked the question above. In our discussions, we've considered several options, which I've flippantly labeled as follows:

  1. They don't know it. (They don't know what works to improve instruction.)
  2. They know it, but the market doesn't care.
  3. They know it, but they'd rather play.
  4. They know it, but don't have the resources to do it.
  5. They know it, but don't think it's important.

Argument 1.
They don't know it. (They don't know what works to improve instruction.)
Let me make this concrete. Do people in our field know that meaningful repetitions are probably our most powerful learning mechanism? Do they know that delayed feedback is usually better than immediate feedback? That spacing learning over time facilitates retention. That it's important to increase learning and decrease forgetting? That interactivity can either be good or bad, depending on what we're asking learners to retrieve from memory? One of my discussants suggested that "everyone knows this stuff and has known it since Gagne talked about it in the 1970's."

Argument 2.
They know it, but the market doesn't care.
The argument: Instructional designers, trainers, performance consultants and others know this stuff, but because the marketplace doesn't demand it, they don't implement what they know will really work. This argument has two variants: The learners don't want it or the clients don't want it.

Argument 3.
They know it, but they'd rather play.
The argument: Designers and developers know this stuff, but they're so focused on utilizing the latest technology or creating the snazziest interface, that they forget to implement what they know.

Argument 4.
They know it, but don't have the resources to use it.
The argument: Everybody knows this stuff, but they don't have the resources to implement it correctly. Either their clients won't pay for it or their organizations don't provide enough resources to do it right.

Argument 5.
They know it, but don't think it's important.
The argument: Everybody knows this stuff, but instructional-design knowledge isn't that important. Organizational, management, and cultural variables are much more important. We can instruct people all we want, but if managers don't reward the learned behaviors, the instruction doesn't matter.

My Thoughts In Brief

First, some data. On the Work-Learning Research website we provide a 15-item quiz that presents people with authentic instructional-design decisions. People in the field should be able to answer these questions with at least some level of proficiency. We might expect them to get at least 60 or 70% correct. Although web-based data-gathering is loaded with pitfalls (we don't really know who is answering the questions, for example), here's what we've found so far: On average, correct responses are running at about 30%. Random guessing would produce 20 to 25% correct. Yes, you've read that correctly---people are doing a little bit better than chance. The verdict: People don't seem to know what works and what doesn't in the way of instructional design.

Some additional data. Our research on learning and performance has revealed that learning can be improved through instruction by up to 220% by utilizing appropriate instructional-design methods. Many of the programs out there do not utilize these methods.

Should we now ignore the other arguments presented above? No, there is truth in them. Our learners and clients don't always know what will work best for them. Developers will always push the envelope and gravitate to new and provocative technologies. Our organizations and our clients will always try to keep costs down. Instruction will never be the only answer. It will never work without organizational supports.

What should we do?

We need to continue our own development and bolster our knowledge of instructional-design. We need to gently educate our learners, clients, and organizations about the benefits of good instructional design and good organizational practices. We need to remind technology's early adopters to remember our learning-and-performance goals. We need to understand instructional-design tradeoffs so that we can make them intelligently. We need to consider organizational realities in determining whether instruction is the most appropriate intervention. We need to develop instruction that will work where it is implemented. We need to build our profession so that we can have a greater impact. We need to keep an open mind and continue to learn from our learners, colleagues, and clients, and from the research on learning and performance.

New Thoughts in 2006

All the above suggestions are worthy, but I have two new answers as well. First, people like me need to do a much better job (me included) communicating research-based ideas. We need to figure out where the current state of knowledge stands and work the new information into that tapestry in a way that makes sense to our audiences. We also have to avoid heavy-handedness in sharing research-based insights, as we must realize that research is not the only means of moving us toward more effective learning interventions.

Secondly, I have come to believe that sharing research-based information like this is not enough. If the field doesn't get better feedback loops into our instructional-design-and-development systems, then nothing much will improve over time, even with the best information presented in the most effective ways.

Friday, 03 February 2006

Who are the best instructional developers?

Who are the best instructional development shops, developers, etc.?

  1. Who's the best custom e-learning development companies?
  2. Who's the best off-the-shelf e-learning development companies?
  3. Who are the best developers?
  4. Who is being the most innovative?

Just curious...

And what criteria would you use to decide?

Wednesday, 07 December 2005

How Google Can Facilitate Learning

Google's mission is "to make the world's information universally accessible and useful."

Good for Google. But implied in this statement is that the world's information should be universally accessible and useful TO ACTUAL INDIVIDUAL HUMAN BEINGS.

This is a very important clarifier. Why? Because IF information is for the use of humans, it must be formulated and delivered in a way that aligns with the human learning system.

Here are some ideas for Google (and its competitors) to consider:

  • People store information in their heads (in their long-term memory systems).
  • People can sometimes access information in other people's heads. For example, my wife might spontaneously remind me of some romantic moment when we first met, I might ask her a question about sustainable agriculture practices (one of her knowledge specialties) and she might tell me what she knows. Thus, there is (1) information from other's heads that is pushed to us and (2) information that we pull from their heads as well (don't visualize this).
  • People can store information intentionally in notes, documents, etc. Information can also be unintentionally stored. In either case, this type of storage has been referred to "external memory" by research psychologists.
  • The information in each person's information storage system degrades with time and experience, and different items of information can degrade at different rates. This process is often called "forgetting." Forgetting is actually an adaptive mechanism because it enables us to access the information most critical to our current performances (in our day-to-day lives).
  • The internet is just one information storage system of importance to an individual person. In its present state, the internet is generally not as effective as an individual's personal storage system. At best, it is a different type of storage system.
  • For the internet and human memory, both storage AND retrieval are critical processes.
  • Information, no matter where it is stored, can be good information or bad. It can be attached to appropriate contextualizing information or inappropriate contextualizing information.
  • We might consider the following six information storage systems as critical to an individual's informational success:
    • their personal memory system
    • their external memory systems (intentional and unintentional)
    • the memory systems of their relatively-contiguous human associates
    • the internet
    • books, magazines, libraries (and all other formal knowledge not yet available on the internet)
    • their immediate surroundings and all the stimuli and cause-and-effect relationships inherent in that wonderful "stimulus swarm" (term heard first from the vocal vibrations of Ernie Rothkopf). Hidden in this reality, is much information, if only we have the knowledge and experience to know how to parse it and make sense of it.

What Google (and its competitors) might do given the information above:

  • Help make the internet forget (or make the retrieval system mimic forgetting)
  • Create reminding systems (or individual learning-management systems, iLMS's) to help people maintain high-importance information in a highly-accessible (easily-retrievable) state (regardless of which storage system we're talking about).
  • Create a methodology to help people work with all these storage systems in a manner that is synergistic.
  • Develop powerful validation systems to help people test or vet their information so they can determine how valid and relevant it is.
  • Do all this in a way that is inuitively simple and easy to use.

Did I forget to mention that I am available to brainstorm ideas for a relatively modest fee (I say modest, because we're talking about the future of all human knowledge). I do realize that this information (that I am available for a fee) is accessible on the internet. But it is better and more useful (for everyone, but especially for me) that this information is highly accessible in your long-term memory, and that you---particularly you folks at Google---utilize that information before you forget it.