Will at Work Learning

Conferences Go Paperless -- How Learning, Remembering, and On-the-Job Application Suffer

Last week I spoke at one of the major conferences in the workplace learning-and-performance industry to a room filled with about 150 people. I am not including the name of the conference organizer because I don't want to single them out. They are not alone in the going-paperless practice.

Unfortunately, two of the conference decisions---(1) not to provide participants with paper handouts---(2) not to allow electronic handouts to include sufficient pages to provide images of the slides that are shown---may have hurt learning results. Perhaps ironic in a conference devoted to help people build better learning interventions.

Setting the Scene of My Presentation

I spoke on the topic of Situation-Based Learning Design and the session seemed to be well-received. I got the following emails afterward:

• "I found your session to be amazing!"
• "One of the highlights of the conference for me."

One of my friends, who sat in the audience, told me afterward that he knew the session was good because (1) hardly anybody left even after 90 minutes, and (2) many people were furiously taking notes.

But was it a successful learning experience? Isn't that the key question?

I could have been entertaining. People could have liked the session. They could have taken lots of notes. But probably the most important reason they came to the conference was so that they could go back and do something better and to think differently because they had attended.

I used some learning-design techniques to help them remember---to help them integrate the new learning with their current conceptions---but was this enough?

I think something was missing because of the paperless practices, but before I get to that, let me recount some of the design tactics I used specifically to ensure remembering and application to their work.

1. Content was verified (by research) to be effective in supporting good work practices around learning design. Some stuff you learn at conferences can actually make you less effective.
2. Participants were provided with realistic learning-design decisions to make before topics were introduced. By activating prior knowledge and situating the learning in situations similar to those folks would face in their work as learning professionals, the learning would become more usable and more memorable.
3. Examples of real-world e-learning courses were provided to again link realistic context cues to the learning points.
4. Learners were encouraged to specifically engage in trigger setting (what researchers call implementation intentions) to increase the likelihood that they would take action in implementing what they learned
5. Participants were presented with spaced repetitions of key learning points to support long-term remembering. Repetitions were achieved by providing (a) prequestions, (b) research explanations, (c) supporting metaphors, (d) real-world examples, (e) follow-up practice, (f) similar topics that reinforced points from previous topics, (g) a final summary, and (h) a reminder to link the learning to their future situations.

Good stuff to be sure, but was anything missing?

I think so.

The biggest thing that is missing is super-strong links back to the workplace. Even though I use great long-term retrieval support (better than most conference sessions provide I would think), having our learners rely only on the learning-remembering link when transportable cues are available is committing learning-to-performance malpractice. If learners could take their annotated slide images back to the workplace, they would remember more, share more, and apply more of the session content.

The Problem of Going Paperless.

Note-taking can prompt learners to engage in deep processing of their learning. By organizing the learning content during note-taking, learners will be more likely to fully understand what they're learning and remember it later. Note-taking has other benefits as well. Learners can go back and study their notes at a later time, providing themselves with spaced repetitions---a proven aid to long-term remembering. Finally, learners who take notes may be more likely to attempt to share what they've learned with others when they return to work. The process of preparing to teach others, provides further deep processing of the learning material and makes it more likely that the learning content will be remembered and utilized on the job.

When a conference goes paperless, it is likely to curtail note-taking. When I asked my 150 learners how many of them had paper copies or were looking at the handouts on their laptops, less than half raised their hands.

Just because participants can print out the handouts beforehand, doesn't mean that they will. This is a lame excuse for conference organizers.

Taking Notes with Slide Images versus Taking Notes on Plain Paper

The conference organizer only allowed us 8 pages of handouts. I could only fit 12 of my slides into these 8 pages, because some other things were required as well. For me, this meant that more than 85% of my slides were NOT provided to my learners.

Ironically, one of the research streams I talked about in my session was the importance of context in learning and remembering. Specifically, research is very clear that if stimuli are available in both the learning context and the retrieval context, people will be better able to remember what they learned. The contextual cues in the retrieval situation remind the learners of what they learned.

Slide images are such stimulus cues. If learners can transport slide images (and any notes they've added to those images) from the learning situation (my session) to their work situation, they will remember more of what they learned. So, conference organizers who deprive their participants (aka delegates) of slide images and concomitant note-taking are hurting learning.

This is compounded because learners will also be less likely to take back their learning to their coworkers. Just yesterday I did a web meeting with a prospective client and last night I got a request from that prospective client to get copies of my slides. They said they took notes, but they wanted the slides to ensure they didn't miss anything and also to prepare to persuade others in their organization.

Finally, if you've ever looked at a copy of somebody else's slide deck, you know how hard it is to make sense of it without the context provided by aural narration or annotations. People need to be able take notes so that they will later have the full context of the slides. Research also shows that labels, annotations, and notes must be geographically close to the objects of those textual elements---so that limited cognitive resources are not overtaxed, hurting learning. Bottom line here: Notes near the paper on slide images are more effective than notes taken without slide images.

The Push To Paperless

The paperless movement is bound to continue. It cuts into profits for conference organizers to print handouts. It goes against the rising tide of the green environmentally-friendly zeitgeist (which I am very sympathetic to).

If everyone brought their laptop to each session and had the slides available on their screens, they could take notes using Adobe Acrobat's commenting features, or perhaps some other program. What is needed is the ability to put a comment or note where you want it to be, and draw simple diagrams in case there is some visual meaning that should be captured. Note that PowerPoint, even the latest 2007 version, has no such capability. 

One must wonder if the environmental costs of laptop energy-use might outweigh the environmental costs of paper printing, shipping, etc.

If organizers expect participants to print their own copies, than there doesn't appear to be an environmental benefit. Just the costs are shifted to the conference participant.

I have found that I am sometimes able to annotate my own slides so that my audience members don't have to---this would create a minor learning benefit perhaps---but it doesn't really solve the underlying issues.

Academic conferences often have session-leaders write short papers that conference participants can access for later reading. This might be a way to transfer more of the learning content back to the workplace---but could we really make this cultural shift. Also, such short papers would provide a strictly post-learning intervention. It wouldn't fully compensate for the loss of context and note-taking.

Summing it Up

When conferences go paperless:

1. They either hurt learning, remembering, and on-the-job application,
2. Or, they create minor environmental improvements.
3. With current state of installed technology usage, they can't create good learning impacts and good environmental impacts (most people don't have a way to take notes on their slides on their computer).
4. Conference organizers save themselves a heap of money, increasing their profits, which probably doesn't trickle down to conference participants in the form of lower registration fees, especially if participants have to print and transport handouts to the conference and back by themselves.
5. You as a learner have to make a conscious decision to support your conference learning, and you can do this by getting a PDF of the slides beforehand and either printing them out or using Adobe Acrobat to take notes on the slides on your laptop.
6. If you as a conference goer want to maximize learning, you can contact conference organizers and complain, or vote with your dollars by not attending conferences that don't support your learning.

• Tell them: You want slide images.
• Tell them: You want a way to take notes on those slide images.

DevLearn 2007

I've just spent 3 wonderful days in San Jose at the eLearning Guild's DevLearn 2007 conference. Here were some of the highlights for me:

• Hanging out with Ruth Clark a few times during lunches, keynotes, etc. We had a blast discussing research, the state of the profession, and the joy and challenge of doing the research-to-practice thing.
• Seeing Ruth Clark and Silke Fleischer (of Adobe Systems) present the research AND practice of Richard Mayer's work on multimedia learning. Silke did a nice job of demonstrating e-learning examples in Adobe's Captivate. Ruth did a wonderful job discussing the research, framing it in terms of practical application, and describing its limitations. Adobe deserves a ton of credit for supporting the dissemination of Ruth's work and helping to distribute it to a wide audience. Three cheers for enlightened companies like Adobe and Questionmark who support research dissemination.
• Having Google Maps change my behavior by including a public transportation option when I did a search in San Jose. I had actually made a reservation to rent a car so I could drive from my hotel to the downtown hotel where the conference was held. When I went to Google Maps to search for the best route, I was offered a public transportation search. I found out I could get downtown in 12 minutes for only $1.75. I cancelled my rental car and happily commuted. Saving me money (Google showed my gas savings), parking fees, aggravation, etc. Awesome!! Technology changes everything.
• Seeing a great keynote by Paul Saffo who talked about technology innovation and reminded my how many failures are required before success is rewarded. It was one of those rare keynotes that was both well-delivered and superbly relevant for the conference. Way to go Heidi Fisk (of the eLearning Guild) for a great keynote selection.
• Wonderful food. Yes. At a conference!! Healthy, fresh, tasty. Way to go Fairmont Hotel.
• The now-famous DevLearn DemoFest where dozens of e-learning developers show off their wares. It's a great way to take a snapshot of the state of the e-learning industry.
• Playing tennis with a Wii remote. My wife and I are not TV people, so I never pay attention to all the new remotes, Xbox's, etc. At the conference, I played tennis for about 2 minutes and had some fun. The cool thing about the Wii  is that it tracks the remote's movement and simulates that on the screen. Actually, with the Wii my first serve percentage was about 100%, much better than real life.
• My Wednesday Breakfast Bytes session on the intersection of e-Technology and Informal Learning. We had a great conversation and I learned some things.
• My Thursday Breakfast Bytes session on Situation-Based Instructional Design. The basic nugget is that people behave by (1) Being in a situation, (2) Evaluating that situation to make sense of it, (3) Deciding what to do, and (4) taking Action. So, we ought to give our learners practice in doing the whole SEDA process (Situation-Evaluation-Decision-Action). And, we can benefit from asking the Magic Question. Yes, there is more to it than that. The bottom line for me is that my clients have found the concept very helpful in helping them design learning that goes beyond the typical topic-based designs.
• The DevLearn Breakfast Bytes sessions do a great job in getting conversations going. As always, Guild members come to the conference with experience and are ready to share their insights and wisdom. I love Breakfast Bytes.
• My regular session on Learning Measurement. Another great discussion with—what seemed to me—like lots of light bulbs going off. Fun, even after several days of conferencing.
• I met a guy at the Demo Fest—John D’Amours—who had actually tried to do a control-group experiment comparing his e-learning design to a traditional design. Yes. Yes. Yes. We need more folks taking this kind of initiative.
• Learning that Windows Vista NORMALLY runs slow with 2GB of memory. Glad it wasn't just my machine.
• AND so many other great conversations and sessions. Sorry if I failed to mention you!! Hugs to all. I really learn a lot at eLearning Guild events. And I have to say, I feel that my contribution is especially appreciated. Thanks Guild members and staff !!!!!!!!!!

eLearning Guild Oct 2006 Conference Review

I recently attended the eLearning Guild's DevLearn06 conference, dedicated to e-learning developers. I gave a preconference workshop on the learning research and two conference sessions, one on the spacing effect and one on assessments (especially as they are illuminated by the learning research).

Here's some quick highlights from the conference:

1. 50 developers showed off their latest and greatest e-learning interventions in the eLearning Guild's Demo Fest. This 2.5 hour opportunity is a great addition to the typical conference fare. I noticed two things:

• The production values and aesthetic quality of the entries were remarkable. Beautiful work. So, if you're developing e-learning, you better start hiring talented graphic designers and managers who have an aesthetic sensibility.
• The designs were generally much better than page turners, but they still weren't completely consistent with research-based practices. There were good interactions and lots of decision scenarios, but still there were too many missed opportunities for authentic practice and decision-making.

2. The eLearning Guild's Research Posse, led by the ingenious and tireless Steve Wexler, has developed an amazing data-gathering tool, that enables eLearning Guild members to find out the market penetration of all kinds of things, including authoring tools, simulation practices, m-Learning, etc., etc. With this foray into industry research, the Guild easily leapfrog's the feeble attempts by ASTD and Bersin, both of which provide only broad strokes analysis of industry data.

3. Great session by Judy Brown on m-Learning, who showed (a) an amazing installed base of m-Learning tools that makes it clear that m-learning tsunami is on its way here, and (b) some interesting statistics showing that the US is way behind Asia and Western Europe in the amount of data that is utilized on cell phones (the more data, the more learning programs).

Learning 2005 Conference Critique

Learning 2005 was a very good conference, but as is typical in our industry (training & development) this conference still lacked some fundamental elements. Some of the good and bad points of the conference are detailed below. They are provided to encourage changes in all future industry conferences.

The Good:

1. High energy and excitement, created by Elliott and his pre-conference emailings, Elliott's energetic facilitation of the general sessions, and the experimental use of new technologies.
2. Taking risks and trying new models for conferences.
3. Good opportunity to spur future thinking on leading edge topics. For example, everyone at the conference will be likely to consider the use of wiki's.
4. The strong encouragement to presenters to be facilitators instead of lecturers created some really good sessions. It also helped a few participants feel free to criticize (or be skeptical of) the ideas of others. This is a good thing, though more of this is necessary.
5. The human element rose higher than at any other conference in the industry. Not-for-profit groups were highlighted. People's good works were celebrated with formal "Pioneers of Learning" awards. Elliott's empathy and general-session conversations channeled a sense of relationship with the speakers and a sense of community.
6. The keynotes actually has something to say about learning and performance. Yippee!!
7. Elliott's strong emphasis on research, metrics, and innovation.
8. The sense that technological innovations were about to really get traction. A tipping point toward better e-learning design.
9. Malcolm Gladwell was a clear and thoughtful thinker!!
10. The Masie Center's efforts to organize the group "The Learning Consortium" provided a sense that we were all in this together with the goal of helping each other.
11. The participants seemed very knowledgable and thoughtful (at least in the sessions I attended).
12. The wiki at www.learningwiki.com provides a nice opportunity for discussion and further community building. It also provides participants with a way connect with sessions/presenters who they missed because two or more sessions of interest were scheduled at the same time. It remains to see how this will work, but it is certainly worth the experiment. We as conference-goers may need to learn how to use this technology to get the most out of it.
13. The Masie Center people are really nice and really helpful.
14. Elliott didn't let the keynote speakers simply deliver their canned speeches. He forced them to focus on what was most relevant by interviewing them. Very effective.
15. The conference was fun!!

The Neutral

1. No exhibition hall. Good and bad. Good because sponsors didn't show up just to sell. Bad because the selling took a more stealthy approach. Bad because sometimes it is nice as a participant to seek out potential vendors and have real conversations with them.
2. The recommendation NOT to use PowerPoints is both good and bad. Sometimes visuals are helpful to make the point, clarify potential confusions, etc. PowerPoint is a tool. It is good for some things and bad for others. For example, I would have loved to see a brief PowerPoint when Elliott invited people up on stage so that I could catch their names. This seems respectful as well as a good learning design. Also, many presenters did not follow the recommendation and used PowerPoints anyway.

The Bad

1. Commercial interests still dominated many conversations. Some sponsors paid as much as $15,000 to lead three conference sessions. Although some sponsors led excellent sessions, others gave thinly-disguised sales pitches. If vendors dominate the conversational agenda, we diminish the wisdom of experts and we diminish the credibility of our field. Vendors should certainly be a part of the conversation, because many are leading the way, but they shouldn't be the most powerful voice.
2. Some of the sessions floundered under the goal of creating audience-generated discussions. Some of the facilatators were unskilled or talked too much. Some were simply new at facilitating, having more experience in traditional conference sessions delivering their presentations. Some sessions, while great as brainstorming sessions, were unencumbered by the hard work of separating the good ideas from the bad ones.
3. There was not enough planned skepticism, making it likely that some bad ideas were accepted as good ideas. This is a major failing in our industry and one of the reasons why we jump from one fad to another. We are simply not skeptical enough and our conferences don't encourage skepticism.
4. It would be nice to have each presenter asked a mandatory set of questions, including (a) what evidence do you have that your recommendations worked in your situation, (b) what evidence or research can you cite that shows that your recommendations are likely to work in other situations, (c) what research supports the basic concepts you've implemented, and/or (d) what negative and positive side effects might your recommendations create?
5. Sometimes the gee-whiz factor went way too far, as when everybody got real excited just because a technology was new.
6. There was very little discussion of the human learning system and how we needed to align our learning designs with it.
7. The price was rather high for the conference admission.
8. Disney's hermetically-sealed hotel rooms, ingrained with toxic cleaning products, did not provide a healthy environment for sleeping or hanging out.

Learning 2005 Review of Day 4

Yesterday was the final day of the conference. In the final general session, one of the speakers, Mike somebody (sorry, but there were no overheads to announce the general session speakers), warned the audience not to oversell Extreme Learning, Elliott's term for pushing the technological boundaries and creating very short and quick learning episodes. Actually, now that I think about it, "Extreme Learning" is not really that clear. I guess it has to do with technical innovation. Anyway, the warning seems reasonable to me. Every time there is a new technology, vendors and consultants and everybody oversells that technology and then there is the inevitable backlash. The key is to keep things aligned with the human learning system.

Bob Pike was the most prominent general-session speaker, and he spoke very knowledgeably about how to deal with difficult participants, and a few other things as well. He then literally went crazy and diagnosed 600 people's personality with just one question. He asked us to choose one of six animals---the one that we most resonated with. He then asked us to get out of our chairs and join our other fellow beavers, dolphins, owls, care bears, and two other animals I can't remember. These six groups were supposed to predict our behavior. It was freakin' unbelievable, and yet most of the participants seemed to buy into it completely. Actually, I probably shouldn't be so bold in that statement because I left the room after he divided us into groups. I sort of had to pee and I knew I wasn't going to miss anything. I returned in five or ten minutes just before we were all allowed to return to our chairs.

In a defense of the participants, it was the last day of the conference, everybody was a bit burned out, many had been up late the night before at the MGM theme park that Elliott rented for participants (too many long falls from the Tower of Terror), and Pike used one of the oldest tricks in the persuasion handbook, introducing the topic as an area of rare scientific inquiry, in this case "The Science of Axiology," developed by Robert S. Hartman. The main idea of this brilliant science is that (and this is a direct quote I think) "something is good when it completely fulfills its characteristics."

What this means I'm not sure. In fact, I can't make sense of it. Let's see, I have many characteristics. What does it mean to fulfill a characteristic? I have brown eyes. How do I fulfill that characteristic? Since brown eyes are supposed to be more suited to climates with lots of sunlight (than blue eyes), can I only be good if I live near the equator? I have a tongue. Tongues can do many things. Must I do them all each day to be good? My nose can smell and help me breathe, but it can also run with boogers when I have a cold. Must it run each day if I am to be good? It would have to run with boogers if it was going to enable the tongue to fulfill all of its characteristics. I think I'll stop there with the body parts. Maybe Pike meant inanimate objects only. Let's take cars. Maybe cars can only be good if they fulfill the characteristic of reaching 120 miles per hour, which most are capable of. My car has never reached that speed, so it must be bad. Cars are large objects that go fast. These characteristics make them perfect for killing small animals and children. Can a car only be good if it fulfills its potential for killing (insects on the windshield count to)? Cars also have the characteristic of breaking down. Do cars have to be bad to be good?

I watched Elliott on the Jumbotron at the end of Pike's delusionary sermon. He looked to be in pain. But he saved the show remarkably by thanking Pike for pushing the boundaries of our thinking (or something like that). Nicely done Elliott.

Pike ended in a classy fashion, giving a rather heartfelt plug to Elliott, hailing him as "The Great Connector." It was a touching moment. And it was true. Throughout the conference, we learned again and again how many people Elliott knew. It was quite amazing, and impressive, really.

And then, the show was over, and everyone went home.

in a later post, I'm going to rate the Learning 2005 conference along several dimensions. Overall, it was one of the better conferences I attended, but it is not near to where our conferences need to be to really generate useful learning.

Learning 2005 Review of Day 3

One of the things Learning 2005 is experimenting with is a wiki, ala wikipedia, the citizen-created encyclopedia. The idea is that we can all help each other learn by having one place to comment on a specific topic. The Learning 2005 wiki is available to all. Check it out. I will include some of the specific links to the sessions I attended today so you can see whether it's working or not. Remember, that it may take some time for folks to generate comments.

Malcolm Gladwell, author of The Tipping Point and the current best seller, Blink, was the highlight of the day for me. I'd read and enjoyed Tipping Point, but hadn't read Blink because I heard some reviews of the book that suggested it oversold the power of intuition. Elliott Masie's interview with Gladwell was just stunning in its ability to showcase Gladwell's clarity of mind. It's not that the concepts were new---as a doctoral student in cognitive/educational psychology I learned about these basic cognitive concepts 15 years ago---but nevertheless, Gladwell's gift for simplicity and metaphor and example pushed the learning happily into my head. And judging by the book-signing line, which lasted at least half an hour, many others learned a lot too.

One of the examples Gladwell used was about driving. We get better at driving with practice because practice gives us the ability to automate the cognitive processing needed to evaluate the environmental stimuli and respond appropriately. Gladwell calls this automatic processing "thinslicing." He asked us to think about teenagers driving a car. Teenagers have better reflexes than any other age group, but they're lousy drivers (and get in many more car accidents than older people) because they haven't had enough practice to develop automatic thinking as drivers. Despite their very-fast reflexes, teenagers are actually really slow in making driving decisions because they don't have experience driving. It's the same in other areas of endeavor as well. "Experts get better at thinslicing as they get more and more experience," said Gladwell.

Gladwell offered many suggestions for performance improvement. He was quite dismayed that all the emphasis in automobile safety is focused on car design, when it could better be focused on training drivers better. He argued that we have the learning designs that could radically improve driving performance---simulators along the lines of what airplane pilots practice in---but we don't use these methodologies.

Malcolm's discussion reminded me of the statistics that show more accidents for drivers who use cell phones. Assuming the relationship is causal, and not just correlational, I wonder whether the higher accident rate is simply a function of all of us needing to develop automatic processing skills in how to use a cell phone and drive at the same time.

A few other interesting riffs from Malcolm Gladwell:

• "I like music in the abstract, but I don't have time for music."
• "The hiring processes we use should be specific for each job. For example, in hiring a perfume retail clerk, we ought to rate them on the impression we get from the first two seconds we see them. Interviews may be irrelevant for many hiring decisions. In hiring professors, we now hire them based on high-stakes presentations, but we should probably just read their articles, talk with the people who have worked with them, talk to their former students."
• "New mothers have poor intuitions about their babies (because they're inexperienced)."
• "Googling is fine, but it's not enough, especially for someone like me who is in an arms race with other thinkers and writers to create unique and important insights. If I can Google it, so can everybody else."

Elliott got Steve Johnson, author of Everything Bad is Good for You, to join Gladwell, and Johnson said a couple very interesting things that seem to have relevance in the design of learning. All my quotations are paraphrases since I'm not a verbatim note-taker.

• "Games are really good at motivating players to persevere through multiple presentations of concepts. One of the reasons is that they are aware of the user's proficiency."
• "Computers are really good today at pushing us toward multi-tasking, but sometimes we really want to be doing single-tasking, and software designers are just beginning to learn how to build this type of competency into their programs."

An aside. Elliott is not only great at doing interviews to get his keynote speakers to focus on what is important to the conference participants, but we all ought to thank him for choosing keynote speakers who have something to tell us about learning and performance. Elliott is fond of challenging the folks who join him onstage, asking them directly to champion specific change efforts going forward. By choosing his speakers so wisely, I hope he has thrown down the challenge to ASTD and other industry organizations to avoid celebrities who have little to contribute to our cause. There are other organizations that avoid the off-topic celebrities (the elearningguild comes to mind), but it's a good reminder for all of us.

I attended another great session on---and you may not believe this---but it was on corporate Learning/Training Research and Development departments. Yes, that's right. It appears that some companies actually have R&D departments within their training and development apparti. The session was led by two folks from PriceWaterhouseCoopers, but there were many other R&D practitioners in the room. As an upcoming area of interest, these departments are still finding their way, but they are often charged with evaluating technological innovations to determine whether they have merit. For example, in talking with the director of learning R&D at 3M after the session, he said, "I want to be able to separate the cool from that which is truly effective." I'm thrilled with this development, and a group of us from the session intend to create a special interest group. I'll report more on this later after I see how this takes off, or feel free to check out the session wiki.

Elliott interviewed six Chief Learning Officers during lunch. Do you get the sense that Elliott was everywhere? The panelists were great. And Elliott left them (and all of us audience members) the following challenge to commit to doing more of over the next year or so:

1. Research
2. Metrics
3. Innovation

I couldn't agree more!!

I'd only add that we ought to remember the human learning-and-performance system in all this effort.

Another session I attended was an audience-generated discussion of Small Hidden Learning Chunks. It was fascinating because as the session got started, lots of people walked out. But as the discussion got going it was clear that the large number of people who stayed were really smart, knowledgable, thoughtful, and were willing to deal happily with the ambiguities that were surfaced in the discussion. Whether this was because of the self-selected group, or something else, I'm not sure. We didn't come to conclusions, but we generated a great brainstorm of ideas for which we may be able to riff with in the future. Ahhhhh, the wonder of the many types of learning. By the way, I learned at least two interesting terms in this session, including the term folksonomy, which I intend to look up, and Blog Fog, which I think I already understand (see my very first post). Wiki here.

Another session I attended---almost against my will (but I went to challenge my assumptions)---was about what the youngest adult learners want from training and learning. It was another well-designed audience-generated discussion. It started with a review of some kind of research on this "Millenial Generation." I can't vouch for the research or against it and I came in too late to hear the source. We split into teams to generate lists of the implications of the research. Reasonable lists were generated.

I'd like to comment on the design of the session. First, this would be an excellent design if we could trust the research that was cited and we all left knowing that we had just developed a list of hypotheses that we should all go back and test. Unfortunately, I'm guessing most people in the room trusted the research and will go back ready to implement it. If the research is bad, or the implications are right generally, but wrong in a particular context, trouble will ensue. What we should have said to ourselves is, "We've had some fun and brainstormed some ideas, but we now have some due-diligence to do. We need to verify that the research has a good methodology and is generalizable to our context. If it isn't we should do better research with our own target audience of young people. If the research is good, we should still do some prototyping and testing of our designs to see if they will really work with our folks. We should make improvements based on our real-world results. And we have to remember, not to just look at how our millenial colleagues like the learning designs, but how well they learn from those designs, and how effective those designs are at improving the millenials on-the-job performance." Wiki here.

Kathleen Kennedy-Townsend was one of the afternoon keynote-interviewees. Former lieutenant governor of Maryland, she has been a national advocate for better education for kids. What struck me most in her comments---probably because I'm the father of a 2.5 year old---was her description of how important learning and courage were in her family. She's the daughter of Robert Kennedy, former Senator and Secretary of Defense under his brother John F. Kennedy. Whether we agree with their politics or not, we have to agree that the Kennedy family has been very influential in American society and politics. When she was growing up, their grandmother would return their letters with red corrections. They'd be quizzed about current events during dinner. Instead of being sent off to daycare, Kennedy-Townsend remembers her siblings and her being taken to Senate hearings when they were 3 years old. Courage was always emphasized. It was always expected that you would try to make a difference in what you did. As I heard these comments, I wanted her to write a book on child-rearing.

That's it for Day 3. Apologies of the lack of pictures, but I'm ready for sleep.

Learning 2005 Review of Day 2

The first full day of the conference, and what I wanted to know most was whether Learning 2005 would reach the goals Elliott Masie had set for it. Would it:

1. Enable high levels of audience participation and knowledge creation.
2. Avoid vendor-sponsored sales pitches.
3. Move away from PowerPoint.

On all three counts it failed, but I think I was asking the wrong question. A better question might have been, "Yo Will, did you learn anything?" On that count Day 2 at Learning 2005 was a great success!

The first session I attended was on Myths in our industry. This was the most audience-oriented session I attended in the day, but something seemed to be missing. The audience created a list of myths that were typed and posted on a big screen. We talked about the myths. Unfortunately, we didn't all agree with the myths, and the discussions generated around each myth were not very enlightening.

The second session I attended was led by Michael Allen, author of Michael Allen's Guide to E-Learning, and founder of Allen Interactions. Michael broke the "No PowerPoint" rule, but it was damn well worth it with slides like the one showing "an honest e-learning menu" (at left). Although Michael had a few introductory words, his wisdom and warmth created the perfect environment for an intelligent discussion. His question to prompt audience participation was "Top ten things we know about e-learning?" The audience discussion was fascinating, leading to a series of analogies of e-learning as diet-food, "Where the nutrition and fun is stripped away, but people buy it because they feel they have to" and fast-food, "Where learners are enticed by a meal that satisfies superficially but delivers only empty calories." Someone cited the hopeful sign that the fast food industry has begun to improve its food choices, and so eventually the e-learning field may improve its product as well. Can we wait 50 years like we've had to do in McDonald-Land? I don't think so.

Michael finished the session with a brilliantly elegant demo of an e-learning course Allen Interactions built for Apple on sexual harassment. No video. No audio. Just extremely well-designed scenarios. Apple folks in the room attested to the unusually high marks Apple's finicky employees have given the program. "They never do this, but we've been receiving emails by the tons from employees who found the program really valuable and enjoyable." The trick to this product's success may have been in the needs-analysis research that the designers did. They found out that (1) most people already think they know about sexual harassment and how to handle it, and (2) they don't know. The design incorporated scenarios that proved to the learners that they really didn't know. My only disappointment with this part of the talk, is that I seemed to be only one of the few who found any irony in the fact that it was Arnold Schwarzenegger (the gropinator) who mandated two-hours of sexual harassment training for all California companies.

The next session was an Elliott-led keynote. As it turns out all the keynotes are facilitated by Elliott. And miraculously, that's a really good thing. Elliott has a real talent as a presenter. He keeps things moving. He incorporates humor, sideshows, fancy technology, and a human touch. He also prevents keynote speakers from delivering canned monologues. Elliott basically interviews everyone. And most of the time it works great.

In the first keynote of the day, one of the most notable interchanges involved Vice Admiral Moran of the Navy. Elliott gave him one of only a handful of Learning Pioneer awards distributed at the conference. Moran was very impressive and received the longest sustained applause of the day. His most stunning pronouncement was when he said that the U.S. military constantly had to compete with one of the best learning organizations in the world, Al Qaida. "Al Qaida improves their tactics every ten days. We have to be better than the competition. We have to learn faster than they do." Moran and his team developed ways to get information from the field---for example after a roadside bomb detonates---bring that information back stateside, figure out what it means and then send that information back to the soldiers in the field.

The next regular session I attended was led by one of the learning industry's most expansive thinkers, Jay Cross, who is writing a book on informal learning. Although sometimes I find Jay's ideas completely nuts, I like his work because it pushes against our old boundaries and forces us to rethink what we're doing. Jay is a collector of ideas and inspirations. In fact, his talk got me thinking about informal learning and how to make sense of what it is and what it can do for us. Here is a sampling of those ideas:

1. One key distinction is between intentional and non-intentional informal learning. Non-intentional learning is important because humans are cue-reactive entities. The question is can we design learning and workplace situations to prompt informal learning.
2. How much can we use social traditions or human tendencies to engender group-enabled learning. For example, if I know Sally is interested in learning objects, I can send her articles on that topic that I encounter. Can we build technologies or ways of working that prompt this sort of thing?
3. How to we handle the information-validation problem when learning happens from the ground up? What ideas do we trust? What ideas do we discard? How to we build structures into out informal learning to facilitate the rise of the good information?
4. Jay presented a pie-chart, which I see everywhere, showing how formal learning is just a small part of the pie and informal learning is the dominant form of learning. I'd like to know how that data was created. How does one measure such a thing? But even if accepted at face value, we need to know how much of informal learning is under our control ("our" could be us as learning designers, or us as learners). One person in the audience wondered whether we should worry that creating intentional informal learning might hurt the more organic informal learning.
5. How do we measure this beast? (So that we're sure we are making things better, not worse).
6. Jay offered an interesting thought (which I concur with) that experts aren't always the best teachers because they may not know how they do what they do, they may be so advanced that they can't bring it down to the level that it would be understood, etc. Jay's implication was that the masses will do fine in creating information, but maybe the expert is needed in this process at some point.
7. Can we prime learners to learn in future situations when they encounter relevant stimuli in their environments? Obviously we can. Research on learning objectives demonstrates that in formal learning situations---where they actually prompt attention on relevant stimuli. And research I did as a grad student at Columbia (research on infogoals---information acquisition goals) found that priming people with questions about certain objects in their day to day environments, spurred them to learn more about those objects in the week following the questioning.

By the way, Jay was the only presenter, besides Elliott, to avoid PowerPoint's, which you can verify in the picture.

During the next session slot, I experienced two horrible sessions, having to leave the first session halfway through the time. It was simply a sales presentation with no audience participation at all.

The second session scared me even more, where the topic was compliance training, and one of the panelists said she really didn't care if one of her learners got help taking the assessment.

I asked a question about whether the compliance assessments were delivered immediately at the end of learning, and all the panelists said yes. They understood the dangers of that, but couldn't conceive that it would be logistically possible to do it any other way. These were intelligent folks and I had a chance to talk one-on-one with several of them after the session. They helped me understand how difficult it would be, but I hold out hope that we can brainstorm a way out of this fix. The bottom line is that a test immediately after learning cannot predict how well the learners will be able to remember the information on the job.

I think I'm discovering that too much took place today to write about it so thoroughly. And there is sleep that is needed. Anyway, let me highlight a few other things.

Grant Thornton, the fifth-largest accounting firm, won an award because they changed the way they choose their partners (the owners of the firm). Instead of interviews and secret decision-making meetings, they opened up the process and invited potential partner candidates into a "Partner Candidate Program" where they provide the candidates with learning-and-performance opportunities. It has been a great success and one of the key reasons is that it enables the current partners to "get many more data points on each candidate." A great idea, especially when we know that interviewing is inherently a poor selection method.

Steve Johnson, author of Everything Bad is Good for You, gave insightful perspective on online gaming. For example, I wouldn't have guessed that the number one demographic segment that plays games online is women aged 40 to 60. They play board games and card games online. I also didn't really understand the complexity of the video games that kids play these days, or how very few of the most popular games are actually violent in nature.

Frances Hesselbein was also a keynote speaker and we sang happy birthday to her twice. Elliott did a wonderful job of connecting on a human level to this ancient leadership guru. She was funny and wise, and the whole interaction reminded us all that we stand on the shoulders of giants.

To summarize my learning day, I was humbled at how much there is to know. And thanks to Elliott and the awards he distributed, the donations he made to charitable organizations, the way he reminded us of the human element in what we do, I was proud to be a part of our learning community and inspired to keep up the hard work.

And yes, that is Elliott on his Segway.

Learning 2005 by Elliott Masie

For the Learning 2005 conference, starting today in Orlando within the movie-set sterility the Disney empire, Elliott Masie of the Masie Center has promised a new type of training-industry conference. No Powerpoints. No exhibit halls. No more conference sessions dominated by vendors and consultants. Instead, Elliott has promised to experiment with the medium, create a community-dominated discussion where conference-goers can learn from each other.

Will this effort at innovation succeed or fall flat?

Certainly, most conferences in the training and e-learning space are a mixed bag. Some excellent sessions. Some engaging. Some spewing misinformation. Some spewing platitudes. But the real problem with most conferences is that there is no way to validate the information learned---and since the information is largely vendor-driven, the information is a bit suspect in the first place.

Can a "1500-heads-are-better-than-one" format work? We shall see. I have my doubts. How will we know whether an idea put forward is a good one or not? How will we know whether the person with the idea is a genius or a nut?

In preparing to come to the conference, I have been impressed with the meeting and interactive technologies in place, though I admit to not fully understanding them. Learning 2005 has a "learning wiki" enabling conference participants to connect with each other, comment on the sessions, etc. As of today however, not much communication has occurred between the participants. This should increase once the sessions plant the seeds for discussion, but not too much prework has been done.

In tonight's Keynote, Elliott told some jokes, talked to a humorous computer-generated talking head (very impressive if really computer-generated, but I'm betting on a human comedian behind the funny banter), interviewed a 19-year old intern to show us how different this generation is (one data point, isn't it?), gave money and awards to a couple of non-profits, gave an award to CNN for their innovative learning design (which seemed to be for one-on-one coaching, but couldn't really be, could it?), and talked with the Chairman and Founder of Boston Scientific, John Abele, who offered the best learning tidbits of the evening.

Abele started by answering Elliott's query about why CEO's worry about learning by saying that it is very simple from a business perspective, "If we can't do learning better and faster, someone else will." Twice Abele mentioned the importance of give and take to get to wisdom. He talked about a live-demonstration course Boston Scientific developed to change the way the marketplace (doctors and medical institutions) viewed the company's new medical methods. The amazing thing about the course was that they invited the world's best surgeons to witness real surgeries and vote on what to do next by using a vote-response system. One example he cited was asking the doctors whether the medical device ought to be inserted half an inch more into the heart. The discussions and arguments that went on were a great learning vehicle. To augment these sessions, the commentators were chosen for being contrarians.

Abele also gave us the audience advice on choosing a doctor if we're going in for surgery. Ask the doctor what research papers show the benefits of the surgery, and what research papers show that the procedure doesn't work that well. Again, the benefits of experts fighting it out.

Finally, Abele talked about how doctors these days are beginning to do simulated surgeries on real people. If you have an abdominal aneurysms, your doctor may "take pictures" of the aneurysm and then basically perform your surgery on your images before he or she does surgery on you. What a great way of aligning the learning and performance contexts, a research topic I've written about many times.

Elliott's main theme in his remarks is that the world is faster and more confusing then ever. Learning design must move from 18 weeks to 18 hours. That's almost an exact quote, by the way.

The keynote was two hours long, but felt longer. Still, it was much better than listening to some celebrity deliver a canned speech, with no learning content to speak of.

I give Elliott lots of credit for this experiment. Whether it's a noble effort, an ingenious publicity stunt, or both, I'm looking forward to the sunrise when we get down to the audience-generated learning.