Will at Work Learning

Learning 2005 Conference Critique

Learning 2005 was a very good conference, but as is typical in our industry (training & development) this conference still lacked some fundamental elements. Some of the good and bad points of the conference are detailed below. They are provided to encourage changes in all future industry conferences.

The Good:

1. High energy and excitement, created by Elliott and his pre-conference emailings, Elliott's energetic facilitation of the general sessions, and the experimental use of new technologies.
2. Taking risks and trying new models for conferences.
3. Good opportunity to spur future thinking on leading edge topics. For example, everyone at the conference will be likely to consider the use of wiki's.
4. The strong encouragement to presenters to be facilitators instead of lecturers created some really good sessions. It also helped a few participants feel free to criticize (or be skeptical of) the ideas of others. This is a good thing, though more of this is necessary.
5. The human element rose higher than at any other conference in the industry. Not-for-profit groups were highlighted. People's good works were celebrated with formal "Pioneers of Learning" awards. Elliott's empathy and general-session conversations channeled a sense of relationship with the speakers and a sense of community.
6. The keynotes actually has something to say about learning and performance. Yippee!!
7. Elliott's strong emphasis on research, metrics, and innovation.
8. The sense that technological innovations were about to really get traction. A tipping point toward better e-learning design.
9. Malcolm Gladwell was a clear and thoughtful thinker!!
10. The Masie Center's efforts to organize the group "The Learning Consortium" provided a sense that we were all in this together with the goal of helping each other.
11. The participants seemed very knowledgable and thoughtful (at least in the sessions I attended).
12. The wiki at www.learningwiki.com provides a nice opportunity for discussion and further community building. It also provides participants with a way connect with sessions/presenters who they missed because two or more sessions of interest were scheduled at the same time. It remains to see how this will work, but it is certainly worth the experiment. We as conference-goers may need to learn how to use this technology to get the most out of it.
13. The Masie Center people are really nice and really helpful.
14. Elliott didn't let the keynote speakers simply deliver their canned speeches. He forced them to focus on what was most relevant by interviewing them. Very effective.
15. The conference was fun!!

The Neutral

1. No exhibition hall. Good and bad. Good because sponsors didn't show up just to sell. Bad because the selling took a more stealthy approach. Bad because sometimes it is nice as a participant to seek out potential vendors and have real conversations with them.
2. The recommendation NOT to use PowerPoints is both good and bad. Sometimes visuals are helpful to make the point, clarify potential confusions, etc. PowerPoint is a tool. It is good for some things and bad for others. For example, I would have loved to see a brief PowerPoint when Elliott invited people up on stage so that I could catch their names. This seems respectful as well as a good learning design. Also, many presenters did not follow the recommendation and used PowerPoints anyway.

The Bad

1. Commercial interests still dominated many conversations. Some sponsors paid as much as $15,000 to lead three conference sessions. Although some sponsors led excellent sessions, others gave thinly-disguised sales pitches. If vendors dominate the conversational agenda, we diminish the wisdom of experts and we diminish the credibility of our field. Vendors should certainly be a part of the conversation, because many are leading the way, but they shouldn't be the most powerful voice.
2. Some of the sessions floundered under the goal of creating audience-generated discussions. Some of the facilatators were unskilled or talked too much. Some were simply new at facilitating, having more experience in traditional conference sessions delivering their presentations. Some sessions, while great as brainstorming sessions, were unencumbered by the hard work of separating the good ideas from the bad ones.
3. There was not enough planned skepticism, making it likely that some bad ideas were accepted as good ideas. This is a major failing in our industry and one of the reasons why we jump from one fad to another. We are simply not skeptical enough and our conferences don't encourage skepticism.
4. It would be nice to have each presenter asked a mandatory set of questions, including (a) what evidence do you have that your recommendations worked in your situation, (b) what evidence or research can you cite that shows that your recommendations are likely to work in other situations, (c) what research supports the basic concepts you've implemented, and/or (d) what negative and positive side effects might your recommendations create?
5. Sometimes the gee-whiz factor went way too far, as when everybody got real excited just because a technology was new.
6. There was very little discussion of the human learning system and how we needed to align our learning designs with it.
7. The price was rather high for the conference admission.
8. Disney's hermetically-sealed hotel rooms, ingrained with toxic cleaning products, did not provide a healthy environment for sleeping or hanging out.

Learning 2005 Review of Day 4

Yesterday was the final day of the conference. In the final general session, one of the speakers, Mike somebody (sorry, but there were no overheads to announce the general session speakers), warned the audience not to oversell Extreme Learning, Elliott's term for pushing the technological boundaries and creating very short and quick learning episodes. Actually, now that I think about it, "Extreme Learning" is not really that clear. I guess it has to do with technical innovation. Anyway, the warning seems reasonable to me. Every time there is a new technology, vendors and consultants and everybody oversells that technology and then there is the inevitable backlash. The key is to keep things aligned with the human learning system.

Bob Pike was the most prominent general-session speaker, and he spoke very knowledgeably about how to deal with difficult participants, and a few other things as well. He then literally went crazy and diagnosed 600 people's personality with just one question. He asked us to choose one of six animals---the one that we most resonated with. He then asked us to get out of our chairs and join our other fellow beavers, dolphins, owls, care bears, and two other animals I can't remember. These six groups were supposed to predict our behavior. It was freakin' unbelievable, and yet most of the participants seemed to buy into it completely. Actually, I probably shouldn't be so bold in that statement because I left the room after he divided us into groups. I sort of had to pee and I knew I wasn't going to miss anything. I returned in five or ten minutes just before we were all allowed to return to our chairs.

In a defense of the participants, it was the last day of the conference, everybody was a bit burned out, many had been up late the night before at the MGM theme park that Elliott rented for participants (too many long falls from the Tower of Terror), and Pike used one of the oldest tricks in the persuasion handbook, introducing the topic as an area of rare scientific inquiry, in this case "The Science of Axiology," developed by Robert S. Hartman. The main idea of this brilliant science is that (and this is a direct quote I think) "something is good when it completely fulfills its characteristics."

What this means I'm not sure. In fact, I can't make sense of it. Let's see, I have many characteristics. What does it mean to fulfill a characteristic? I have brown eyes. How do I fulfill that characteristic? Since brown eyes are supposed to be more suited to climates with lots of sunlight (than blue eyes), can I only be good if I live near the equator? I have a tongue. Tongues can do many things. Must I do them all each day to be good? My nose can smell and help me breathe, but it can also run with boogers when I have a cold. Must it run each day if I am to be good? It would have to run with boogers if it was going to enable the tongue to fulfill all of its characteristics. I think I'll stop there with the body parts. Maybe Pike meant inanimate objects only. Let's take cars. Maybe cars can only be good if they fulfill the characteristic of reaching 120 miles per hour, which most are capable of. My car has never reached that speed, so it must be bad. Cars are large objects that go fast. These characteristics make them perfect for killing small animals and children. Can a car only be good if it fulfills its potential for killing (insects on the windshield count to)? Cars also have the characteristic of breaking down. Do cars have to be bad to be good?

I watched Elliott on the Jumbotron at the end of Pike's delusionary sermon. He looked to be in pain. But he saved the show remarkably by thanking Pike for pushing the boundaries of our thinking (or something like that). Nicely done Elliott.

Pike ended in a classy fashion, giving a rather heartfelt plug to Elliott, hailing him as "The Great Connector." It was a touching moment. And it was true. Throughout the conference, we learned again and again how many people Elliott knew. It was quite amazing, and impressive, really.

And then, the show was over, and everyone went home.

in a later post, I'm going to rate the Learning 2005 conference along several dimensions. Overall, it was one of the better conferences I attended, but it is not near to where our conferences need to be to really generate useful learning.

Learning 2005 Review of Day 3

One of the things Learning 2005 is experimenting with is a wiki, ala wikipedia, the citizen-created encyclopedia. The idea is that we can all help each other learn by having one place to comment on a specific topic. The Learning 2005 wiki is available to all. Check it out. I will include some of the specific links to the sessions I attended today so you can see whether it's working or not. Remember, that it may take some time for folks to generate comments.

Malcolm Gladwell, author of The Tipping Point and the current best seller, Blink, was the highlight of the day for me. I'd read and enjoyed Tipping Point, but hadn't read Blink because I heard some reviews of the book that suggested it oversold the power of intuition. Elliott Masie's interview with Gladwell was just stunning in its ability to showcase Gladwell's clarity of mind. It's not that the concepts were new---as a doctoral student in cognitive/educational psychology I learned about these basic cognitive concepts 15 years ago---but nevertheless, Gladwell's gift for simplicity and metaphor and example pushed the learning happily into my head. And judging by the book-signing line, which lasted at least half an hour, many others learned a lot too.

One of the examples Gladwell used was about driving. We get better at driving with practice because practice gives us the ability to automate the cognitive processing needed to evaluate the environmental stimuli and respond appropriately. Gladwell calls this automatic processing "thinslicing." He asked us to think about teenagers driving a car. Teenagers have better reflexes than any other age group, but they're lousy drivers (and get in many more car accidents than older people) because they haven't had enough practice to develop automatic thinking as drivers. Despite their very-fast reflexes, teenagers are actually really slow in making driving decisions because they don't have experience driving. It's the same in other areas of endeavor as well. "Experts get better at thinslicing as they get more and more experience," said Gladwell.

Gladwell offered many suggestions for performance improvement. He was quite dismayed that all the emphasis in automobile safety is focused on car design, when it could better be focused on training drivers better. He argued that we have the learning designs that could radically improve driving performance---simulators along the lines of what airplane pilots practice in---but we don't use these methodologies.

Malcolm's discussion reminded me of the statistics that show more accidents for drivers who use cell phones. Assuming the relationship is causal, and not just correlational, I wonder whether the higher accident rate is simply a function of all of us needing to develop automatic processing skills in how to use a cell phone and drive at the same time.

A few other interesting riffs from Malcolm Gladwell:

• "I like music in the abstract, but I don't have time for music."
• "The hiring processes we use should be specific for each job. For example, in hiring a perfume retail clerk, we ought to rate them on the impression we get from the first two seconds we see them. Interviews may be irrelevant for many hiring decisions. In hiring professors, we now hire them based on high-stakes presentations, but we should probably just read their articles, talk with the people who have worked with them, talk to their former students."
• "New mothers have poor intuitions about their babies (because they're inexperienced)."
• "Googling is fine, but it's not enough, especially for someone like me who is in an arms race with other thinkers and writers to create unique and important insights. If I can Google it, so can everybody else."

Elliott got Steve Johnson, author of Everything Bad is Good for You, to join Gladwell, and Johnson said a couple very interesting things that seem to have relevance in the design of learning. All my quotations are paraphrases since I'm not a verbatim note-taker.

• "Games are really good at motivating players to persevere through multiple presentations of concepts. One of the reasons is that they are aware of the user's proficiency."
• "Computers are really good today at pushing us toward multi-tasking, but sometimes we really want to be doing single-tasking, and software designers are just beginning to learn how to build this type of competency into their programs."

An aside. Elliott is not only great at doing interviews to get his keynote speakers to focus on what is important to the conference participants, but we all ought to thank him for choosing keynote speakers who have something to tell us about learning and performance. Elliott is fond of challenging the folks who join him onstage, asking them directly to champion specific change efforts going forward. By choosing his speakers so wisely, I hope he has thrown down the challenge to ASTD and other industry organizations to avoid celebrities who have little to contribute to our cause. There are other organizations that avoid the off-topic celebrities (the elearningguild comes to mind), but it's a good reminder for all of us.

I attended another great session on---and you may not believe this---but it was on corporate Learning/Training Research and Development departments. Yes, that's right. It appears that some companies actually have R&D departments within their training and development apparti. The session was led by two folks from PriceWaterhouseCoopers, but there were many other R&D practitioners in the room. As an upcoming area of interest, these departments are still finding their way, but they are often charged with evaluating technological innovations to determine whether they have merit. For example, in talking with the director of learning R&D at 3M after the session, he said, "I want to be able to separate the cool from that which is truly effective." I'm thrilled with this development, and a group of us from the session intend to create a special interest group. I'll report more on this later after I see how this takes off, or feel free to check out the session wiki.

Elliott interviewed six Chief Learning Officers during lunch. Do you get the sense that Elliott was everywhere? The panelists were great. And Elliott left them (and all of us audience members) the following challenge to commit to doing more of over the next year or so:

1. Research
2. Metrics
3. Innovation

I couldn't agree more!!

I'd only add that we ought to remember the human learning-and-performance system in all this effort.

Another session I attended was an audience-generated discussion of Small Hidden Learning Chunks. It was fascinating because as the session got started, lots of people walked out. But as the discussion got going it was clear that the large number of people who stayed were really smart, knowledgable, thoughtful, and were willing to deal happily with the ambiguities that were surfaced in the discussion. Whether this was because of the self-selected group, or something else, I'm not sure. We didn't come to conclusions, but we generated a great brainstorm of ideas for which we may be able to riff with in the future. Ahhhhh, the wonder of the many types of learning. By the way, I learned at least two interesting terms in this session, including the term folksonomy, which I intend to look up, and Blog Fog, which I think I already understand (see my very first post). Wiki here.

Another session I attended---almost against my will (but I went to challenge my assumptions)---was about what the youngest adult learners want from training and learning. It was another well-designed audience-generated discussion. It started with a review of some kind of research on this "Millenial Generation." I can't vouch for the research or against it and I came in too late to hear the source. We split into teams to generate lists of the implications of the research. Reasonable lists were generated.

I'd like to comment on the design of the session. First, this would be an excellent design if we could trust the research that was cited and we all left knowing that we had just developed a list of hypotheses that we should all go back and test. Unfortunately, I'm guessing most people in the room trusted the research and will go back ready to implement it. If the research is bad, or the implications are right generally, but wrong in a particular context, trouble will ensue. What we should have said to ourselves is, "We've had some fun and brainstormed some ideas, but we now have some due-diligence to do. We need to verify that the research has a good methodology and is generalizable to our context. If it isn't we should do better research with our own target audience of young people. If the research is good, we should still do some prototyping and testing of our designs to see if they will really work with our folks. We should make improvements based on our real-world results. And we have to remember, not to just look at how our millenial colleagues like the learning designs, but how well they learn from those designs, and how effective those designs are at improving the millenials on-the-job performance." Wiki here.

Kathleen Kennedy-Townsend was one of the afternoon keynote-interviewees. Former lieutenant governor of Maryland, she has been a national advocate for better education for kids. What struck me most in her comments---probably because I'm the father of a 2.5 year old---was her description of how important learning and courage were in her family. She's the daughter of Robert Kennedy, former Senator and Secretary of Defense under his brother John F. Kennedy. Whether we agree with their politics or not, we have to agree that the Kennedy family has been very influential in American society and politics. When she was growing up, their grandmother would return their letters with red corrections. They'd be quizzed about current events during dinner. Instead of being sent off to daycare, Kennedy-Townsend remembers her siblings and her being taken to Senate hearings when they were 3 years old. Courage was always emphasized. It was always expected that you would try to make a difference in what you did. As I heard these comments, I wanted her to write a book on child-rearing.

That's it for Day 3. Apologies of the lack of pictures, but I'm ready for sleep.

Learning 2005 Review of Day 2

The first full day of the conference, and what I wanted to know most was whether Learning 2005 would reach the goals Elliott Masie had set for it. Would it:

1. Enable high levels of audience participation and knowledge creation.
2. Avoid vendor-sponsored sales pitches.
3. Move away from PowerPoint.

On all three counts it failed, but I think I was asking the wrong question. A better question might have been, "Yo Will, did you learn anything?" On that count Day 2 at Learning 2005 was a great success!

The first session I attended was on Myths in our industry. This was the most audience-oriented session I attended in the day, but something seemed to be missing. The audience created a list of myths that were typed and posted on a big screen. We talked about the myths. Unfortunately, we didn't all agree with the myths, and the discussions generated around each myth were not very enlightening.

The second session I attended was led by Michael Allen, author of Michael Allen's Guide to E-Learning, and founder of Allen Interactions. Michael broke the "No PowerPoint" rule, but it was damn well worth it with slides like the one showing "an honest e-learning menu" (at left). Although Michael had a few introductory words, his wisdom and warmth created the perfect environment for an intelligent discussion. His question to prompt audience participation was "Top ten things we know about e-learning?" The audience discussion was fascinating, leading to a series of analogies of e-learning as diet-food, "Where the nutrition and fun is stripped away, but people buy it because they feel they have to" and fast-food, "Where learners are enticed by a meal that satisfies superficially but delivers only empty calories." Someone cited the hopeful sign that the fast food industry has begun to improve its food choices, and so eventually the e-learning field may improve its product as well. Can we wait 50 years like we've had to do in McDonald-Land? I don't think so.

Michael finished the session with a brilliantly elegant demo of an e-learning course Allen Interactions built for Apple on sexual harassment. No video. No audio. Just extremely well-designed scenarios. Apple folks in the room attested to the unusually high marks Apple's finicky employees have given the program. "They never do this, but we've been receiving emails by the tons from employees who found the program really valuable and enjoyable." The trick to this product's success may have been in the needs-analysis research that the designers did. They found out that (1) most people already think they know about sexual harassment and how to handle it, and (2) they don't know. The design incorporated scenarios that proved to the learners that they really didn't know. My only disappointment with this part of the talk, is that I seemed to be only one of the few who found any irony in the fact that it was Arnold Schwarzenegger (the gropinator) who mandated two-hours of sexual harassment training for all California companies.

The next session was an Elliott-led keynote. As it turns out all the keynotes are facilitated by Elliott. And miraculously, that's a really good thing. Elliott has a real talent as a presenter. He keeps things moving. He incorporates humor, sideshows, fancy technology, and a human touch. He also prevents keynote speakers from delivering canned monologues. Elliott basically interviews everyone. And most of the time it works great.

In the first keynote of the day, one of the most notable interchanges involved Vice Admiral Moran of the Navy. Elliott gave him one of only a handful of Learning Pioneer awards distributed at the conference. Moran was very impressive and received the longest sustained applause of the day. His most stunning pronouncement was when he said that the U.S. military constantly had to compete with one of the best learning organizations in the world, Al Qaida. "Al Qaida improves their tactics every ten days. We have to be better than the competition. We have to learn faster than they do." Moran and his team developed ways to get information from the field---for example after a roadside bomb detonates---bring that information back stateside, figure out what it means and then send that information back to the soldiers in the field.

The next regular session I attended was led by one of the learning industry's most expansive thinkers, Jay Cross, who is writing a book on informal learning. Although sometimes I find Jay's ideas completely nuts, I like his work because it pushes against our old boundaries and forces us to rethink what we're doing. Jay is a collector of ideas and inspirations. In fact, his talk got me thinking about informal learning and how to make sense of what it is and what it can do for us. Here is a sampling of those ideas:

1. One key distinction is between intentional and non-intentional informal learning. Non-intentional learning is important because humans are cue-reactive entities. The question is can we design learning and workplace situations to prompt informal learning.
2. How much can we use social traditions or human tendencies to engender group-enabled learning. For example, if I know Sally is interested in learning objects, I can send her articles on that topic that I encounter. Can we build technologies or ways of working that prompt this sort of thing?
3. How to we handle the information-validation problem when learning happens from the ground up? What ideas do we trust? What ideas do we discard? How to we build structures into out informal learning to facilitate the rise of the good information?
4. Jay presented a pie-chart, which I see everywhere, showing how formal learning is just a small part of the pie and informal learning is the dominant form of learning. I'd like to know how that data was created. How does one measure such a thing? But even if accepted at face value, we need to know how much of informal learning is under our control ("our" could be us as learning designers, or us as learners). One person in the audience wondered whether we should worry that creating intentional informal learning might hurt the more organic informal learning.
5. How do we measure this beast? (So that we're sure we are making things better, not worse).
6. Jay offered an interesting thought (which I concur with) that experts aren't always the best teachers because they may not know how they do what they do, they may be so advanced that they can't bring it down to the level that it would be understood, etc. Jay's implication was that the masses will do fine in creating information, but maybe the expert is needed in this process at some point.
7. Can we prime learners to learn in future situations when they encounter relevant stimuli in their environments? Obviously we can. Research on learning objectives demonstrates that in formal learning situations---where they actually prompt attention on relevant stimuli. And research I did as a grad student at Columbia (research on infogoals---information acquisition goals) found that priming people with questions about certain objects in their day to day environments, spurred them to learn more about those objects in the week following the questioning.

By the way, Jay was the only presenter, besides Elliott, to avoid PowerPoint's, which you can verify in the picture.

During the next session slot, I experienced two horrible sessions, having to leave the first session halfway through the time. It was simply a sales presentation with no audience participation at all.

The second session scared me even more, where the topic was compliance training, and one of the panelists said she really didn't care if one of her learners got help taking the assessment.

I asked a question about whether the compliance assessments were delivered immediately at the end of learning, and all the panelists said yes. They understood the dangers of that, but couldn't conceive that it would be logistically possible to do it any other way. These were intelligent folks and I had a chance to talk one-on-one with several of them after the session. They helped me understand how difficult it would be, but I hold out hope that we can brainstorm a way out of this fix. The bottom line is that a test immediately after learning cannot predict how well the learners will be able to remember the information on the job.

I think I'm discovering that too much took place today to write about it so thoroughly. And there is sleep that is needed. Anyway, let me highlight a few other things.

Grant Thornton, the fifth-largest accounting firm, won an award because they changed the way they choose their partners (the owners of the firm). Instead of interviews and secret decision-making meetings, they opened up the process and invited potential partner candidates into a "Partner Candidate Program" where they provide the candidates with learning-and-performance opportunities. It has been a great success and one of the key reasons is that it enables the current partners to "get many more data points on each candidate." A great idea, especially when we know that interviewing is inherently a poor selection method.

Steve Johnson, author of Everything Bad is Good for You, gave insightful perspective on online gaming. For example, I wouldn't have guessed that the number one demographic segment that plays games online is women aged 40 to 60. They play board games and card games online. I also didn't really understand the complexity of the video games that kids play these days, or how very few of the most popular games are actually violent in nature.

Frances Hesselbein was also a keynote speaker and we sang happy birthday to her twice. Elliott did a wonderful job of connecting on a human level to this ancient leadership guru. She was funny and wise, and the whole interaction reminded us all that we stand on the shoulders of giants.

To summarize my learning day, I was humbled at how much there is to know. And thanks to Elliott and the awards he distributed, the donations he made to charitable organizations, the way he reminded us of the human element in what we do, I was proud to be a part of our learning community and inspired to keep up the hard work.

And yes, that is Elliott on his Segway.

Learning 2005 by Elliott Masie

For the Learning 2005 conference, starting today in Orlando within the movie-set sterility the Disney empire, Elliott Masie of the Masie Center has promised a new type of training-industry conference. No Powerpoints. No exhibit halls. No more conference sessions dominated by vendors and consultants. Instead, Elliott has promised to experiment with the medium, create a community-dominated discussion where conference-goers can learn from each other.

Will this effort at innovation succeed or fall flat?

Certainly, most conferences in the training and e-learning space are a mixed bag. Some excellent sessions. Some engaging. Some spewing misinformation. Some spewing platitudes. But the real problem with most conferences is that there is no way to validate the information learned---and since the information is largely vendor-driven, the information is a bit suspect in the first place.

Can a "1500-heads-are-better-than-one" format work? We shall see. I have my doubts. How will we know whether an idea put forward is a good one or not? How will we know whether the person with the idea is a genius or a nut?

In preparing to come to the conference, I have been impressed with the meeting and interactive technologies in place, though I admit to not fully understanding them. Learning 2005 has a "learning wiki" enabling conference participants to connect with each other, comment on the sessions, etc. As of today however, not much communication has occurred between the participants. This should increase once the sessions plant the seeds for discussion, but not too much prework has been done.

In tonight's Keynote, Elliott told some jokes, talked to a humorous computer-generated talking head (very impressive if really computer-generated, but I'm betting on a human comedian behind the funny banter), interviewed a 19-year old intern to show us how different this generation is (one data point, isn't it?), gave money and awards to a couple of non-profits, gave an award to CNN for their innovative learning design (which seemed to be for one-on-one coaching, but couldn't really be, could it?), and talked with the Chairman and Founder of Boston Scientific, John Abele, who offered the best learning tidbits of the evening.

Abele started by answering Elliott's query about why CEO's worry about learning by saying that it is very simple from a business perspective, "If we can't do learning better and faster, someone else will." Twice Abele mentioned the importance of give and take to get to wisdom. He talked about a live-demonstration course Boston Scientific developed to change the way the marketplace (doctors and medical institutions) viewed the company's new medical methods. The amazing thing about the course was that they invited the world's best surgeons to witness real surgeries and vote on what to do next by using a vote-response system. One example he cited was asking the doctors whether the medical device ought to be inserted half an inch more into the heart. The discussions and arguments that went on were a great learning vehicle. To augment these sessions, the commentators were chosen for being contrarians.

Abele also gave us the audience advice on choosing a doctor if we're going in for surgery. Ask the doctor what research papers show the benefits of the surgery, and what research papers show that the procedure doesn't work that well. Again, the benefits of experts fighting it out.

Finally, Abele talked about how doctors these days are beginning to do simulated surgeries on real people. If you have an abdominal aneurysms, your doctor may "take pictures" of the aneurysm and then basically perform your surgery on your images before he or she does surgery on you. What a great way of aligning the learning and performance contexts, a research topic I've written about many times.

Elliott's main theme in his remarks is that the world is faster and more confusing then ever. Learning design must move from 18 weeks to 18 hours. That's almost an exact quote, by the way.

The keynote was two hours long, but felt longer. Still, it was much better than listening to some celebrity deliver a canned speech, with no learning content to speak of.

I give Elliott lots of credit for this experiment. Whether it's a noble effort, an ingenious publicity stunt, or both, I'm looking forward to the sunrise when we get down to the audience-generated learning.