Learning Styles Instructional-Design Challenge

I will give $1000 (US dollars) to the first person or group who can prove that taking learning styles into account in designing instruction can produce meaningful learning benefits.

I've been suspicious about the learning-styles bandwagon for many years. The learning-style argument has gone something like this: If instructional designers know the learning style of their learners, they can develop material specifically to help those learners, and such extra efforts are worth the trouble.

I have my doubts, but am open to being proven wrong.

Here's the criteria for my Learning-Styles Instructional-Design Challenge:

1. The learning program must diagnose learners' learning styles. It must then provide different learning materials/experiences to those who have different styles.
2. The learning program must be compared against a similar program that does not differentiate the material based on learning styles.
3. The programs must be of similar quality and provide similar information. The only thing that should vary is the learning-styles manipulation.
4. The comparison between the two versions (the learning-style version and the non-learning-style version) must be fair, valid, and reliable. At least 70 learners must be randomly assigned to the two groups (with at least 35 minimum in each group completing the experience). The two programs must have approximately the same running time. For example, the time required by the learning-style program to diagnose learning styles can be used by the non-learning-styles program to deliver learning. The median learning time for the programs must be no shorter than 25 minutes.
5. Learners must be adults involved in a formal workplace training program delivered through a computer program (e-learning or CBT) without a live instructor. This requirement is to ensure the reproducability of the effects, as instructor-led training cannot be precisely reproduced.
6. The learning-style program must be created in an instructional-development shop that is dedicated to creating learning programs for real-world use. Programs developed only for research purposes are excluded. My claim is that real-world instructional design is unlikely to be able to utilize learning styles to create learning gains.
7. The results must be assessed in a manner that is relatively authentic--at a minimum level learners should be asked to make scenario-based decisions or perform activities that simulate the real-world performance the program teaches them to accomplish. Assessments that only ask for information at the knowledge level (e.g., definitions, terminology, labels) are NOT acceptable. The final assessment must be delayed at least one week after the end of the training. The same final assessment must be used for both groups. It must fairly assess the whole learning experience.
8. The magnitude of the difference in results between the learning-style program and the non-learning-style program must be at least 10%. (In other words, the average of the learning-styles scores subtracted by the average of the non-learning-styles scores must be more than 10% of the non-learning-styles scores). So for example, if the non-learning-styles average is 50, then the learning-styles score must be equal to 55 or more. This magnitude is to ensure that the learning-styles program produces meaningful benefits. 10% is not too much to ask.
9. The results must be statistically significant at the p<.05 level. Appropriate statistical procedures must be used to gauge the reliability of the results. Cohen's d effect size should be equal to .4 or more (a small to medium effect size according to Cohen, 1992).
10. The learning-style program cannot cost more than twice as much as the non-learning-style program to develop, nor can it take more than twice as long to develop. I want to be generous here.
11. The results can be documented by unbiased parties.

To reiterate, the challenge is this:

Can an e-learning program that utilizes learning-style information outperform an e-learning program that doesn't utilize such information by 10% or more on a realistic test of learning, even it is allowed to cost up to twice as much to build?

$1,000 says it just doesn't happen in the real-world of instructional design. $1,000 says we ought to stop wasting millions trying to cater to this phantom curse.

Comments

[larry]

I don't think that the distinctions in teaching materials really exists as much as people make of it. It takes a lot of time and trouble to explain the same ideas in several learning styles. I think that you should use learning styles in your presentation (write on the board, use overheads; explain the points clearly, have a discussion and ask questions; have students take notes, make them copy some things you write on the board, have them do a project, etc.) We already do many of these things but they are not done purposely or effectively. The learning style difference I think is for the student to focus on how they take in and process ideas and use those strengths to review the information they learn in class. More concrete examples are needed and explanations of what this ideas 'is like' are necessary to help those who find this subject hard. Lots of programs are being devloped to fit the student but how mmany learning styles do we use - 3 basic, 7 intelligences, other considerations or theories? Lots of trouble for little good results.

[John Lloyd]

Will, nice bit. I'm with you, as documented in these posts. I doubt you'll lose your $1K. However, there may be some passionate challenges, predicated on work by Carbo or Dunn. It'll be interesting to see to what extent your criteria will preclude their demonstrations.

I'm dropping an entry into TeachEffectively! for your post.

[J Haag]

I'm curious to know what kind of response you get. I'm curious to know if any organization has done any post analysis on this front. Most importantly, I have yet to see an elearning implementation that can dynamically adjust learning objectives based on the learner's style. In order for this to happen, the learner would have to have a profile with a track record of past performance (comptency). I like your challenge, but given the current state of elearning I don't think you will find cases comprehensive enough to answer this challenge any time soon.

[clopha deshotel]

All learning so-called styles or pathways to the brain can be reached together in practice. The cognitive side of this activity is hard enough to measure and label that we tend to justify ignoring the affective side... skepticism is good to maintain but multiple conferences are a lot of fun.

[Rich]

If you are so sure of your convictions, why can't you produce your own research to prove it instead of a measly $1K for someone else to do it for you?

You define a pretty narrow range of inquiry from which you want conclude we suffer from a "phantom curse."

You may be right. Prove it.

[Will Thalheimer]

Rich, Thanks for your counter challenge. Unfortunately, it isn't possible to prove the negative. If I do an experiment and find no learning-style effect, it could be rightly argued that other conditions might enable one.

How does one prove there are no black swans? Even if I gather 1 million white swans, I have not proven there are no black swans.

My challenge simply asks for one example of when the learning-styles' notion has produced practical results. If learning styles can really help us build better instructional designs, it should be easy.

By the way, $1,000 may be measly to you, but it's a lot of damn money to me!! Did you notice the tip jar on my blog? Maybe you could contribute? SMILE

NOTES: The black swan question is a well-known philosophical dilemma sometimes mistakenly attributed to David Hume and his writings on inductive logic in 1748.

[Katica Roy]

Great timing! I did an action research study on this a few years ago and it's being published in the November/December 2006 issue of the Perfomance Improvement Journal (http://ispi.org/publications/pij.htm) Here's the title and abstract:
Title: The Impact of Learning Styles on Interactivity in
Asynchronous eLearning Abstract: This action research examines the effect, if any, that learning styles have on interactions in an asynchronous eLearning environment. Through an examination of pertinent literature this research gathers expert opinions on the definition of interactivity in eLearning, learning styles and efforts to accommodate them (especially in eLearning), interactivity devices, and decision making regarding the diversity and number of interactions in eLearning. This research also includes a survey of graduate students on their views of learning styles and their impact on eLearning interactions. As well, three distance learning experts were interviewed to help determine best practices when considering learning styles in the design of asynchronous eLearning interactions. This report concludes with recommendations when considering learning styles in the design and development of asynchronous eLearning activities.

[Harold Jarche]

Will, I think you're going to hang on to your $1K. Even the above abstract doesn't seem to address your challenge.

If I had the extra cash I'd add to your pot.

[Bianca Buckridee]

I don't know if this is still open but here goes. I taught 3rd grade at a Title 1 school and did factor in learning styles into my teaching and saw amazing results. After assessing where each child was on the learning scale, I implemented customized learning paths utilizing powerpoints, iBooks, Leapfrog and online game activities. Each child received instruction on the same concepts however I added/omitted details according to the ability level. I saw amazing results at the ending of the school year. It was an extremely difficult thing to do but well worth it.

[claudia]

So how long is the challenge running for? I will ask if I can take this project on as part of my college course. Are you flexible with the age range?

[Will Thalheimer]

To Bianca,

I'm delighted you found great results by introducing your interventions. However, you don't seem to have a control group to show that it was really the learning-styles intervention that made a difference. For example, it could have been your enthusiasm that made the difference.

To Claudia:

The Challenge will run until we have a winner or until I go bankrupt, whichever comes first. Please go for it. However, note that you'll need to show this effect in a workplace e-learning environment, not in a college classroom.

[Valarie A Washington]

Will, I will take you up on your challenge. As I am writing almost a year past your post I want to make sure that you are still certain of your conviction. Learning styles do matter and it can be demonstrated. I won't tip my hand but you have left out some very important details in your challenge.

I'll meet you here and we can test the results.

[Rutger van de Sande]

Will,
First of all: great website!
Second: I don't understand your arguments for not doing this study yourself as an other contributor proposed. If you believe that learning styles don't matter, why not test this hypothesis yourself? Now that would be 'good science' acoording to Karl Popper. Let's see if you can prove yourself wrong ;o)

Rutger

[Sam H]

Man, I wish I could meet the restrictive parameters of this challenge (sample size, no academic environments).
I have had many opportunities to redesign teaching styles (in prisons, alternative schools and in commerce) to suit the learning styles and characteristics of learners. Just as one example, in prison we had a number of inmates who were 30 years old or more and had never learned to read - in spite of the best efforts of a long line of teachers. We were able to help them achieve grade 7 reading levels within a year. The content did not change, the learners' intelligence and abilities did not change - only the teaching method changed.
The "white swan" is out there. Absence of evidence is not evidence of absence.

[Robert Costello]

Learning styles are just a psychometric system used to identify qualities of the individual. Which have been widely used and recongised by top researchers (Felder, Kolbs, etc) for many years. Research has demonstrated that learning styles do help the individual (class room) & within on-line learning; however, seams to me that you are intersted in adaptive/personalised on-line learning.

[Robert Costello]

Learning styles are just a psychometric system used to identify qualities of the individual. Which have been widely used and recongised by top researchers (Felder, Kolbs, etc) for many years. Research has demonstrated that learning styles do help the individual (class room) & within on-line learning; however, seams to me that you are intersted in adaptive/personalised on-line learning.

[John Lloyd]

Will, I see that no one's yet come through with clear evidence of the LS effects. I'm not surprised.

You might enjoy this post on the topic: Willingham on learning styles

[Julian King]

I suppose it depends what you mean by learning styles. There are dozens of types of learning styles. I'd be surprised if you don't incorporate some of them in your work.

Check out Kolbs list of references, with such extensive refereed literature already I don't see the point in someone setting up an experiment which won't prove much. (i.e. an on-line experiment is not applicable to a facilitated lead learning environment).

http://www.learningfromexperience.com/research-library/

As an aside - you can test a negative. As a biological scientist for many years, I often tested that something had no effect. For example "Learning styles has no positive effect on test results...".