Will at Work Learning

Learning and Forgetting Curves -- In Depth On Video

I've been sharing the ideas behind the learning and forgetting curves for years.

In this new video, I go into depth about the powerful implications of these curves.

• On Vimeo
• On YouTube

 

 

2011 Social-Media Measurement Survey (for Workplace Learning Field)

Social Media is hot, but it is not clear how well we are measuring social media.

A couple of years ago I wrote an article for the eLearning Guild about measuring social media. But it's not clear that we've got this nailed yet.

With this worry in mind, I've created a research survey to begin a process to see how best social-media (of the kind we might use to bolster workplace learning-and-performance) can be measured.

Here's the survey link. Please take the survey yourself. You don't have to be an expert to take it.

Here's my thinking so far on this. Please send wisdom if I've missed something.

1. We can think about measuring social media the same way we measure any learning intervention.
2. We can also create a list of all the proposed benefits for social media, and the proposed costs, and all the proposed harms, and we can see how people are measuring these now. The survey will help us with this second approach.

Note: Survey results will be made available for free. If you take the survey, you'll get early releases of the survey results and recommendations.

Also, this is not the kind of survey that needs good representative sampling, so feel free to share this far and wide.

Here is the direct link to the survey:   http://tinyurl.com/4tlslol

Here is the direct link to this blog post:   http://tinyurl.com/465ekpa

Why You Should Avoid Benchmarking Against Other Companies

See the full post on my book blog...

Click here to view...

 

IBM's Watson (Jeopardy Winner). Will Our Folks Be Replaced?

IBM's Watson beat two Jeopardy champs in answering questions this week. 

Does this mean that soon computers will be able to replace people in answering questions? For us as workplace learning-and-performance professionals, does this mean we'll be doing less training of people, and more training of machines?

Probably depends on lots of things as to how soon this happens. For example, folks doing technical support probably won't be replaced as soon if the systems/equipment on which they are providing support are updated regularly or modified substantially.

Anyway, this will affect our field---indeed it is already affecting our field to some extent as computers already provide support for folks who are answering questions---but when this reaches a tipping point we'll just have to wait and see.

• Article in NYTimes
• NPR On Point Radio with Tom Ashbrook

 

 

How Entrepreneurs Differ and are the Same (Article Review)

As an entrepreneur, I'm always curious about entrepreneurship. I came across this article by Cardiff Garcia in The Observer (the magazine of the Association for Psychological Science). It highlights similarities and differences between entrepreneurs and most people. You can read the article (best method), or just read my synopsis (okay method):

How Entrepreneurs are the Same (as most other people):

• They don't care any more about financial success than most people.
• They don't care more about money.
• They don't care any more about following family traditions.
• They score the same on the trait of conscentiousness.
• They do not crave more risk.
• They are no more ruthless.
• They are not more spontaneous.
• They are no more outgoing.
• They are not more agreeable.
• They don't have better problem solving approaches.

How Entrepreneurs are Different (compared to most other people):

• They possess a greater belief that they will succeed when compared to most people.
• They are less likely to care what others think of them.
• They display more intensity toward their work.
• They are willing to sacrifice more (in terms of time with family and/or money earned).
• They find it harder to balance work and their family lives.
• They find it harder to get support from friends and family for what they do.
• They score higher on "need for achievement," for example by setting difficult but obtainable goals.
• They score higher on "self efficacy," the belief that they can accomplish certain tasks (not tasks in general as it turns out). 
• They are much more stress tolerant.
• They are more likely to expect success (generally and in specific situations).

How Entrepreneurs are About the Same or Just Slightly Different (compared to most other people):

• Slightly more autonomous than most other people.
• Slightly more likely to believe they control their own destinies. 
• Just marginally more risk takers.

How entreprenuers are Sometimes Different (than other people):

• Entrepreneurs in tech industries are more innovative, but entrepreneurs in other industries are not.

The Article's Bottom Line

Being an entrepreneur is tough. Entrepreneurs need to be up for the rigors of entrepreneurship.

Learning Landscape Model Video

The Learning Landscape Model is a research-based model---evolved over the last decade---that can be used to guide workplace learning-and-performance designs, discussions, and explorations.

It is based on the fundamental cognitive architectures of learning, remembering, and prompting as three distinct cognitive operations, all of which are needed to maximize workplace learning-and-performance results. While previous models have often forgotten forgetting or forgotten prompting mechanisms (like job aids), the Learning Landscape is complete. Perhaps more importantly, it is actionable, for example, it can be utilized to have productive discussions between us as learning professionals and our business partners. Finally, the Learning Landscape Model can be used to improve learning measurement significantly over the 4-levels or roi models.

Check out the video of the Learning Landscape Model...

Or utilize this link if your filters don't allow YouTube...

Neon Elephant Award 2010

The Neon Elephant Award 2010 went to a researcher whose work has had a profound impact on the workplace learning-and-performance field.

Click here to see who won...

Click here to learn more about the Neon Elephant Award...

Brain-Based Explanations Rebutted Again

Check out this article which claims:

"Science of mind is one of the most important intellectual developments in the last half century. It should not be obscured by neurobabble."

This might be a follow-up post to an earlier one I wrote that showed how easily we are fooled by scientific claims.

 

How Much Do People Forget?

This blog post is excerpted from the full report, How Much Do People Forget? Click here to download the full report. You may also access the report---and many other reports---by going to my catalog page by clicking here.

Everybody Wants to Know—How Much Do People Forget?

For years, people have been asking me, “How much do people forget?” and I’ve told them, “It depends.” When I make this statement, most people scowl at me and walk away frustrated and unrequited. I also suspect that some of them think less of me—perhaps that I am just hiding my ignorance.

But I try. I try to explain the complexity of human learning. I explain that forgetting depends on many things, for example:

• The type of material that is being learned
• The learners’ prior knowledge
• The learners’ motivation to learn
• The power of the learning methods used
• The contextual cues in the learning and remembering situations
• The amount of time the learning has to be retained
• The difficulty of the retention test
• Etc.

More meaningful materials (like stories) tend to be easier to remember than less meaningful material (like nonsense syllables). More relevant concepts tend to be easier to remember than less relevant concepts. Learners who have more prior knowledge in a topic area are likely to be better able to remember new concepts learned in that area. More motivated learners are more likely to remember than less motivated learners. Learners who receive repetitions, retrieval practice, feedback, variety (and other potent learning methods) are more likely to remember than learners who do not receive such learning supports. Learners who are provided with learning and practice in the situations where they will be asked to remember the information will be better able to remember. Learners who are asked to retrieve information shortly after learning it will retrieve more than learners who are asked to retrieve information a long time after learning it.

I try to explain all this, but still people keep asking.

And then there are the statistics I keep hearing—that are passed around the learning field from person to person through the years as if they were immutable truths carved by Old Moses Ebbinghaus on granite stones. Here is some information so cited (as of December 2010):

• People forget 40% of what they learned in 20 minutes and 77% of what they learned in six days (http://www.festo-didactic.co.uk/gb-en/news/forgetting-curve-its-up-to-you.htm?fbid=Z2IuZW4uNTUwLjE3LjE2LjM0Mzc). 
• People forget 90% after one month. (http://www.reneevations.com/management/ebbinghaus-curve/)
• People forget 50-80% of what they’ve learned after one day and 97-98% after a month. (http://www.adm.uwaterloo.ca/infocs/study/curve.html)

Never mind that these immutable truths conflict with each other.

So, I will try one more time to convince the world that forgetting depends.

To accomplish this, I explored 14 research articles, examining 69 conditions to see how much forgetting occured, representing over 1,000 learners.

The following graph details the amount of forgetting for each of the 69 conditions:

Conclusions

This graph and the indepth analysis in the full article revealed four critical concepts in human learning—truths that every learning professional should deeply understand.

1. The amount a learner will forget varies depending on many things. We as learning professionals will be more effective if we make decisions based on a deep understanding of how to minimize forgetting and enhance remembering.
2. Rules-of-thumb that show people forgetting at some pre-defined rate are just plain false. In other words, learning gurus and earnest bloggers are wrong when they make blanket statements like, “People will forget 40% of what they learned within a day of learning it.”
3. Learning interventions can produce profound improvements in long-term remembering. In other words, learning gurus are wrong when they say that training is not effective.
4. Different learning methods produce widely different amounts of forgetting. We as learning professionals can be more effective if we take a research-based approach and utilize those learning methods that are most effective.

Telling Findings From the Research

1. People in the reviewed experiments forgot from 0% to 94% of what they had learned. The bottom line is that forgetting varies widely.
2. Even within a restricted time range, learners forgot at wildly differing rates. For example, in the 1-2 day range, learners forgot from 0 to 73%. Learners in the 2-8 year range forgot from 16% to 94%. The obvious conclusion here is that learning varies widely (and wildly) and cannot be predetermined (except perhaps by deities, of whom, I think, we have not even a few in the learning field). To be specific, when we hear statements like, “People will forget 60% of what they learned within 7 days,” we should ignore such advice and instead reflect on our own superiority and good looks until we are decidedly pleased with ourselves.
3. Even when we looked at only one type of learning material, forgetting varied widely. For example, in Bahrick’s classic 1979 experiment where learners were learning English-Spanish word pairs, learners forgot from 12% to 63%. Even more remarkably, if we include those cases where learners actually remembered more on the second test than the first test, learners’ “forgetting” varied from -41% to 63%, a swing of 104 percentage points! Again, we must conclude that forgetting varies widely.
4. Many of the experiments reviewed in this report showed clearly that learning methods matter. For example, in the Bahrick 1979 study, the best learning methods produced an average forgetting score of -29% forgetting, whereas the worst learning methods produced forgetting at 47%, a swing of 76% points. In Runquist’s 1983 study, the best learning method produced average forgetting at 34%, whereas all the other learning methods produced average forgetting of 78%. In Allen, Mahler, and Estes’ 1969 experiment, the learners given the best learning methods forgot an average of 2.3%, whereas the learners who got middling learning methods forgot an average of 14.3%, and learners given the worst learning methods forgot approximately 21.7%. The bottom line is that the learning methods we choose make all the difference!!

Check out the full report to learn more about the following:

• What you should do as a learning professional (in light of these findings).
• Whether the learning-curve notion still applies.
• What wisdom each of the 14 research articles revealed.
• The methodology used in the research.
• The calculation of forgetting.

 

K-12 Classrooms: Research Review of Multigrade, Multiage, Combination Classrooms

I don't usually delve into K-12 classroom-based research to any great extent. However, as my daughter's elementary school is starting a "redesign" process---and I heard arguments on both sides of the multigrade-classroom issue---I thought perhaps I'd give a quick look at the research available.

I had hoped that this would be a quick review, where I would find one or two definitive research reviews in scientific refereed journals, but unfortunately, the research base is rather frail and unclear.

Still, I think the following review does provide some wisdom about how to think about multiage classrooms.

-------------------------------------------------

Extent of Review             

I did a moderately quick review—not an exhaustive review—of a couple dozen recent research articles on multigrade classrooms.

This review was conducted in September and October of 2010.

-------------------------------------------------

Major Conclusions (See More Specific Recommendations Further Below):

The research, although being too scant and too difficult to interpret to make definitive recommendations, generally suggests that multigrade classroom approaches are not likely to produce results that differ substantially from single-grade classrooms.

Specifically, it is likely that the quality of the learning methods utilized and the teacher’s performance in the classroom makes more of difference than whether a multigrade or single-grade approach is utilized.

Multigrade teaching is generally considered more difficult and onerous than single-grade teaching.

If a multigrade approach is utilized, then it should be utilized with due diligence—providing teacher support and development, utilizing team teaching, encouraging many diagnostic opportunities (so that learning can be tailored to learner’s current levels), and so forth.

------------------------------------------------- 

Context:             

Researchers and practitioners use many labels for multigrade education, including the following: multigrade, multiage, mixed-age, vertical grouping, combination, composite, double-grade, split-grade, dual-age, hyphenated, nongraded, etc. These terms are often confused, redundant, etc., making researching and thinking about related issues difficult. In this report, I will use the following terms:

• Single-grade classrooms are comprised of one grade.
• Multigrade classrooms are comprised of two (or more) grades.
  • Combination Multigrade classrooms are multigrade classrooms that are utilized for logistical reasons.
  • Pedagogic Multigrade classrooms are multigrade classrooms that are utilized for pedagogic (learning-related) reasons.

Sometimes multigrade classrooms are created because of logistical reasons such as declining or uneven enrollments (Veenman, 1996; Burns & Mason, 2002; Mulryan-Kyne, 2007).

Sometimes multigrade classrooms are created for philosophical and pedagogical rationale. There are strong advocacy groups for pedagogic multigrade classrooms. Some of this advocacy is at odds with the research—in the sense that they claim overwhelming benefits for multigrade classrooms when the research is more balanced and uncertain (I noticed this myself in reviewing the research, but also see Burns & Mason, 1997).

It can be helpful to view classes on a continuum from single-grade classes, to combination multigrade classes, to pedagogic multigrade classes (See Mason & Burns, 1997). While many different arrangements are possible, we can talk in generalities as follows:

• Single-grade classes have students only from one grade.
• Combination multigrade classes have students from two (or more) grades and one teacher who teaches them.
• Pedagogic multigrade classes have students from two (or more) grades but are taught by a team of teachers. Pedagogic multigrade classes also tend to be more focused on providing learners individually-tailored learning content than either single-grade or combination multigrade classes.

Teachers in combination multigrade classes are less likely to receive the support and structure they need than teachers in pedagogic multigrade classes (Mason & Burns, 1997).

Multigrade classrooms are not aberrations, but are used throughout the world, and are likely to continue and grow in use in the future (Mulryan-Kyne, 2007).

-------------------------------------------------

Quality of Available Research:     

Unfortunately, it seems that much of the available research is tainted by methodological weaknesses. Specifically, it appears that more experienced teachers tend to teach multi-age classrooms—so that when advantages appear in the educational results, those advantages may be due to teacher experience as opposed to the multi-age classroom itself. In addition, more affluent students are the ones who tend to be taught in multigrade classrooms—so that when research results show advantages, those could be due to socio-economic and educational advantages of parents as opposed to the multigrade classrooms.

Quotes from the Research:

“Effective research in the area of multiage education is still in its infancy.” From Kinsey (2001).

“The literature on multigrade teaching is relatively sparse, some of it anecdotal in nature and/or of poor quality.” From Mulryan-Kyne (2007).

“There is considerable evidence that principals, in an effort to reduce the burden on multigrade teachers, place more able, more independent, and more cooperative students in multigrade classes.” From Mason and Burns (1996). Note: This is relevant in that good results in comparison to single-age classrooms may be due to having better students in the class.

“No studies in which students were randomly assigned to experimental and control groups were found.” From Veenman (1995). Note: While such random assignment is the gold standard in research, it is difficult to implement in the classroom.

“Because of the lack of distinction between combination and multiage/nongraded classes and the omission of important studies and methodological considerations, it appears that researchers have drawn overly optimistic and erroneous conclusions about the effects of combination classes.” From Mason & Burns (1997).

Conclusion based on Quality of Available Research:

These methodological weaknesses make firm conclusions difficult. Tentative conclusions are still possible.

-------------------------------------------------

Teaching is Often More Difficult in Multigrade Classrooms:            

It appears that teaching in a multigrade classroom is more difficult than teaching in a single-age classroom. This conclusion comes from those on all sides of the debate, so it is a fairly strong conclusion.

Quotes from the Research:

“It is commonly stated in the literature that multigrade teaching is more difficult than single grade teaching” From Mulryan-Kyne (2007).

“Multigrade classes hold instructional potential for some, but they are potentially onerous for most. Indeed, we argue that multigrade classes lead to a negative instructional effect and that they increase teachers' stress and may jeopardize teachers' motivation and commitment to teaching.” From Mason and Burns (1996).

“Most teachers, when asked about their feelings toward and organizational and teaching strategies for combination classes, responded negatively and preferred not to teach them.” From Mason and Burns (1995).

“The professional knowledge and skills that are relevant and necessary to teaching effectively in single-grade contexts are also relevant and necessary for effective multigrade teaching… However, many of these skills need heightened emphasis in the context of the preparation of teachers for multigrade teaching.” From Mulryan-Kyne (2007).

“[Teachers] report that these classes require more planning, are more difficult to teach, and diminish instruction and curriculum coverage.” From Mason & Burns (1997).

“We concluded that the difficulties teachers face in multigrade classes are centered around five problem areas: (a) the efficient use of instructional time, (b) the design of effective instruction, (c) classroom management, (d) the organization of independent practice or learning, and (e) the formulation of clear and collectively agreed-upon goals for making the multigrade school work.” From Veenman (1996).

“…multigrade teachers pressed for instructional time and the mastery of basic skills (e.g., reading, writing, and mathematics) might neglect science, social studies, and other subjects, which would lead to negative achievement effects in these areas.” From Mason & Burns (1996).

-------------------------------------------------

Academic Achievement Results from Multigrade Classrooms:

In the two most credible recent reviews of the research, the findings have shown no differences in academic achievement between multigrade classrooms and single-grade classrooms (Veenman, 1995; Mason & Burns, 1997). However, because of the likelihood that this research is tainted in having better students and teachers in multi-grade classrooms, it is possible that multigrade classrooms produce “at least small negative effects” (Mason & Burns, 1997).

Separating the results for combination multigrade classrooms and pedagogical multigrade classrooms is difficult because researchers haven’t always noted this difference. In Veenman’s (1995) review, both combination and pedagogic multigrade classrooms produced non-significant results—in other words, they were found to produce the same academic results as single-grade classrooms. However, as Mason and Burns (1996) pointed out, Veenman didn’t take into account some important potential biases. Mason and Burns write: “We conclude that multigrade classes have at least a small negative effect on achievement as well as potentially negative effects on teacher motivation.”

Unfortunately, in Mason and Burns (1997) follow-up research review, they only focused on combination multigrade classrooms—NOT pedagogic multigrade classrooms. Their results on combination multigrade classrooms were consistent with Veenman’s in finding no statistical differences, but they warned that “all things being equal, combination classes have at least small negative effects.” Such a negative outcome was found in a 2008 study of combination multigrade classrooms in California (Sims, 2008).

Conclusions on Academic Achievement:

The research is not definitive because there are factors that have not really been teased out as of yet.

In terms of academic results, the tentative conclusion is that multigrade classes on average are probably no better and no worse than single-grade classes, but if they are different from single-grade classes, they may be slightly worse.

-------------------------------------------------

Social/Emotional Results from Multi-age Classrooms:

Veenman’s (1995) research found that a majority of combination multigrade classes performed no better than single-grade classes in terms of attitudes towards school, self-concept, and personal and social adjustment.  Mason and Burns (1997) examination of combination multigrade classes found similar non-significant affective benefits.

For pedagogic multigrade classes, however, Veenman (1996) found a “very small” effect, showing slight benefits for pedagogic multigrade classes in terms of attitudes towards school, self-concept, and personal and social adjustment. Again, because of the potential biasing effects in terms of student and teacher selection, these results could be due to bias instead of any benefits for pedagogic multigrade classes.

Conclusions on Social/Emotional Results:

The research is not definitive because there are factors that have not really been teased out as of yet.

In terms of social/emotional results, the tentative conclusion is that multigrade classes on average are probably no better and no worse than single-grade classes. If there are very small benefits, they might be obtained in well-designed pedagogic multigrade classrooms—as opposed to logistically-driven combination multigrade classrooms.

-------------------------------------------------

Do some Students Benefit More than Others from Multigrade Classrooms?             

Although this was beyond the scope of my review—I did come across one recent study that dealt specifically with the question of how different types of students might be affected by a multigrade approach (Ong, Allison, & Haladyna, 2000).

First, it should be noted that in this one study, multigrade classrooms tended to outperform single-grade classrooms for all students, regardless of their background. Given that as background, the study found the following:

• It found no differences between boys and girls.
• It found that regular students seemed to benefit more from multigrade classrooms than disadvantaged students (that is, Title 1 students).
• It found that overall non-Hispanic students seemed to benefit more from multigrade classrooms than Hispanic students.

Caveat: This was just one study and should be evaluated with caution.

-------------------------------------------------

Overall Recommendations Regarding Multigrade, Multiage, Combination Classrooms:         

The scarcity, frailty, and equivocation in the research make strong recommendations impossible. Instead, I offer these tentative ideas for consideration:

1. It is likely that the quality of the learning methods utilized and the teacher’s performance in the classroom makes more of difference than whether a multigrade or single-grade approach is utilized. Given this, it appears that choosing a multigrade approach would be acceptable, though the following points should also be kept in mind.

2. If a multigrade approach is utilized, then it should be utilized with due diligence—providing teacher support and development, utilizing team teaching, encouraging many diagnostic opportunities (so that learning can be tailored to learner’s current levels*), and so forth.

* Recent research indicates that teachers, tutors, and other learning professionals tend not to be very good at providing instructional explanations at learner’s appropriate levels—but that they can improve on this by specifically being informed of their learners’ level of understanding (Wittwer, Nückles, & Renkl, 2010).

3. Multigrade teaching seems to offer the possibility for alternative methods of learning, including group work targeted to different developmental levels. On the other hand, multigrade teaching by itself is no guarantee of good teaching methodology.

4. It should be recognized that multigrade teaching probably has costs associated with its use. Specifically, teachers may have to invest more effort and care in the process and may have a tendency to tradeoff other desirable educational goals.

5. In some forms of multigrade teaching, especially those that utilize team teaching, teachers have to learn the skill levels (in each discipline) of many more students. Because one of the most important aspects of teaching is providing learners with just the right level of instruction, this may cause students to be underserved at first as the teacher learns their students' skill levels. One way to ameliorate this problem is for students entering a multigrade cycle to be diagnosed through testing, problem solving, and other performance metrics early on. This “slow-start” issue has a flip side—because multigrade classes stretch into two (or more) years, students in subsequent years will probably experience accelerated learning due to deeper understanding of each student by teachers and hence better instructional scaffolding.

6. Teachers with less experience may be especially unprepared for multigrade teaching. With experience, teachers learn how to automate aspects of their classroom performance so that they can utilize their limited working memory capacity to focus on supporting their learners in learning (for example, differentiating their instruction, etc.). Newer teachers will be unlikely to provide the necessary instructional scaffolding to perform at a high level.

7. Students who have behavioral problems or who are less competent in learning may make the task of multigrade teaching more difficult. This may suggest that extra effort and guidance in the early grades is warranted or that a multigrade approach should be delayed until a time when most students are ready to engage fully in the process.

-------------------------------------------------

Research References:

Burns, R. & Mason, D. (2002). Class composition and student achievement in elementary schools. American Educational Research Journal, 39 (1), 207-233.

Burns, R. B., & Mason, D. A. (1998). Class formation and composition in elementary schools. American Educational Research Journal, 35(4), 739-772.

Kinsey, S. (2001). Multiage Grouping and Academic Achievement. ERIC Digest, January 2001.

Mason, D. A., & Burns, R. B. (1995). Teachers' views of combination classes. Journal of Educational Research, 89(1), 36-45.

Mason, D. A., & Burns, R. B. (1996). "Simply no worse and simply no better" may simply be wrong: A critique of Veenman's conclusion about multigrade classes. Review of Educational Research, 66(3), 307-322.

Mason, D., & Burns, R. (1997). Reassessing the effects of combination classes. Educational Research and Evaluation, 3(1), 1–53.

Mulryan-Kyne, C. (2007). The preparation of teachers for multigrade teaching. Teaching and Teacher Education, 23(4), 501-514.

Ong, W., Allison, J., & Haladyna, T. M. (2000). Student achievement of 3rd-graders in comparable single-age and multiage classrooms. Journal of Research in Childhood Education, 14(2), 205-215.

Sims, D. (2008). A strategic response to class size reduction: Combination classes and student achievement in California. Journal of Policy Analysis and Management, 27(3), 457-478.

Veenman, S. (1995). Cognitive and noncognitive effects of multigrade and multi-age classes: A best-evidence synthesis. Review of Educational Research, 65, 319-381.

Veenman, S. (1996). Effects of multigrade and multi-age classes reconsidered. Review of Educational Research, 66(3), 323-340.

Wittwer, J., Nückles, M., & Renkl, A. (2010). Using a diagnosis-based approach to individualize instructional explanations in computer-mediated communication. Educational Psychology Review, 22(1), 9-23.

-------------------------------------------------

Some of the articles reviewed but not cited:

Song, R.  Spradlin, T. E., & and Plucker, J. A. (2009). The Advantages and Disadvantages of Multiage Classrooms in the Era of NCLB Accountability. Education Policy Brief, 7, 1-7. Published by the Center for Evaluation and Education Policy of Indiana University.

Why not cited:
This article is not from refereed journal so it may not have been fully vetted. Also, there are hints of bias in the article, for example, (a) providing invited proponents of multiage classrooms to describe its value, without providing a similar counterpoint, and (b) talking about the lack of good research but then plowing ahead with a list of the “benefits (perceived and real)” of multiage classrooms—so in a real sense the article ignores the research by plowing ahead to benefits. Still, I wouldn’t discount this article completely. Its research review seems good when it is limited to reviewing the actual research. Also, it makes recommendations that are mostly consistent with the findings contained in this review, so I can’t dismiss their conclusions. The bottom line is that this review does not add anything to a review of the research, and, because it is not vetted in a refereed journal and appears slightly biased, I think it safer not to cite it as separate evidence.

Hoffman, J. (2003). Multiage teachers beliefs and practices. Journal of Research in Childhood Education, 18, 5-17.

Why not cited:
This article looked at only 4 teachers, all who had chosen to be teaching in multiage classrooms. It was really a descriptive research project and did not look at actual learning outcomes. No single-grade teachers were examined so we don’t really know how different results with single-grade teaching might be.

Linley, L. (1999). Multi-Age Classes and High Ability Students. Review of Educational Research, 69, 187-212.

Why not cited:
This article was a review of other reviews, not a review of research studies itself. Also, it focused only on high-ability students.

Gerard, M. (2005). Bridging the gap: Understanding young children’s thinking in multiage groups. Journal of Research in Childhood Education, 19, 243-250.

Why not cited:
This article utilized an incredibly biased research design. It took one classroom at one school that used multiage grouping and compared it on standardized exams with the national average.

Mariano, S., & Kirby N. (2009). Achievement of Students in Multigrade Classrooms Evidence from the Los Angeles Unified School District. From the RAND Education Working Paper Series (WR-685-IES).

Why not cited:
This article is not from refereed journal so it may not have been fully vetted. It also uses somewhat opaque statistical methods to estimate findings, not looking directly at actual comparisons between multigrade and single-grade classrooms. I must admit that I don’t fully understand all the statistical employments utilized (for example, “doubly robust regression,” “non-parametric generalized boosting,” “the Kolmogorov-Smirnov (K-S) statistic”). Thus, I may be missing the full implications of the research employed.