The National Center on Intensive Intervention (NCII) is a valuable resource that provides educators with evaluations of commercially available intervention programs. NCII is a good source for educators because the reviewers consider a study’s design, purpose, fidelity of implementation, and assessment alignment. (In contrast, What Works Clearinghouse only considers a study’s design.)
NCII has posted three reviews of Read Naturally, showing statistically significant effects for comprehension, fluency, rate, and accuracy:
The Heistad study showed an effect size for reading comprehension of .38 on the Minnesota Comprehensive Assessments (MCA).
The Christ & Davie study showed effect sizes of .66 for fluency with the Grey Oral Reading Test–Fourth Edition: Fluency (GORT 4: Fluency) and .66 for accuracy with the GORT 4: Accuracy.
The Tucker & Jones study showed effect sizes of .51 for rate with the GORT 4: Rate, .87 for accuracy with the GORT 4: Accuracy, and .75 for fluency with the GORT 4: Fluency.
The NCII pays attention to study purpose, design, and implementation. In Read Naturally's experience, the NCII, as well as the Florida Center for Reading Research (FCRR) and the University of Oregon, produce accurate and helpful reviews of programs.
Unfortunately, not all evaluators take into account the stated purpose of a study and may post reviews of studies that were not intended to evaluate the program. As a result, they provide educators with misleading information.
On the one hand, a review of the Christ & Davie study by What Works Clearinghouse (WWC) concluded that the study showed a "statistically significant positive effect" for fluency, and the WWC review of the Heistad study showed positive effects for comprehension.
WWC’s report on Read Naturally: Positive effects on general reading achievement
But in other cases, the WWC has posted evaluations of Read Naturally that are based on studies that did not use or evaluate the Read Naturally strategy properly. In each case, the study authors clearly state that using their study as an evaluation of Read Naturally would be a misapplication of the data, and that WWC should not post these studies:
"While I used Read Naturally materials, I did NOT fully implement the Read Naturally strategy and my study was NOT intended to evaluate the Read Naturally strategy."
– Carrie Hancock, Study Author
"My study was a study of sustained silent reading and the purpose was not to evaluate Read Naturally."
– Susan Kemp, Study Author
What Works Clearinghouse (WWC) includes the Arvans (2010) study in its intervention report for Read Naturally programs. In this report, WWC asserts that the Read Naturally group in the Arvans study does not demonstrate significant gains in oral reading fluency. However, an analysis of the Arvans study reveals that the Read Naturally group’s fluency gains were quite significant.
Research Documents in the Knowledgebase
Please let us know what questions you have so we can assist. For Technical Support, please call us or submit a software support request.