The Relationship Between Study Quality and the Effects of Supplemental Reading Interventions: A Meta-Analysis
Open accessAbstract
Empirical studies investigating supplemental reading interventions for students with or at risk for reading disabilities in the early elementary grades have demonstrated a range of effect sizes. Identifying the findings from high-quality research can provide greater certainty of findings related to the effectiveness of supplemental reading interventions. This meta-analysis investigated how four variables of study quality (study design, statistical treatment, Type I error, and fidelity of implementation) were related to effect sizes from standardized measures of foundational reading skills and language and comprehension. The results from 88 studies indicated that year of publication was a significant predictor of effect sizes for both standardized measures of foundational reading skills and language and comprehension, with more recent studies demonstrating smaller effect sizes. Results also demonstrated that with the exception of research design predicting effect sizes on foundational reading skills measures, study quality was not related to the effects of supplemental reading interventions. Implications for research and practice are discussed.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The quality of the included studies was assessed based on guidelines by Austin et al. (2019) and following the quality assessment of Merke et al. (2024).
“Finally, we assessed the quality of the included studies, based on guidelines by Austin et al. (2019) and following the quality assessment of Merke et al. (2024).”
From Learning a Language Through Reading: A Meta-analysis of Studies on the Effects of Extensive Reading on Second and Foreign Language Learning (2025) · cites this paper as Austin et al. (2019)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099 · see citing works
Robust variance estimation in meta‐regression with dependent effect size estimates
Hedges, L. V., Tipton, E., & Johnson, M. C. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.5 · Cited by 2,086 · see citing works
Small sample adjustments for robust variance estimation with meta-regression.
Tipton, E. (2015) · Psychological Methods · doi:10.1037/met0000011 · Cited by 810 · see citing works
Research-based implications from extensive early reading interventions
Wanzek, J., & Vaughn, S. (2007) · School Psychology Review · doi:10.1080/02796015.2007.12087917 · Cited by 424 · see citing works
Self-citationA Meta-Analysis of Interventions for Struggling Readers in Grades 4–12: 1980–2011
Scammacca, N. K., Roberts, G., Vaughn, S., & Stuebing, K. K. (2015) · Journal of Learning Disabilities · doi:10.1177/0022219413504995 · Cited by 372 · see citing works
Self-citationTipton, E., & Pustejovsky, J. E. (2015) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998615606099 · Cited by 362 · see citing works
Interventions for students with learning disabilities: a meta-analysis of treatment outcomes
(1999) · Choice Reviews Online · doi:10.5860/choice.37-2302 · Cited by 359 · see citing works
Mathes, P. G., Denton, C. A., Fletcher, J. M., Anthony, J. L., Francis, D. J., & Schatschneider, C. W. (2005) · Reading Research Quarterly · doi:10.1598/rrq.40.2.2 · Cited by 341 · see citing works
Harn, B., Parisi, D., & Stoolmiller, M. (2013) · Exceptional Children · doi:10.1177/001440291307900204 · Cited by 297 · see citing works
robumeta: An R-package for robust variance estimation in meta-analysis
Fisher, Z., & Tipton, E. (2015) · arXiv · doi:10.48550/arxiv.1503.02220 · Cited by 280 · see citing works
clubSandwich: Cluster-Robust (Sandwich) Variance Estimators with Small-Sample Corrections
Pustejovsky, J. E. (2016) · doi:10.32614/cran.package.clubsandwich · Cited by 233 · see citing works
Meta-Analyses of the Effects of Tier 2 Type Reading Interventions in Grades K-3
Wanzek, J., Vaughn, S. R., Scammacca, N. K., Gatlin, B., Walker, M. A., & Capin, P. (2015) · Educational Psychology Review · doi:10.1007/s10648-015-9321-7 · Cited by 218 · see citing works
Self-citationCurrent Evidence on the Effects of Intensive Early Reading Interventions
Wanzek, J., Stevens, E. A., Williams, K. J., Scammacca, N., Vaughn, S., & Sargent, K. (2018) · Journal of Learning Disabilities · doi:10.1177/0022219418775110 · Cited by 189 · see citing works
Self-citationResponse to Early Reading Intervention Examining Higher and Lower Responders
Vaughn, S. R., Wanzek, J., Murray, C. S., et al. (2009) · Exceptional Children · doi:10.1177/001440290907500203 · Cited by 144 · see citing works
Self-citationLayers of Reading Intervention in Kindergarten Through Third Grade
O’Connor, R. E., Fulmer, D., Harty, K. R., & Bell, K. M. (2005) · Journal of Learning Disabilities · doi:10.1177/00222194050380050701 · Cited by 128 · see citing works
Evidence-Based Practices in a Changing World
Lemons, C. J., Fuchs, D., Gilbert, J. K., & Fuchs, L. S. (2014) · Educational Researcher · doi:10.3102/0013189x14539189 · Cited by 121 · see citing works
Scammacca, N. K., Roberts, G. J., Cho, E., Williams, K. J., Roberts, G., Vaughn, S. R., & Carroll, M. (2016) · Review of Educational Research · doi:10.3102/0034654316652942 · Cited by 103 · see citing works
Self-citationA Meta-Analysis of the RTI Literature for Children at Risk for Reading Disabilities
Tran, L., Sanchez, V. M., Arellano, B., et al. (2011) · Journal of Learning Disabilities · doi:10.1177/0022219410378447 · Cited by 98 · see citing works
Inclusion Versus Specialized Intervention for Very-Low-Performing Students
Fuchs, L. S., Fuchs, D., Compton, D. L., Wehby, J. H., Schumacher, R. F., Gersten, R. M., & Jordan, N. C. (2015) · Exceptional Children · doi:10.1177/0014402914551743 · Cited by 97 · see citing works
A New Framework for Systematic Reviews
Goldstein, H., Crawford, K. C., & Schneider, N. (2014) · Exceptional Children · doi:10.1177/0014402914522423 · Cited by 63 · see citing works
Carney, K. J., & Stiefel, G. S. (2008) · Learning disabilities · Cited by 41 · see citing works
Swanson, H. L., & Hoskyn, M. (1998) · Review of Educational Research · doi:10.2307/1170599 · Cited by 41 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Merke, S., Ganushchak, L., & van Steensel, R. (2024) · Educational Research Review · doi:10.1016/j.edurev.2023.100572 · Cited by 15 · see citing works
Sangers, N. L., van der Sande, L., Welie, C., Dobber, M., & van Steensel, R. (2025) · Educational Psychology Review · doi:10.1007/s10648-025-10068-6 · Cited by 6 · see citing works
“Finally, we assessed the quality of the included studies, based on guidelines by Austin et al. (2019) and following the quality assessment of Merke et al. (2024).”
- Authors
- Austin, C. R., Wanzek, J., Scammacca, N. K., Vaughn, S., Gesel, S. A., Donegan, R. E., & Engelmann, M. L.
- Affiliations
- The University of Texas at Austin (US)
- Vanderbilt University (US)
- Corresponding author
- Austin, Christy R.
- ORCID
- Venue
- Exceptional Children
- Publisher
- SAGE Publications
- Published
- 2018-09-14
- Volume
- 85
- Issue
- 3
- Pages
- 347-366
- ISSN
- 0014-4029 (ISSN-L)
- 2163-5560
- PMC
- PMC6777867
- OpenAlex
- W2891086319
- Cited by
- 25
- References
- 22
- Citations by year
- 2026: 1 · 2025: 6 · 2024: 4 · 2023: 6 · 2022: 4 · 2021: 4
- FWCI
- 0.21
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Exceptional Children (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- Office of Special Education and Rehabilitative Services (Grant H325H140001)
- Eunice Kennedy Shriver National Institute of Child Health and Human Development (Grant P50 HD052117)
- Institute of Education Sciences (Grant R324A150269)
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Reading and Literacy Development
- Social Sciences › Psychology › Developmental and Educational Psychology › Behavioral and Psychological Studies
- Social Sciences › Education › Early Childhood Education and Development
- Keywords
- reading disorders, at-risk readers, early elementary education, study quality, effect size, research design, type I error, implementation fidelity, foundational reading skills, reading intervention effectiveness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
