The Effectiveness of Direct Instruction Curricula: A Meta-Analysis of a Half Century of Research
The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Abstract
Quantitative mixed models were used to examine literature published from 1966 through 2016 on the effectiveness of Direct Instruction. Analyses were based on 328 studies involving 413 study designs and almost 4,000 effects. Results are reported for the total set and subareas regarding reading, math, language, spelling, and multiple or other academic subjects; ability measures; affective outcomes; teacher and parent views; and single-subject designs. All of the estimated effects were positive and all were statistically significant except results from metaregressions involving affective outcomes. Characteristics of the publications, methodology, and sample were not systematically related to effect estimates. Effects showed little decline during maintenance, and effects for academic subjects were greater when students had more exposure to the programs. Estimated effects were educationally significant, moderate to large when using the traditional psychological benchmarks, and similar in magnitude to effect sizes that reflect performance gaps between more and less advantaged students.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,395
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
(2002) · Cited by 13,211
(1979) · Medical Entomology and Zoology · Cited by 11,252
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
(2008) · Cited by 6,939
Quasi-experimentation: Design and analysis issues for field settings
Shrout, P. E. (1980) · Evaluation and Program Planning · doi:10.1016/0149-7189(80)90063-4 · Cited by 6,010
Visible learning: a synthesis of over 800 meta‐analyses relating to achievement
Carter, M. (2009) · Educational Psychology · doi:10.1080/01443410903415150 · Cited by 4,581
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553 · cited 2×
“This approach is considered especially appropriate for meta-analyses that include studies with “nested” results, that is, with potentially multiple effects within a given study (Kalaian & Raudenbush, 1996; Raudenbush, 2009; see also Dedrick et al., 2009).”
MethodConjectures and Refutations: The Growth of Scientific Knowledge.
Hutchison, T. W., & Popper, K. R. (1964) · Economica · doi:10.2307/2551371 · Cited by 4,197
Conjectures and Refutations: The Growth of Scientific Knowledge.
Hesse, M., & Popper, K. R. (1965) · The Philosophical Quarterly · doi:10.2307/2218271 · Cited by 4,189
Education and Treatment of Children
Hickman, J. A., Wheatley, J. K., & Sheridan, K. D. (2014) · Encyclopedia of Special Education · doi:10.1002/9781118660584.ese0816 · Cited by 3,223
Hattie, J. (2008) · doi:10.4324/9780203887332 · Cited by 3,051
How Does Professional Development Improve Teaching?
Kennedy, M. M. (2016) · Review of Educational Research · doi:10.3102/0034654315626800 · Cited by 1,305
Comprehensive School Reform and Achievement: A Meta-Analysis
Borman, G. D., Hewes, G. M., Overman, L. T., & Brown, S. (2003) · Review of Educational Research · doi:10.3102/00346543073002125 · Cited by 795
Theory of Instruction: Principles and Applications
Engelmann, S., & Carnine, D. W. (1982) · Medical Entomology and Zoology · Cited by 697
Teaching Disadvantaged Children in the Preschool
King, M. A. (1967) · Teachers College Record The Voice of Scholarship in Education · Cited by 638
Bloom, H. S., Hill, C. J., Black, A. R., & Lipsey, M. W. (2008) · Journal of Research on Educational Effectiveness · doi:10.1080/19345740802400072 · Cited by 477
Analyzing effect sizes: Random-effects models.
Raudenbush, S. W. (2009) · Cited by 418
(2012) · TigerPrints (Clemson University) · Cited by 402
Research on Direct Instruction: 25 Years beyond DISTAR.
Adams, G. L., & Engelmann, S. (1996) · Cited by 357
The Concepts of Power Analysis
Cohen, J. (1977) · Statistical Power Analysis for the Behavioral Sciences · doi:10.1016/b978-0-12-179060-8.50006-2 · Cited by 299
Estimating the Difference Between Published and Unpublished Effect Sizes: A Meta-Review
Polanin, J. R., Tanner-Smith, E. E., & Hennessy, E. A. (2016) · Review of Educational Research · doi:10.3102/0034654315582067 · Cited by 234 · cited 1×
“Similarly, while literature across the field of education has suggested that reported effects would be stronger in published than in unpublished sources (Polanin et al., 2016), we found no indication of this pattern.”
BackgroundA multivariate mixed linear model for meta-analysis.
Kalaian, H. A., & Raudenbush, S. W. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.3.227 · Cited by 229 · cited 1×
“This approach is considered especially appropriate for meta-analyses that include studies with “nested” results, that is, with potentially multiple effects within a given study (Kalaian & Raudenbush, 1996; Raudenbush, 2009; see also Dedrick et al., 2009).”
MethodMultilevel Modeling: A Review of Methodological Issues and Applications
Dedrick, R. F., Ferron, J. M., Hess, M. R., Hogarty, K. Y., Kromrey, J. D., Lang, T. R., Niles, J. D., & Lee, R. S. (2009) · Review of Educational Research · doi:10.3102/0034654308325581 · Cited by 209
Becker, W. C., & Gersten, R. M. (1982) · American Educational Research Journal · doi:10.2307/1162369 · Cited by 197
The Direct Instruction Follow Through Model: Design and Outcomes.
Engelmann, S. (1988) · Education and Treatment of Children · Cited by 132
Benner, G. J., Nelson, J. R., Stage, S. A., & Ralston, N. C. (2010) · Remedial and Special Education · doi:10.1177/0741932510361265 · Cited by 100
Is Inquiry-Based Science Teaching Worth the Effort? Some Thoughts Worth Considering
Zhang, L. (2016) · Science & Education · doi:10.1007/s11191-016-9856-0 · Cited by 98
Morgan, P. L., Farkas, G., & Maczuga, S. (2014) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373714536608 · Cited by 95
A Meta-Analysis of the Effects of Direct Instruction in Special Education.
White, W. A. (1988) · Education and Treatment of Children · Cited by 93
Findings from the Follow Through Planned Variation Study
Kennedy, M. M. (1978) · Educational Researcher · doi:10.3102/0013189x007006003 · Cited by 82
A constructive look at follow through results
Bereiter, C., & Kurland, M. (1981) · Interchange · doi:10.1007/bf01807392 · Cited by 73
Long-Term Academic Effects of the Direct Instruction Project Follow Through
Meyer, L. A. (1984) · The Elementary School Journal · doi:10.1086/461371 · Cited by 71
Carlson, C. D., & Francis, D. J. (2002) · Journal of Education for Students Placed at Risk (JESPAR) · doi:10.1207/s15327671espr0702_3 · Cited by 67
A Multifaceted Study of Change in Seven Inner-City Schools
Gersten, R. M., Carnine, D. W., Zoref, L., & Cronin, D. P. (1986) · The Elementary School Journal · doi:10.1086/461448 · Cited by 56
Ross, S. M., Nunnery, J. A., Goldfeder, E., McDonald, A., Rachor, R. E., Hornbeck, M., & Fleischman, S. (2004) · Journal of Education for Students Placed at Risk (JESPAR) · doi:10.1207/s15327671espr0904_3 · Cited by 53
Gersten, R. M., Carnine, D. W., & Williams, P. B. (1982) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737004001067 · Cited by 44
Direct Instruction Mathematics Programs: An Overview and Research Summary.
Przychodzin-Havis, A. M., Marchand‐Martella, N. E., Martella, R. C., & Azim, D. (2004) · Cited by 43
Special Education and Direct Instruction: An Effective Combination.
Kinder, D., Kubina, R. M., & Marchand‐Martella, N. E. (2005) · University of Washington Tacoma Digital Commons (University of Washington Tacoma) · Cited by 38
Iver, M. A. M., & Kemper, E. (2002) · Journal of Education for Students Placed at Risk (JESPAR) · doi:10.1207/s15327671espr0702_5 · Cited by 36
Stockard, J. (2010) · Journal of Education for Students Placed at Risk (JESPAR) · doi:10.1080/10824669.2010.495687 · Cited by 35
Self-citationCollins, M., & Carnine, D. W. (1988) · Journal of Learning Disabilities · doi:10.1177/002221948802100610 · Cited by 29
An Analysis of "Corrective Reading" Research.
Przychodzin-Havis, A. M., Marchand‐Martella, N. E., Martella, R. C., et al. (2005) · Cited by 28
An Analysis of the Reading Mastery Program: Effective Components and Research Review.
Schieffer, C., Marchand‐Martella, N. E., Martella, R. C., et al. (2002) · Cited by 26
Shadish, W. R. (2001) · International Encyclopedia of the Social & Behavioral Sciences · doi:10.1016/b0-08-043076-7/00419-8 · Cited by 24
Best Practices in Spelling Instruction: A Research Summary.
Simonsen, F. L., & Gunter, L. E. (2001) · Cited by 23
The Threshold and Inclusive Approaches to Determining “Best Available Evidence”
Stockard, J., & Wood, T. W. (2016) · American Journal of Evaluation · doi:10.1177/1098214016662338 · Cited by 21
Self-citationImplementing Research-Based Reading Programs in the Fort Worth Independent School District
O'Brien, D. M., & Ware, A. M. (2002) · Journal of Education for Students Placed at Risk (JESPAR) · doi:10.1207/s15327671espr0702_4 · Cited by 15
Stockard, J. (2011) · Scholars' Bank (University of Oregon) · Cited by 13
Self-citationStockard, J. (2011) · Quality & Quantity · doi:10.1007/s11135-011-9652-5 · Cited by 13
Self-citationVitale, M. R., & Joseph, B. L. (2008) · Cited by 7
Inferred Functions of Performance and Learning
Engelmann, S., & Steely, D. (2003) · Psychology Press eBooks · doi:10.4324/9781410609601 · Cited by 5
Vitale, M. R., & Kaniuka, T. S. (2012) · Cited by 2
Research works printed but not linked: 28
- Adams G., 1996, Effective School Practices, 15, 43
- American Institutes for Research, 1999, An educator’s guide to schoolwide reform
- Barbash S., 2012, Clear teaching: With Direct Instruction, Siegfried Engelmann discovered a better way of teaching
- Bereiter C., 1996, Effective School Practices, 15, 17
- Coughlin C., 2011, Research on the effectiveness of Direct Instruction
- Coughlin C., 2014, The science and success of Engelmann’s Direct Instruction, 25
- Engelmann S., 1999, Issues in educating students and disabilities, 177
- Engelmann S., 2007, Teaching needy kids in our backward system: 42 years of trying
- Engelmann S., 2014, Engelmann’s Direct Instruction: Selected writings from the past half century, 414
- Engelmann S., 2014, The science and success of Engelmann’s Direct Instruction, 3
- Engelmann S., 2014, Successful and confident students with Direct Instruction
- Engelmann S., 2011, Could John Stuart Mill have saved our schools?
- Engelmann S., 2006, Rubric for identifying authentic Direct Instruction programs
- Grossen B., 1996, Effective School Practices, 15, 4
- Huitt W. G., 2009, Instructional-design theories and models: Vol. 3. Building a common knowledge base, 73
- Kinder D., 2005, Journal of Direct Instruction, 5, 1
- National Reading Panel, 2000, Report of the National Reading Panel: Teaching children to read: an evidence-based assessment of the scientific research literature on reading and its implications
- Przychodzin A. M., 2004, Journal of Direct Instruction, 4, 53
- Przychodzin-Havis A. M., 2005, Journal of Direct Instruction, 5, 37
- Schieffer C., 2002, Journal of Direct Instruction, 2, 87
- Simonsen F., 2001, Journal of Direct Instruction, 1, 97
- Stockard J., 2011, Journal of Direct Instruction, 11, 31
- Tallmadge G. K., 1977, The Joint Dissemination Review Panel idea book
- Vitale M. R., 2008, Journal of Direct Instruction, 8, 1
- Vitale M. R., 2012, Journal of Direct Instruction, 12, 25
- Watkins C., 1996, Effective School Practices, 15, 56
- White W. A. T., 1988, Education & Treatment of Children, 11, 364
- Wood T. W., 2014, Engelmann’s Direct Instruction: Selected writings from the past half century
Cited by
The Development of Academic Achievement and Cognitive Abilities: A Bidirectional Perspective
Peng, P., & Kievit, R. A. (2020) · Child Development Perspectives · doi:10.1111/cdep.12352 · Cited by 497
“…and multiple or other academic subjects) among school-age children, academic instruction positively affected academic performance, and those effects transferred to performance on measures of ability and IQ—although the authors did not describe these measures in detail (Stockard et al., 2018).”
BackgroundThe cognitive benefits of learning computer programming: A meta-analysis of transfer effects.
Scherer, R., Siddiq, F., & Sánchez Viveros, B. (2019) · Journal of Educational Psychology · doi:10.1037/edu0000314 · Cited by 205
“.05; Jacobse & Harskamp, 2011; Stockard, Wood, Coughlin, & Khoury, 2018).”
When Problem Solving Followed by Instruction Works: Evidence for Productive Failure
Sinha, T., & Kapur, M. (2021) · Review of Educational Research · doi:10.3102/00346543211019105 · Cited by 212
“…in direct instruction, therefore, is that once targeted domain concepts have been formally introduced and worked examples presented to support solution schema construction, only then are students prepared to be subjected to (un)-guided problem solving in a subsequent phase (Stockard et al., 2018).”
Children’s Creativity: A Theoretical Framework and Systematic Review
Kupers, E., Lehmann-Wermser, A., McPherson, G., & van Geert, P. (2019) · Review of Educational Research · doi:10.3102/0034654318815707 · Cited by 195
Let's talk evidence – The case for combining inquiry-based and direct instruction
de Jong, T., Lazonder, A. W., Chinn, C. A., Fischer, F., Gobert, J., Hmelo-Silver, C. E., Koedinger, K. R., Krajcik, J. S., Kyza, E. A., Linn, M. C., et al. (2023) · Educational Research Review · doi:10.1016/j.edurev.2023.100536 · Cited by 190
Metadata
- Authors
- Stockard, J., Wood, T. W., Coughlin, C., & Rasplica Khoury, C.
- Affiliations
- University of Oregon (US)
- Safe and Civil Schools
- The Children’s Clinic, P.C.
- ORCID
- Stockard, Jean
- Venue
- Review of Educational Research
- Published
- 2018-01-07
- Volume
- 88
- Issue
- 4
- Pages
- 479-507
- ISSN
- 0034-6543 (ISSN-L)
- 1935-1046
- OpenAlex
- W2783703226
- Cited by
- 371
- References
- 54
- Citations by year
- 2026: 31 · 2025: 61 · 2024: 58 · 2023: 51 · 2022: 54 · 2021: 48 · 2020: 42 · 2019: 15 · 2018: 11
- FWCI
- 6.18
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Review of Educational Research (publishedVersion)
- Scholars' Bank (University of Oregon) (submittedVersion)
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Reading and Literacy Development
- Social Sciences › Psychology › Developmental and Educational Psychology › Behavioral and Psychological Studies
- Social Sciences › Education › Early Childhood Education and Development
- Keywords
- direct instruction, curriculum effectiveness, effect size, reading achievement, mathematics achievement, language arts, spelling, academic performance, affective learning outcomes, teacher and parent perceptions, single-case experimental design, software maintenance, educational disadvantage
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.