The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Interpreting Effect Sizes of Education Interventions
Open accessAbstract
Researchers commonly interpret effect sizes by applying benchmarks proposed by Jacob Cohen over a half century ago. However, effects that are small by Cohen’s standards are large relative to the impacts of most field-based interventions. These benchmarks also fail to consider important differences in study features, program costs, and scalability. In this article, I present five broad guidelines for interpreting effect sizes that are applicable across the social sciences. I then propose a more structured schema with new empirical benchmarks for interpreting a specific class of studies: causal research on education interventions with standardized achievement outcomes. Together, these tools provide a practical approach for incorporating study features, costs, and scalability into the process of interpreting the policy importance of effect sizes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“the math finding would be considered large and the literacy effect size medium‐large (Kraft, 2020).”
From Montessori education's impact on academic and nonacademic outcomes: A systematic review (2023) · cites this paper as Kraft, 2020
In the citing paper
“Effect sizes tend to be larger with small‐N studies (Kraft, 2020)”
From Montessori education's impact on academic and nonacademic outcomes: A systematic review (2023) · cites this paper as Kraft, 2020
In the citing paper
“Indeed, wise interventions are often low cost, disproportionately benefit low performing groups, have effects that can be sustained over time and context, and, in contrast to interventions that target specialized knowledge, can have effects on a broad range of academic, behavioral, and psychological outcomes.”
From Tackling Educational Inequalities with Social Psychology: Identities, Contexts, and Interventions (2021) · cites this paper as Kraft, 2020
In the citing paper
“Indeed, wise interventions are often low cost, disproportionately benefit low performing groups, have effects that can be sustained over time and context, and, in contrast to interventions that target specialized knowledge, can have effects on a broad range of academic, behavioral, and psychological outcomes.”
From Tackling Educational Inequalities with Social Psychology: Identities, Contexts, and Interventions (2021) · cites this paper as Kraft, 2020
In the citing paper
“These characteristics make wise interventions particularly suitable for scaling up, even where the average size of their effect on the overall population is relatively small (Bakker et al., 2019;Kraft, 2020;Slavin & Madden, 2011).”
From Tackling Educational Inequalities with Social Psychology: Identities, Contexts, and Interventions (2021) · cites this paper as Kraft, 2020
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011) · Child Development · doi:10.1111/j.1467-8624.2010.01564.x · Cited by 10,854 · cited 1×
“…effects on teachers’ instructional practice (0.47 SD) than on students’ achievement (0.18 SD; Kraft, Blazar, & Hogan, 2018), and social-emotional learning (SEL) programs have much larger effects on students’ SEL skills (0.57 SD) compared to their academic performance (0.27 SD; Durlak et al., 2011).”
BackgroundInfluentialEvaluating Effect Size in Psychological Research: Sense and Nonsense
Funder, D. C., & Ozer, D. J. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919847202 · Cited by 3,295 · cited 2×
“First, I update prior reviews (Coe, 2002; Bloom et al., 2008, Lipsey et al. 2012) with insights from a number of new articles (e.g. Cheung & Slavin, 2016; Simpson, 2017; Soland & Meng Thum, 2019; Lortie-Forgues & Inglis, 2019; Baird & Pane, 2019; Funder & Ozer, 2019; Schäfer & Schwarz, 2019).”
The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring
BLOOM, B. S. (1984) · Educational Researcher · doi:10.3102/0013189x013006004 · Cited by 2,734 · cited 3×
“The Bloom (1984) “2-sigma” effects on probability and cartography tests after a month of tutoring are 8 to 20 times larger than the effects on standardized math tests found in several recent studies of daily tutoring over an entire school year (Kraft, 2015; Cook et al.”
InfluentialLipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824
Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood
Chetty, R., Friedman, J. N., & Rockoff, J. E. (2014) · American Economic Review · doi:10.1257/aer.104.9.2633 · Cited by 1,717
On the Specification and Estimation of the Production Function for Cognitive Achievement
Todd, P. E., & Wolpin, K. I. (2003) · The Economic Journal · doi:10.1111/1468-0297.00097 · Cited by 1,514
A national experiment reveals where a growth mindset improves achievement
Yeager, D. S., Hanselman, P., Walton, G. M., Murray, J. S., Crosnoe, R., Muller, C., Tipton, E., Schneider, B., Hulleman, C. S., Hinojosa, C. P., et al. (2019) · Nature · doi:10.1038/s41586-019-1466-y · Cited by 1,513
The rate of return to the HighScope Perry Preschool Program
Heckman, J. J., Moon, S. H., Pinto, R., Savelyev, P. A., & Yavitz, A. (2010) · Journal of Public Economics · doi:10.1016/j.jpubeco.2009.11.001 · Cited by 1,341 · cited 1×
“For example, some early evaluations of center-based early childhood education programs, such as the HighScope Perry Preschool Project, compare treatment students to control group students who were almost exclusively cared for by guardians at home (Heckman et al., 2010).”
Kraft, M. A., Blazar, D., & Hogan, D. (2018) · Review of Educational Research · doi:10.3102/0034654318759268 · Cited by 1,202
Self-citation
Research works printed but not linked: 17
- Boulay B., 2018, The Investing in Innovation Fund: Summary of 67 evaluations. Final report. NCEE 2018-4013
- Boyd D., 2008, Overview of measuring effect sizes: The effect of measurement error. Brief 2
- Cohen J., 1969, Statistical power analysis for the behavioral sciences
- Cohen J., 1988, Statistical power analysis for the behavioral sciences
- Coe R., 2002, It’s the effect size, stupid: What effect size is and why it is important
- Cook P. J., 2015, Not too late: Improving academic outcomes for disadvantaged youth
- Fryer R. G, 2017, Handbook of economic field experiments, 2, 95
- Glass G. V., 1981, Meta-analysis in social research
- Gueron J. M., 2013, Fighting for reliable evidence
- Hattie J., 2009, Visible learning: A synthesis of 800+ meta-analyses on achievement
- Kraft M. A., 2018, Bush-Obama school reform: Lessons learned, 69
- Lipsey M. W., 2012, Translating the statistical representation of the effects of education interventions into more readily interpretable forms
- Puma M., 2010, Head Start impact study. Final report
- Rosenthal R., 2000, Contrasts and effect sizes in behavioral research: A correlational approach
- Schwartz A. E., Journal of Policy Analysis and Management
- 10.3102/00346543057002175 · doi:10.3102/00346543057002175 · Paper page
- What Works Clearinghouse, What Works Clearninghouse procedures handbook, Version 4.0
Cited by
Lakens, D. (2022) · Collabra Psychology · doi:10.1525/collabra.33267 · Cited by 1,962
Learning loss due to school closures during the COVID-19 pandemic
Engzell, P., Frey, A., & Verhagen, M. D. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2022376118 · Cited by 1,848
What can be learned from growth mindset controversies?
Yeager, D. S., & Dweck, C. S. (2020) · American Psychologist · doi:10.1037/amp0000794 · Cited by 868
Cipriano, C., Strambler, M. J., Naples, L. H., Ha, C., Kirk, M., Wood, M., Sehgal, K., Zieher, A. K., Eveleigh, A., McCarthy, M., et al. (2023) · Child Development · doi:10.1111/cdev.13968 · Cited by 636
Small Effects: The Indispensable Foundation for a Cumulative Psychological Science
Götz, F. M., Gosling, S. D., & Rentfrow, P. J. (2022) · Perspectives on Psychological Science · doi:10.1177/1745691620984483 · Cited by 538
Metadata
- Authors
- Kraft, M. A.
- Affiliations
- Brown University (US)
- Corresponding author
- Kraft, Matthew A.
- ORCID
- Kraft, Matthew A.
- Venue
- Educational Researcher
- Published
- 2020-04-27
- Volume
- 49
- Issue
- 4
- Pages
- 241-253
- ISSN
- 0013-189X (ISSN-L)
- 1935-102X
- OpenAlex
- W2968077443
- Cited by
- 1,281
- References
- 52
- Citations by year
- 2026: 236 · 2025: 226 · 2024: 199 · 2023: 265 · 2022: 165 · 2021: 121 · 2020: 57
- FWCI
- 92.35
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Educational Researcher (publishedVersion)
- Brown Digital Repository
- EdWorkingPapers.com
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Social Sciences › Education › Educational Research and Methods
- Keywords
- effect size interpretation, educational intervention, program costs, scalability, causal research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Interpreting Effect Sizes of Education Interventions. https://app.cochat.ai/papers/10.3102/0013189x20912798
- The paper
- Kraft, M. A. (2020). Interpreting Effect Sizes of Education Interventions. Educational Researcher. https://doi.org/10.3102/0013189x20912798
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.