Impacts of Academic Recovery Interventions on Student Achievement in 2022 to 2023
Abstract
The COVID-19 pandemic caused large, persistent declines in student achievement. This paper examines 2022–2023 recovery efforts across eight districts, including tutoring, small-group instruction, after-school, extended year, double-dose, digital learning, and expert teacher interventions. The study includes 12 math programs and 15 literacy programs (three covered both subjects). Most served fewer students and provided lower dosage than planned. Using value-added models, we find positive, robust effects in just five subject-intervention pairs: one subject-specific tutoring program (+0.22 SD math, +0.22 SD reading), a small-group reading intervention (+0.33 SD), and assignment to “expert” teachers (+0.06 SD math, +0.11 SD reading). Results highlight the promise of intensive interventions while underscoring the difficulty of scaling them to meet recovery needs.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · see citing works
Unobservable Selection and Coefficient Stability: Theory and Evidence
Oster, E. (2019) · Journal of Business & Economic Statistics · doi:10.1080/07350015.2016.1227711 · Cited by 5,367 · see citing works
Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design
Hahn, J., Todd, P., & Klaauw, W. (2001) · Econometrica · doi:10.1111/1468-0262.00183 · Cited by 2,679 · see citing works
When Should You Adjust Standard Errors for Clustering?
Abadie, A., Athey, S., Imbens, G. W., & Wooldridge, J. M. (2022) · The Quarterly Journal of Economics · doi:10.1093/qje/qjac038 · Cited by 1,460 · see citing works
Interpreting effect sizes of education interventions
Kraft, M. A. (2020) · Educational Researcher · doi:10.3102/0013189x20912798 · Cited by 1,281 · see citing works
Education Technology: An Evidence-Based Review
Escueta, M., Quan, V., Nickow, A. J., & Oreopoulos, P. (2017) · doi:10.3386/w23744 · Cited by 389 · see citing works
The Effect of Grade Retention on High School Completion
Jacob, B. A., & Lefgren, L. J. (2009) · American Economic Journal Applied Economics · doi:10.1257/app.1.3.33 · Cited by 213 · see citing works
Kim, J. S., & Quinn, D. M. (2013) · Review of Educational Research · doi:10.3102/0034654313483906 · Cited by 213 · cited 1× · see citing works
“High-impact tutoring programs (Nickow et al., 2024), summer learning programs (Kim & Quinn, 2013; Lynch et al., 2023; McCombs et al., 2014), and double-dose math courses (Nomi & Allensworth, 2013) all have strong pre-pandemic evidence improving student achievement.”
Sacerdote, B. (2012) · American Economic Journal: Applied Economics · doi:10.1257/app.4.1.109 · Cited by 181 · see citing works
Using Implementation Fidelity to Aid in Interpreting Program Impacts: A Brief Review
Hill, H. C., & Erickson, A. (2019) · Educational Researcher · doi:10.3102/0013189x19891436 · Cited by 99 · see citing works
Nickow, A., Oreopoulos, P., & Quan, V. (2024) · American Educational Research Journal · doi:10.3102/00028312231208687 · Cited by 82 · cited 4× · see citing works
“Recent tutoring meta-analyses (Kraft, Schueler, & Falken, 2024; Nickow et al., 2024), for example, note more effective tutoring programs tend to use teachers or paraprofessionals as tutors, serve students in earlier grades, deliver tutoring during the school day, and occur at least 3 days per week…”
InfluentialThe Value of Out-of-School Time Programs
McCombs, J. S., Whitaker, A. A., & Yoo, P. (2017) · RAND Corporation eBooks · doi:10.7249/pe267 · Cited by 56 · see citing works
The Educational Consequences of Remote and Hybrid Instruction during the Pandemic
Goldhaber, D., Kane, T. J., McEachin, A., Morton, E., Patterson, T., & Staiger, D. O. (2023) · American Economic Review: Insights · doi:10.1257/aeri.20220180 · Cited by 49 · see citing works
Self-citationNomi, T., & Allensworth, E. M. (2012) · American Educational Research Journal · doi:10.3102/0002831212469997 · Cited by 45 · see citing works
Time in School: A Conceptual Framework, Synthesis of the Causal Research, and Empirical Exploration
Kraft, M. A., & Novicoff, S. (2024) · American Educational Research Journal · doi:10.3102/00028312241251857 · Cited by 40 · see citing works
How to Make Additional Time Matter: Integrating Individualized Tutorials into an Extended Day
Kraft, M. A. (2014) · Education Finance and Policy · doi:10.1162/edfp_a_00152 · Cited by 39 · see citing works
Combining independent estimators in research synthesis
Hedges, L. V. (1983) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1983.tb00768.x · Cited by 36 · see citing works
The Impact of Summer Programs on Student Mathematics Achievement: A Meta-Analysis
Lynch, K., An, L., & Mancenido, Z. (2022) · Review of Educational Research · doi:10.3102/00346543221105543 · Cited by 34 · see citing works
The Impact of Summer Learning Loss on Measures of School Performance
McEachin, A. J., & Atteberry, A. (2016) · Education Finance and Policy · doi:10.1162/edfp_a_00213 · Cited by 32 · see citing works
Self-citationMcCombs, J., Pane, J., Augustine, C., Schwartz, H., Martorell, P., & Zakaras, L. (2014) · doi:10.7249/rr815 · Cited by 28 · see citing works
Diminishing Marginal Returns to Computer‐Assisted Learning
Bettinger, E., Fairlie, R. W., Kapuza, A., Kardanova, E. Y., Loyalka, P. K., & Zakharov, A. (2022) · Journal of Policy Analysis and Management · doi:10.1002/pam.22442 · Cited by 21 · see citing works
Can Technology Facilitate Scale? Evidence from a Randomized Evaluation of High Dosage Tutoring
Bhatt, M., Guryan, J., Khan, S. H., LaForest, M., & Mishra, B. (2024) · National Bureau of Economic Research · doi:10.3386/w32510 · Cited by 17 · see citing works
Özek, U. (2021) · Journal of Public Economics · doi:10.1016/j.jpubeco.2021.104518 · Cited by 15 · see citing works
Language and Reading Research Consortium, Arthur, A. M., & Davis, D. L. (2016) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2015.1126875 · Cited by 14 · see citing works
A scalable approach to high-impact tutoring for young readers
Cortes, K. E., Kortecamp, K., Loeb, S., et al. (2024) · Learning and Instruction · doi:10.1016/j.learninstruc.2024.102021 · Cited by 12 · see citing works
What Do Changes in State Test Scores Imply for Later Life Outcomes?
Doty, E., Kane, T., Patterson, T., & Staiger, D. (2022) · doi:10.3386/w30701 · Cited by 11 · see citing works
Self-citationFederal Pandemic Relief and Academic Recovery
Dewey, D., Fahle, E. M., Kane, T. J., et al. (2024) · National Bureau of Economic Research · doi:10.3386/w32897 · Cited by 10 · see citing works
Self-citationCarbonari, M. V., Davison, M., DeArmond, M., Dewey, D., Dizon-Ross, E., Goldhaber, D. D., Hashim, A. K., Kane, T. J., McEachin, A. J., Morton, E., et al. (2024) · AERA Open · doi:10.1177/23328584241281286 · Cited by 10 · see citing works
Self-citationNomi, T. (2015) · Social Science Research · doi:10.1016/j.ssresearch.2014.12.012 · Cited by 5 · see citing works
The Scaling Dynamics and Causal Effects of a District-Operated Tutoring ProgramPreprint
Kraft, M. A., Edwards, D. S., & Cannata, M. A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5116286 · Cited by 4 · see citing works
A Global Regression Discontinuity Design: Theory and Application to Grade Retention PoliciesPreprint
Opper, I., & Özek, U. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4748020 · Cited by 3 · see citing works
The Effect of Student–Tutor Ratios: Experimental Evidence From a Pilot Online Math Tutoring Program
Kraft, M. A., & Lovison, V. S. (2025) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737251364573 · Cited by 3 · see citing works
The Scaling Dynamics and Causal Effects of a District-Operated Tutoring Program
(2024) · edworkingpapers.com · doi:10.26300/zcw7-4547
Flagged first, then by citations
1 of the 34 references couldn’t be matched to a known paper.
Research works printed but not linked: 13
- Barry S. S. Sass T. R. (2022). The impact of a 2021 Summer School Program on student achievement. Georgia Policy Labs. https://doi.org/10.57709/FAJ9-8597 · DOI
- Checkoway A., Gamse B., Velez M., Linkow T. (2013). Evaluation of the Massachusetts expanded learning time (ELT) initiative: Final study findings. Society for Research on Educational Effectiveness.
- Fahle E. M., Kane T. J., Patterson T., Reardon S. F., Staiger D. O., Stuart E. A. (2023). School district and community factors associated with learning loss during the COVID-19 pandemic. Center for Education Policy Research at Harvard University.
- Fahle E. M., Kane T. J., Reardon S. F., Staiger D. O. (2024). The first year of pandemic recovery: A district-level analysis. Center for Education Policy Research at Harvard University.
- Kuhfeld M., Diliberti M., McEachin A., Schweig J., Mariano L. T. (2023). Typical learning for whom? Guidelines for selecting benchmarks to calculate months of learning (Research brief). NWEA.
- Lewis K., Kuhfeld M. (2022). Progress toward pandemic recovery: Continued signs of rebounding achievement at the start of the 2022–23 school year (Research brief). NWEA.
- Lewis K., Kuhfeld M. (2023). Education’s long COVID: 2022–23 Achievement data reveal stalled progress toward pandemic recovery (Research brief). NWEA.
- Lewis K., Kuhfeld M. (2024). Recovery still elusive: 2023–24 Student achievement highlights persistent achievement gaps and a long road ahead (Research brief). NWEA.
- Özek U., Mariano L. T. (2023). Think again: Is grade retention bad for kids?. Thomas B. Fordham Institute.
- Raudenbush S. W. (2009). Analyzing effect sizes: Random-effects models. In Cooper H., Hedges L. V., Valentine J. C. (Eds.), The handbook of research synthesis and meta-analysis (pp. 295–315). Russell Sage Foundation.
- Ready D. D., McCormick S. G., Shmoys R. J. (2024). The effects of in-school virtual tutoring on student reading development: Evidence from a short-cycle randomized controlled trial (EdWorkingPaper No. 24-942). Annenberg Institute at Brown University.
- U.S. Department of Education. (2021). Education in a pandemic: The disparate impacts of COVID-19 on America’s students. U.S. Department of Education, Office of Civil Rights.
- U.S. Department of Education. (2023). Digest of education statistics: 2023. Institute of Education Sciences, National Center for Education Statistics.
Cited by
Sorted by citations
Exploring the Impact of COVID-19 on District Spending for English Learners
Santibañez, L., & Martin, M. S. (2025) · Teachers College Record · doi:10.1177/01614681251378290
“These students reported lower test scores and higher absenteeism, arguably undoing nearly two decades of progress toward educational equity in the United States (Carbonari et al., 2024; Santibañez et al., 2024).”
- Authors
- Carbonari, M. V., DeArmond, M., Dewey, D., Dizon-Ross, E., Goldhaber, D., Kane, T. J., McDonald, A., McEachin, A., Morton, E., Muroga, A., Salazar, A., & Staiger, D. O.
- Affiliations
- Harvard University (US)
- American Institutes for Research (US)
- Northwest Evaluation Association (US)
- Dartmouth College (US)
- Corresponding author
- Morton, Emily
- ORCID
- Venue
- Educational Evaluation and Policy Analysis
- Published
- 2026-02-10
- Volume
- 48
- Issue
- 3
- Pages
- 1045-1072
- ISSN
- 0162-3737 (ISSN-L)
- 1935-1062
- OpenAlex
- W7128516997
- Semantic Scholar
- 97ae31bc30aa2a5ffe2700ebbe2e9c4f5d1480d0
- Cited by
- 0
- References
- 34
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Educational Evaluation and Policy Analysis (publishedVersion)
- Funders
- AIR Equity Initiative: Educational Equity through Policy Implementation (AIR Equity Initiative: Educational Equity through Policy Implementation, 256843-5124839)
- Topics
- Social Sciences › Education › Education during COVID-19 pandemic
- Social Sciences › Education › School Choice and Performance
- Social Sciences › Demography › Global Educational Reforms and Inequalities
- Keywords
- COVID-19 pandemic, academic performance, tutoring, small group instruction, after-school programs, extended school year, online learning, literacy programs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review