Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 62 of the 74 references it cites were found in a registry or an index; 2 of the 20 sources its claims cite were checked for support.

Not Too Late: Improving Academic Outcomes among Adolescents

Abstract

Improving academic outcomes for economically disadvantaged students has proven challenging, particularly for children at older ages. We present two large-scale randomized controlled trials of a high-dosage tutoring program delivered to secondary school students in Chicago. One innovation is to use paraprofessional tutors to hold down cost, thereby increasing scalability. Participating in math tutoring increases math test scores by 0.18 to 0.40 standard deviations, and increases math and nonmath course grades. These effects persist into future years. The data are consistent with increased personalization of instruction as a mechanism. The benefit-cost ratio is comparable to many successful early childhood programs. (JEL H75, I21, I24, I26, I32, J13, J15)

Overview

CoChat Summary

TL;DR

Two large randomized trials of Saga Education’s in-school, high-dosage math tutoring for Chicago public high school students found substantial gains in math test scores and grades, with effects persisting into 11th grade. The authors argue that intensive tutoring can improve adolescent academic outcomes at a cost of $3,500–$4,300 per participant per year, with benefit-cost ratios comparable to model early-childhood programs.

Key findings

  • In study 1, the treatment-on-the-treated (TOT) effect after one year was +0.16 SD on standardized math tests, +0.56 points on math GPA (control complier mean 1.63), and a 49% decline in math course failures. In study 2, TOT effects were +0.37 SD on math tests, +0.42 points on math GPA (CCM 1.79), and a 44% decline in math course failures. Pooled year-1 TOT effects were +0.26 SD on math tests, +0.52 math GPA points (CCM 1.67), and a 47% decline in math course failures.
  • Pooled 11th-grade outcomes showed TOT effects of +0.22 SD on math tests and +0.18 math GPA points. The TOT estimate for on-time high school graduation was +1.9 percentage points relative to a CCM of 77.2%, but imprecise: the standard error of 3.4 percentage points means declines as large as −4.8 points or gains as large as +8.6 points cannot be ruled out.
  • Mechanism tests found no detectable effects on adult supports, grit, conscientiousness, locus of control, or social networks, and no evidence that classroom management was a key driver. There were gains in liking math and caring about math grades, and suggestive evidence consistent with personalization/floor effects.
  • Estimated benefit-cost ratios were 2.1–2.6 in study 1 and 4.9–6.0 in study 2, with per-participant costs of $3,500–$4,300 per year.

Methods

  • Two large-scale randomized controlled trials in Chicago Public Schools. Study 1 randomized 2,633 rising 9th/10th grade male students in summer 2013 across 12 CPS high schools; study 2 randomized 2,710 students (2,645 unique) in summer 2014 across 15 schools, including 9th/10th grade girls in seven schools. Randomization was conditional on school-by-grade blocks.
  • The intervention was Saga Education’s in-school tutoring: one class period daily, two students per tutor, up to 140 contact hours per year, using Saga’s Algebra curriculum. Study 1 provided up to two years of intervention; study 2 ran for one year. Data included CPS administrative records, Saga attendance and internal assessments, Chicago Police arrest data, and two waves of in-person ISR surveys with researcher-administered math tests.
  • Analysis estimated intent-to-treat (ITT) and treatment-on-the-treated (TOT) effects using randomization as an instrument for participation, with randomization-block fixed effects, baseline controls, permutation tests, false discovery rate q-values, and robustness checks using multiple imputation and quantile regression.

Limitations

  • The paper states that graduation estimates are imprecise: the TOT estimate is +1.9 percentage points with a standard error of 3.4 percentage points, so effects ranging from −4.8 to +8.6 points cannot be ruled out.
  • The paper states that floor effects on CPS math tests complicate interpretation of heterogeneity and personalization; item-level tests are underpowered (p=.53), and the authors cannot directly document the instructional-level mechanism because they lack direct measures of the academic level at which tutors or classroom teachers target instruction.
  • Post-randomization data have missingness (e.g., study 1 attendance 4.4% for controls vs. 6.2% for treatment; grades 14.6% vs. 14.8%; test scores 29.8% vs. 29.5%), though treatment-control differences are not statistically significant. The provided text is an NBER working paper and states it has not been peer-reviewed.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1257/aer.20210434

Cites

62 of 62 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 SUTVA Analysis: Block-level randomization rate plotted against TOT effect on math test score
Figure 1 SUTVA Analysis: Block-level randomization rate plotted against TOT effect on math test scoreFrom the PDF
Figure 2 Random Forests for Treatment Heterogenity: Predicted treatment effects against their rank, standardized math test data. Average PTE's for each quartile are presented.
Figure 2 Random Forests for Treatment Heterogenity: Predicted treatment effects against their rank, standardized math test data. Average PTE's for each quartile are presented.From the PDF
Figure 3 Random Forests for Treatment Heterogenity: Plotting PTEs of multiple outcomes - Math GPA vs Math test scores. Clusters estimated using k-means, k=4; k chosen through data-driven selection method.
Figure 3 Random Forests for Treatment Heterogenity: Plotting PTEs of multiple outcomes - Math GPA vs Math test scores. Clusters estimated using k-means, k=4; k chosen through data-driven selection method.From the PDF
Figure 4 Treatment effects on individual ISR items against item-level difficulty for students above/below the median baseline standardized math test score. Above/Below median is calculated for all students taking the ISR math asessment. The lines shown are local polynomial fitted lines to see any underlying nonlinearities in the relationships between item-level impacts and difficulty. We also include [0.5, 0.95] confidence intervals on the fitted lines. This plot truncates the 3 easiest items (P-plus > 0.75) to better display the graph; the full version is included in the appendices.
Figure 4 Treatment effects on individual ISR items against item-level difficulty for students above/below the median baseline standardized math test score. Above/Below median is calculated for all students taking the ISR math asessment. The lines shown are local polynomial fitted lines to see any underlying nonlinearities in the relationships between item-level impacts and difficulty. We also include [0.5, 0.95] confidence intervals on the fitted lines. This plot truncates the 3 easiest items (P-plus > 0.75) to better display the graph; the full version is included in the appendices.From the PDF
Figure 1 Random Forests for Treatment Heterogeneity - Best Linear Predictor test
Figure 1 Random Forests for Treatment Heterogeneity - Best Linear Predictor testFrom the PDF
Figure 2 Random Forests for Treatment Heterogenity: Predicted treatment effects against their rank, math GPA. Average PTE's for each quartile are presented.
Figure 2 Random Forests for Treatment Heterogenity: Predicted treatment effects against their rank, math GPA. Average PTE's for each quartile are presented.From the PDF
Figure 3 Percent of students correctly answering math items given on the ISR math assessment. Each point corresponds to a single item.
Figure 3 Percent of students correctly answering math items given on the ISR math assessment. Each point corresponds to a single item.From the PDF
Figure 4 Treatment effects on individual ISR items against item-level difficulty for students above/below the median baseline standardized math test score. Above/Below median is calculated for all students taking the ISR math asessment. The lines shown are local polynomial fitted lines to see any underlying nonlinearities in the relationships between item-level impacts and difficulty. We also include [0.5, 0.95] confidence intervals on the fitted lines.
Figure 4 Treatment effects on individual ISR items against item-level difficulty for students above/below the median baseline standardized math test score. Above/Below median is calculated for all students taking the ISR math asessment. The lines shown are local polynomial fitted lines to see any underlying nonlinearities in the relationships between item-level impacts and difficulty. We also include [0.5, 0.95] confidence intervals on the fitted lines.From the PDF

Verification

Claims the paper makes

2 Valid · 3 Not valid · 15 Not checked · 215 Own results and interpretations

69 more claims

Create a free account to see all 235 claims and to verify the 1 reference still open.

See all claims — free account
  • Probably not a substantial claim. 70
  • It may state two separate claims as one. 80
  • The quote only partly states this claim. 62

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Research works printed but not linked: 5

  1. Gordon, Robert, Thomas J. Kane, and Douglas O. Staiger. 2006. Identifying Effective Teachers Using Performance on the Job. Washington, DC: Brookings Institution Press.
  2. Heckman, James J. 2013. Giving Kids a Fair Chance. Cambridge, MA: MIT Press.
  3. Kraft Matthew A, 2021, AERA Open, 7, 1
  4. Krueger, Alan B. 2003. "Inequality, Too Much of a Good Thing." In Inequality in America: What Role for Human Capital Policies? Edited by James J. Heckman and Alan B. Krueger, 1-76. Cambridge, MA: MIT Press.
  5. Pedregosa Fabian, 2011, Journal of Machine Learning Research, 12, 2825

Cited by

See all 78

Metadata

Authors
Guryan, J., Ludwig, J., Bhatt, M. P., Cook, P. J., Davis, J. M. V., Dodge, K., Farkas, G., Fryer, R. G., Mayer, S., Pollack, H., Steinberg, L., & Stoddard, G.
Affiliations
  • Northwestern University (US)
  • University of Chicago (US)
  • Duke University (US)
  • University of Oregon (US)
  • University of California, Irvine (US)
  • Harvard University (US)
  • Temple University (US)
Venue
American Economic Review
Published
2023-03-01
Volume
113
Issue
3
Pages
738-765
ISSN
0002-8282 (ISSN-L)
OpenAlex
W3135190335
Cited by
77
References
74
Citations by year
2026: 18 · 2025: 24 · 2024: 17 · 2023: 15
FWCI
56.32
Type
journal article
Language
en
Open access
closed
Locations
  • American Economic Review (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
Topics
  • Social Sciences › Education › School Choice and Performance
  • Social Sciences › Education › Global Educational Reforms and Inequalities
  • Social Sciences › Education › Early Childhood Education and Development
Keywords
academic tutoring, low-income students, adolescents, high school students, mathematics scores, course grades, scalability, benefit-cost ratio, late intervention, Chicago

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.