The DOI matches its Crossref record; 36 of the 37 references it cites were found in a registry or an index.
Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects
Abstract
The view that the returns to educational investments are highest for early childhood interventions is widely held and stems primarily from several influential randomized trials—Abecedarian, Perry, and the Early Training Project—that point to super-normal returns to early interventions. This article presents a de novo analysis of these experiments, focusing on two core issues that have received limited attention in previous analyses: treatment effect heterogeneity by gender and overrejection of the null hypothesis due to multiple inference. To address the latter issue, a statistical framework that combines summary index tests with familywise error rate and false discovery rate corrections is implemented. The first technique reduces the number of tests conducted; the latter two techniques adjust the p values for multiple inference. The primary finding of the reanalysis is that girls garnered substantial short- and long-term benefits from the interventions, but there were no significant long-term benefits for boys. These conclusions, which have appeared ambiguous when using “naive” estimators that fail to adjust for multiple testing, contribute to a growing literature on the emerging female–male academic achievement gap. They also demonstrate that in complex studies where multiple questions are asked of the same data set, it can be important to declare the family of tests under consideration and to either consolidate measures or report adjusted and unadjusted p values.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In the results presented in Extended Data Tables 6 and 7, we also adjusted for the fact that we conducted multiple tests on the same dataset by implementing FDR corrections and report the FDR q values50. We also report the bootstrapped P value51 to account for regressions with a small number of clusters.”
From Last-mile delivery increases vaccine uptake in Sierra Leone (2024) · cites this paper as [50]
In the citing paper
“Finally, we note that our findings are robust to Type I error concerns. 10 We calculate sharpened two-stage q-values in Table 4 (Benjamini et al., 2006b; Anderson, 2008) and the observed impacts remain statistically significant at the 1 percent level.”
From Food Coma is Real: The Effect of Digestive Fatigue on Adolescents' Cognitive Performance (2024) · cites this paper as Anderson, 2008
In the citing paper
“We perform False Discovery Rate (FDR) adjustment using the method described in (77), considering each experiment (Reduced and Increased Exposure) separately and grouping the variables as:”
From Reranking partisan animosity in algorithmic social media feeds alters affective polarization (2025) · cites this paper as [77]
In the citing paper
“The outcomes used to create each index are quite diverse and group a variety of very different outcomes (e.g., crime, employment, education). It is difficult to interpret his indices. Moreover, the components of his indices change with age.”
From Analyzing social experiments as implemented: A reexamination of the evidence from the HighScope Perry Preschool Program (2010) · cites this paper as Anderson (2008)
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,417
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity
White, H. (1980) · Econometrica · doi:10.2307/1912934 · Cited by 25,861
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
A sharper Bonferroni procedure for multiple tests of significance
HOCHBERG, Y. (1988) · Biometrika · doi:10.1093/biomet/75.4.800 · Cited by 4,829
Adaptive linear step-up procedures that control the false discovery rate
Benjamini, Y., Krieger, A. M., & Yekutieli, D. (2006) · Biometrika · doi:10.1093/biomet/93.3.491 · Cited by 3,228
Experimental Analysis of Neighborhood Effects
Kling, J. R., Liebman, J. B., & Katz, L. F. (2007) · Econometrica · doi:10.1111/j.1468-0262.2007.00733.x · Cited by 2,719
(1986) · Journal of Economic Literature · Cited by 2,705
The importance of noncognitive skills: lessons from the GED testing program
Heckman, J. J., & Rubinstein, Y. (2001) · American Economic Review · doi:10.1257/aer.91.2.145 · Cited by 1,488
Carneiro, P. M., & Heckman, J. J. (2003) · doi:10.2139/ssrn.434544 · Cited by 1,325
(2005) · Medical Entomology and Zoology · Cited by 1,313
Significant Benefits: The High/Scope Perry Preschool Study Through Age 27
Schweinhart, L. J., Barnes, H., Weikart, D. P., & Ypsilanti, M. (1993) · Cited by 1,217
Procedures for Comparing Samples with Multiple Endpoints
O’Brien, P. C. (1984) · Biometrics · doi:10.2307/2531158 · Cited by 1,163
Early Childhood Education: Young Adult Outcomes From the Abecedarian Project
Campbell, F. A., Ramey, C. T., Pungello, E., Sparling, J., & Miller-Johnson, S. (2002) · Applied Developmental Science · doi:10.1207/s1532480xads0601_05 · Cited by 1,144
Stepwise Multiple Testing as Formalized Data Snooping
Romano, J. P., & Wolf, M. (2005) · Econometrica · doi:10.1111/j.1468-0262.2005.00615.x · Cited by 1,019
Early Childhood Education Programs
Currie, J. (2001) · The Journal of Economic Perspectives · doi:10.1257/jep.15.2.213 · Cited by 1,002
Empirical Benchmarks for Interpreting Effect Sizes in Research
Hill, C. J., Bloom, H. S., Black, A. R., & Lipsey, M. W. (2008) · Child Development Perspectives · doi:10.1111/j.1750-8606.2008.00061.x · Cited by 950
Does Head Start Make a Difference?
Currie, J., & Thomas, D. (1993) · National Bureau of Economic Research · doi:10.3386/w4406 · Cited by 927
Resampling-Based Multiple Testing.
Booth, J. G., Westfall, P. H., & Young, S. S. (1994) · Journal of the American Statistical Association · doi:10.2307/2291234 · Cited by 909
Abadie, A., Angrist, J. D., & Imbens, G. W. (2002) · Econometrica · doi:10.1111/1468-0262.00270 · Cited by 740
Exact and Approximate Stepdown Methods for Multiple Hypothesis Testing
Romano, J. P., & Wolf, M. (2005) · Journal of the American Statistical Association · doi:10.1198/016214504000000539 · Cited by 609
Campbell, F. A., & Ramey, C. T. (1995) · American Educational Research Journal · doi:10.3102/00028312032004743 · Cited by 383
Interference Between Units in Randomized Experiments
Rosenbaum, P. R. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001112 · Cited by 371
Head Start Impact Study: First Year Findings.
Puma, M. J., Bell, S., Cook, R., Heid, C. A., & López, M. (2005) · Cited by 278
Williams, V. S. L., Jones, L. V., & Tukey, J. W. (1999) · Journal of Educational and Behavioral Statistics · doi:10.2307/1165261 · Cited by 251
Resampling: The new statistics
Simon, J. L. (1992) · Cited by 230
Stepwise Multiple Testing as Formalized Data SnoopingPreprint
Wolf, M., & Romano, J. P. (2004) · SSRN Electronic Journal · doi:10.2139/ssrn.563209 · Cited by 143
Inequality, Too Much of a Good Thing
Krueger, A. B. (2018) · Routledge eBooks · doi:10.4324/9780429494468-4 · Cited by 119
Early Training Project, 1962-1979
Gray, S. W. (1988) · Harvard Dataverse · doi:10.7910/dvn/kez9tw · Cited by 75
Can intervention early prevent crime later? The abecedarian project compared with other programs
Clarke, S. H., & Campbell, F. A. (1998) · Early Childhood Research Quarterly · doi:10.1016/s0885-2006(99)80042-8 · Cited by 73
Into Adulthood: A Study of the Effects of Head Start
Oden, S. L., Schweinhart, L. J., & Weikart, D. P. (2000) · Cited by 66
Mizell, C. A., & Steelman, L. C. (2000) · Journal of Family Issues · doi:10.1177/019251300021007003 · Cited by 44
Anderson, M. (2005) · SSRN Electronic Journal · doi:10.2139/ssrn.812426 · Cited by 28
Self-citationRelationship between Exposureto Class Size Reductionand Student Achievementin California
Stecher, B. M., McCaffrey, D. F., & Bugliari, D. (2003) · Education Policy Analysis Archives · doi:10.14507/epaa.v11n40.2003 · Cited by 6
Flagged first, then by citations
1 of the 37 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2009) · doi:10.1515/9781400829828 · Cited by 9,467
The Oregon Health Insurance Experiment: Evidence from the First Year*
Finkelstein, A. N., Taubman, S. L., Wright, B. J., Mira Bernstein, A., B, J. G. A., Newhouse, J. P., ALLEN, H. L., D, K. B. A., & Oregon Health Study Group (2012) · The Quarterly Journal of Economics · doi:10.1093/qje/qjs020 · Cited by 1,661
Heckman, J., Pinto, R., & Savelyev, P. (2013) · American Economic Review · doi:10.1257/aer.103.6.2052 · Cited by 1,628
The rate of return to the HighScope Perry Preschool Program
Heckman, J. J., Moon, S. H., Pinto, R., Savelyev, P. A., & Yavitz, A. (2010) · Journal of Public Economics · doi:10.1016/j.jpubeco.2009.11.001 · Cited by 1,341
Human capital development before age five
Currie, J., & Almond, D. V. (2011) · Handbook of labour economics · doi:10.1016/s0169-7218(11)02413-0 · Cited by 1,229
“Anderson (2008) conducts a re-analysis of the Perry Preschool and Abcedarian data (and a third intervention called the Earkly Training Project) and finds that like MTO public housing experiment, the significant effects of the intervention was largely concentrated among girls.”
Metadata
- Authors
- Anderson, M. L.
- Affiliations
- University of California, Berkeley (US)
- Corresponding author
- Anderson, Michael L.
- ORCID
- Anderson, Michael L.
- Venue
- Journal of the American Statistical Association
- Publisher
- Informa UK Limited
- Published
- 2008-12-01
- Volume
- 103
- Issue
- 484
- Pages
- 1481-1495
- ISSN
- 0162-1459 (ISSN-L)
- 1537-274X
- OpenAlex
- W2099805722
- Cited by
- 2,410
- References
- 37
- Citations by year
- 2026: 210 · 2025: 258 · 2024: 262 · 2023: 266 · 2022: 258 · 2021: 240 · 2020: 187 · 2019: 147 · 2018: 117 · 2017: 85 · 2016: 93 · 2015: 65 · 2014: 47 · 2013: 68 · 2012: 37
- FWCI
- 99.90
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Topics
- Social Sciences › Education › Early Childhood Education and Development
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- gender differences, early childhood intervention, Abecedarian Project, treatment effect heterogeneity, family-wise error rate, false discovery rate, returns to education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
