Can Research Design Explain Variation in Head Start Research Results? A Meta-Analysis of Cognitive and Achievement Outcomes
The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
This study explores the extent to which differences in research design explain variation in Head Start program impacts. We employ meta-analytic techniques to predict effect sizes for cognitive and achievement outcomes as a function of the type and rigor of research design, quality and type of outcome measure, activity level of control group, and attrition. Across program evaluations, the average program-level effect size was 0.27 standard deviations. About 41% of the variation in estimates across evaluations can be explained by research design features, including the extent to which the control group experienced other forms of early care or education, and 11% of the variation within programs can be explained by the quality and type of the outcome measures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cook, T. D., Shadish, W. R., & Wong, V. C. (2008) · Journal of Policy Analysis and Management · doi:10.1002/pam.20375 · Cited by 604 · cited 2×
“…in certain circumstances rigorous quasi-experimental methods can produce causal estimates similar to those produced by randomized controlled trials (Cook et al., 2008) and further support the use of such methods to evaluate programs when randomized controlled trials are not feasible, as is often…”
BackgroundMethodShadish, W. R., Clark, M. H., & Steiner, P. M. (2008) · Journal of the American Statistical Association · doi:10.1198/016214508000000733 · Cited by 479 · cited 3×
“, 2008; Shadish et al., 2008; Shadish et al., 2011). Our meta-analysis is a useful and robust method to summarize studies from Head Start and to improve our understanding of how research design predicts effect sizes. Nevertheless, this study should serve only as an important first step in this line of inquiry. Future…”
Background
Cited by
Frank, K. A., Maroulis, S. J., Duong, M. Q., & Kelcey, B. M. (2013) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373713493129 · Cited by 406
Impacts of Early Childhood Education on Medium- and Long-Term Educational Outcomes
McCoy, D. C., Yoshikawa, H., Ziol-Guest, K. M., Duncan, G. J., Schindler, H. S., Magnuson, K., Yang, R., Koepp, A., & Shonkoff, J. P. (2017) · Educational Researcher · doi:10.3102/0013189x17737739 · Cited by 284
Self-citationSchindler, H. S., Kholoptseva, J., Oh, S. S., Yoshikawa, H., Duncan, G. J., Magnuson, K. A., & Shonkoff, J. P. (2015) · Journal of School Psychology · doi:10.1016/j.jsp.2015.04.001 · Cited by 121
Self-citationA framework for learning from null results
Jacob, R. T., Doolittle, F., Kemple, J., & Somers, M.-A. (2019) · Educational Researcher · doi:10.3102/0013189x19891955 · Cited by 65
“Researchers have approached the problem of null results from a variety of perspectives over the past decade, noting a number of potential contributors (see e.g., Landis et al., 2014; Lemons et al., 2014; Shager et al., 2013).”
BackgroundEarly Childhood Education by Television: Lessons from Sesame Street
Kearney, M. S., & Levine, P. B. (2019) · American Economic Journal: Applied Economics · doi:10.1257/app.20170300 · Cited by 48
Metadata
- Authors
- Shager, H. M., Schindler, H. S., Magnuson, K. A., Duncan, G. J., Yoshikawa, H., & Hart, C. M. D.
- Venue
- Educational Evaluation and Policy Analysis
- Published
- 2013-03
- Volume
- 35
- Issue
- 1
- Pages
- 76-95
- ISSN
- 0162-3737 (ISSN-L)
- 1935-1062
- OpenAlex
- W2116581208
- Cited by
- 65
- References
- 2
- FWCI
- 6.05
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Can Research Design Explain Variation in Head Start Research Results? A Meta-Analysis of Cognitive and Achievement Outcomes. https://app.cochat.ai/papers/10.3102/0162373712462453
- The paper
- Shager, H. M., Schindler, H. S., Magnuson, K. A., Duncan, G. J., Yoshikawa, H., & Hart, C. M. D. (2013). Can Research Design Explain Variation in Head Start Research Results? A Meta-Analysis of Cognitive and Achievement Outcomes. Educational Evaluation and Policy Analysis. https://doi.org/10.3102/0162373712462453
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.