Quality is Critical for Meaningful Synthesis of Afterschool Program Effects: A Systematic Review and Meta-analysis
The DOI matches its Crossref record; 39 of the 41 references it cites were found in a registry or an index.
Abstract
Afterschool programs are designed to promote academic achievement and youth development as well as mitigate risk for students who attend. Systematic reviews have reported that afterschool programs are associated with increased academic proficiency, school bonding, prosocial behavior, and decreased problem behavior. However, the majority of meta-analyses that report on these outcomes limit their samples to only rigorously conducted studies, which is not representative of the larger literature base. This systematic review and meta-analysis is designed to extend the literature by meta-analyzing a comprehensive sample of studies and examine the impact of study quality on meta-analytical results. A comprehensive literature and effect size extraction process search yielded 130 effect sizes from 30 studies that examined the effects of afterschool programs on secondary students' academic achievement and social / behavioral development. The majority of studies were unpublished reports, and study quality in the sample represented a high risk of bias. The results confirm the hypothesis that the inclusion of lower quality studies significantly impacts overall outcomes in comparison to prior meta-analyses that have limited samples. As such, it is imperative that researchers and practitioners increase the rigor of study design across individual program evaluations so that fewer reports will be excluded. This will improve future meta-analytic outcomes by ensuring a more representative sample of the literature base. Only through improved rigor of study design will a more clear understanding of program impacts on secondary students be possible.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We double coded each indicator and merged codes into a composite index score for analyses (Kremer et al., 2015;Lester et al., 2020).”
From The state of evidence for social and emotional learning: A contemporary meta-analysis of universal school-based SEL interventions (2023) · cites this paper as Lester et al., 2020
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
American Journal of Community Psychology
(2016) · American Journal of Community Psychology · doi:10.1002/(issn)1573-2770 · Cited by 1,449
Judd, C. H. (1928) · Journal of Education · doi:10.1177/002205742810700706 · Cited by 1,408
Durlak, J. A., Weissberg, R. P., & Pachan, M. (2010) · American Journal of Community Psychology · doi:10.1007/s10464-010-9300-6 · Cited by 1,216 · cited 10×
“As such, each study was coded dichotomously on four indicators of program quality that represent recommended practices in afterschool spaces (Durlak et al. 2010).”
BackgroundResultInfluentialQuality Indicators for Group Experimental and Quasi-Experimental Research in Special Education
Gersten, R., Fuchs, L. S., Compton, D., Coyne, M., Greenwood, C., & Innocenti, M. S. (2005) · Exceptional Children · doi:10.1177/001440290507100202 · Cited by 1,023
Out-of-School-Time Programs: A Meta-Analysis of Effects for At-Risk Students
Lauer, P. A., Akiba, M., Wilkerson, S. B., Apthorp, H. S., Snow, D., & Martin-Glenn, M. L. (2006) · Review of Educational Research · doi:10.3102/00346543076002275 · Cited by 579 · cited 3×
“For example, a meta-analysis of the effects of afterschool programs on reading and math achievement of K-12 students who are at risk of failure found a significant and positive treatment effect on reading and math (Lauer et al. 2006).”
BackgroundInfluentialCONSORT 2010 Statement: updated guidelines for reporting parallel group randomized trials.
Schulz, K. F., Altman, D. G., & Moher, D. (2010) · PubMed Central · Cited by 541
Research works printed but not linked: 3
- Alliance, A. (2016). America after 3pm special report: afterschool in communities of concentrated poverty. Washington, DC: Afterschool Alliance.
- *Carbone, P. M. (2010). The effects of an after-school tutoring program on the Pennsylvania System of Assessment. US: ProQuest Information & Learning.
- *Jarratt, K. (2014). Mathematics Achievement Outcomes for Middle School Students Attending School-Based Afterschool Mathematics Programs (Ed.D.). United States—Tennessee: Union University.
Cited by
Cipriano, C., Strambler, M. J., Naples, L. H., Ha, C., Kirk, M., Wood, M., Sehgal, K., Zieher, A. K., Eveleigh, A., McCarthy, M., et al. (2023) · Child Development · doi:10.1111/cdev.13968 · Cited by 636
“We double coded each indicator and merged codes into a composite index score for analyses (Kremer et al., 2015; Lester et al., 2020).”
Self-citationAdolescent peer processes in extracurricular activities: Identifying developmental opportunities
Berger, C., Deutsch, N. L., Cuadros, O., Franco-Chalco, E., Rojas, M., Roux, G., & Sánchez, F. (2020) · Children and Youth Services Review · doi:10.1016/j.childyouth.2020.105457 · Cited by 93
A meta‐analysis of the effects of after‐school programmes among youth with marginalized identities
Christensen, K. M., Kremer, K. P., Poon, C. Y. S., & Rhodes, J. E. (2023) · Journal of Community & Applied Social Psychology · doi:10.1002/casp.2681 · Cited by 30
“Regardless, after conducting a recent meta-analysis of primarily unpublished reports with high risk of bias, Lester et al. (2020) noted the importance of examining less rigorous studies, as they are more reflective of the current literature base on ASPs and how programmes operate in the ‘ real…”
MethodSantamaría, J. S., Cervantes, B. I. B., López-Garrido, F.-M., & Alcalá, D. H. (2021) · Sustainability · doi:10.3390/su13074047 · Cited by 25
“Evaluation/Assessment (n = 28) Author Year Country Studies (f) Sample (n) Purpose Content Outcome Zhao, Xu, Jiang and Ding [46] 2020 China 15 816 To study the performance of virtual reality (VR) anatomy education Augmented and virtual reality, improving classroom, teaching, learning strategies The finding confirms tha…”
ResultCipriano, C., Strambler, M. J., Naples, L. H., Ha, C., Kirk, M., Wood, M., Sehgal, K., Zieher, A. K., Eveleigh, A., McCarthy, M. F., et al. (2023) · doi:10.31219/osf.io/mk35u · Cited by 15
Self-citation
Metadata
- Authors
- Lester, A. M., Chow, J. C., & Melton, T. N.
- Affiliations
- Virginia Commonwealth University (US)
- University of Virginia (US)
- Corresponding author
- Lester, Ashlee M.
- Venue
- Journal of Youth and Adolescence
- Published
- 2020-01-02
- Volume
- 49
- Issue
- 2
- Pages
- 369-382
- ISSN
- 0047-2891 (ISSN-L)
- 1573-6601
- OpenAlex
- W2998324306
- Cited by
- 29
- References
- 41
- Citations by year
- 2026: 5 · 2025: 6 · 2024: 3 · 2023: 5 · 2022: 3 · 2021: 5 · 2020: 2
- FWCI
- 0.72
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Journal of Youth and Adolescence (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Youth Development and Social Support
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Keywords
- after-school programs, study quality, risk of bias, effect size heterogeneity, high school students, academic performance, youth development, prosocial behavior, behavioral problems, school connectedness, unpublished research
- MeSH terms
- Academic Success, Adolescent, Educational Measurement, Humans, Object Attachment, Peer Group, School Health Services, Schools, Students, Program Evaluation, Qualitative Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Quality is Critical for Meaningful Synthesis of Afterschool Program Effects: A Systematic Review and Meta-analysis. https://app.cochat.ai/papers/10.1007/s10964-019-01188-8
- The paper
- Lester, A. M., Chow, J. C., & Melton, T. N. (2020). Quality is Critical for Meaningful Synthesis of Afterschool Program Effects: A Systematic Review and Meta-analysis. Journal of Youth and Adolescence. https://doi.org/10.1007/s10964-019-01188-8
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.