Fidelity in After-School Program Intervention Research: A Systematic Review
Abstract
Objective: This study examined whether and to what extent researchers addressed intervention fidelity in research of after-school programs serving at-risk students. Method: Systematic review procedures were used to search, retrieve, select, and analyze studies for this review. Fifty-five intervention studies were assessed on the following components of intervention fidelity: strategies to enhance fidelity, measurement of fidelity, and use of fidelity data in data analysis and interpretation. Results: Of the 55 studies examined, only 55% reported well-defined intervention procedures, 42% used an intervention manual, 33% provided training on the intervention, 24% provided supervision for the implementers, 29% measured fidelity, only 4% used fidelity data in their analysis, and no studies reported the reliability of fidelity measures. Conclusion: Findings indicate an overall lack of attention to and reporting of intervention fidelity in after-school intervention studies. Implications for practice, policy, and research are discussed.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Clinicians are often faced with the challenge of choosing the best intervention for a child's school attendance problems that appear to be of various types.
“Clinicians are thus often faced with the challenge of choosing the best intervention for a child’s school attendance problems that appear to be of various types”
From Reconciling Contemporary Approaches to School Attendance and School Absenteeism: Toward Promotion and Nimble Response, Global Policy Review and Implementation, and Future Adaptability (Part 1) (2019) · cites this paper as Maynard et al., 2013
After-school program studies provide limited training and supervision for implementers.
“Research has found after-school program studies lack well-defined theories of change and intervention procedures, have poor utilization of treatment manuals, provide limited training and supervision for implementers, and infrequently measure fidelity (Maynard et al. 2013).”
From Effects of after-school programs with at-risk youth on attendance and externalizing behaviors: A systematic review and meta-analysis (2015) · cites this paper as Maynard et al. 2013
After-school program research often lacks clear theories of change and defined intervention procedures.
“Research has found after-school program studies lack well-defined theories of change and intervention procedures, have poor utilization of treatment manuals, provide limited training and supervision for implementers, and infrequently measure fidelity (Maynard et al. 2013).”
From Effects of after-school programs with at-risk youth on attendance and externalizing behaviors: A systematic review and meta-analysis (2015) · cites this paper as Maynard et al. 2013
After-school program studies infrequently measure fidelity.
“Research has found after-school program studies lack well-defined theories of change and intervention procedures, have poor utilization of treatment manuals, provide limited training and supervision for implementers, and infrequently measure fidelity (Maynard et al. 2013).”
From Effects of after-school programs with at-risk youth on attendance and externalizing behaviors: A systematic review and meta-analysis (2015) · cites this paper as Maynard et al. 2013
After-school program studies have poor utilization of treatment manuals.
“Research has found after-school program studies lack well-defined theories of change and intervention procedures, have poor utilization of treatment manuals, provide limited training and supervision for implementers, and infrequently measure fidelity (Maynard et al. 2013).”
From Effects of after-school programs with at-risk youth on attendance and externalizing behaviors: A systematic review and meta-analysis (2015) · cites this paper as Maynard et al. 2013
Cites
(2002) · Cited by 13,211 · see citing works
Durlak, J. A., & DuPre, E. P. (2008) · American Journal of Community Psychology · doi:10.1007/s10464-008-9165-0 · Cited by 5,304 · see citing works
(2005) · Cited by 3,748 · see citing works
Bellg, A. J., Treatment Fidelity Workgroup of the NIH Behavior Change Consortium, Borrelli, B., Resnick, B., Hecht, J., Minicucci, D. S., Ory, M., Ogedegbe, G., Orwig, D., Ernst, D., et al. (2004) · Health Psychology · doi:10.1037/0278-6133.23.5.443 · Cited by 2,620 · see citing works
A conceptual framework for implementation fidelity
Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007) · Implementation Science · doi:10.1186/1748-5908-2-40 · Cited by 2,559 · see citing works
Education and Treatment of Children
Hickman, J. A., & Wheatley, J. K. (2008) · Encyclopedia of Special Education · doi:10.1002/9780470373699.speced0741 · Cited by 1,938 · see citing works
Dusenbury, L. (2003) · Health Education Research · doi:10.1093/her/18.2.237 · Cited by 1,671 · see citing works
Durlak, J. A., Weissberg, R. P., & Pachan, M. (2010) · American Journal of Community Psychology · doi:10.1007/s10464-010-9300-6 · Cited by 1,216 · see citing works
O’Donnell, C. (2008) · Review of Educational Research · doi:10.3102/0034654307313793 · Cited by 1,037 · see citing works
Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence
Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. S. (1993) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.61.4.620 · Cited by 873 · see citing works
Treatment fidelity in outcome studies
Moncher, F. J., & Prinz, R. J. (1991) · Clinical Psychology Review · doi:10.1016/0272-7358(91)90103-2 · Cited by 797 · see citing works
Treatment integrity and therapeutic change: issues and research recommendations
Perepletchikova, F., & Kazdin, A. E. (2005) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.bpi045 · Cited by 638 · see citing works
Gearing, R. E., El-Bassel, N., Ghesquiere, A., Baldwin, S., Gillies, J., & Ngeow, E. (2011) · Clinical Psychology Review · doi:10.1016/j.cpr.2010.09.007 · Cited by 610 · see citing works
Out-of-School-Time Programs: A Meta-Analysis of Effects for At-Risk Students
Lauer, P. A., Akiba, M., Wilkerson, S. B., Apthorp, H. S., Snow, D., & Martin-Glenn, M. L. (2006) · Review of Educational Research · doi:10.3102/00346543076002275 · Cited by 579 · see citing works
Perepletchikova, F., Treat, T. A., & Kazdin, A. E. (2007) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.75.6.829 · Cited by 533 · see citing works
Noell, G. H., Witt, J. C., Slider, N. J., Connell, J. E., Gatti, S. L., Williams, K. L., Koenig, J. L., Resetar, J. L., & Duhon, G. J. (2005) · School Psychology Review · doi:10.1080/02796015.2005.12086277 · Cited by 453 · see citing works
Toward the Effective and Efficient Measurement of Implementation Fidelity
Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam‐Gerow, M. A. (2010) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0321-0 · Cited by 422 · see citing works
Chen, H.-T., & Rossi, P. H. (1983) · Evaluation Review · doi:10.1177/0193841x8300700301 · Cited by 420 · see citing works
Youth Psychotherapy Outcome Research: A Review and Critique of the Evidence Base
Weisz, J. R., Doss, A. J., & Hawley, K. M. (2004) · Annual Review of Psychology · doi:10.1146/annurev.psych.55.090902.141449 · Cited by 356 · see citing works
(2019) · Campbell Systematic Reviews · doi:10.1002/(issn)1891-1803 · Cited by 323 · see citing works
Moving From the Lab to the Field: The Role of Fidelity and Achieved Relative Intervention Strength
Hulleman, C. S., & Cordray, D. S. (2009) · Journal of Research on Educational Effectiveness · doi:10.1080/19345740802539325 · Cited by 291 · see citing works
Promoting intervention fidelity
Dumas, J. E., Lynch, A. M., Laughlin, J. E., Smith, E. P., & Prinz, R. J. (2001) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(00)00272-5 · Cited by 277 · see citing works
McIntyre, L. L., Gresham, F. M., Reed, F. D. D., & Reed, D. D. (2007) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2007.659-672 · Cited by 243 · see citing works
Treatment Integrity of School-Based Behavioral Intervention Studies: 1980–1990
Gresham, F. M., Gansle, K. A., Noell, G. H., Cohen, S., & Rosenblum, S. (1993) · School Psychology Review · doi:10.1080/02796015.1993.12085651 · Cited by 241 · see citing works
Resnick, B. M., Bellg, A. J., Borrelli, B., Francesco, C. D., Breger, R. K. R., Hecht, J., Sharp, D. L., Levesque, C. S., Orwig, D., Ernst, D., et al. (2005) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm2902s_8 · Cited by 215 · see citing works
Noell, G. H., Duhon, G. J., Gatti, S. L., & Connell, J. E. (2002) · School Psychology Review · doi:10.1080/02796015.2002.12086152 · Cited by 166 · see citing works
Scott-Little, C., Hamann, M. S., & Jurs, S. G. (2002) · American Journal of Evaluation · doi:10.1016/s1098-2140(02)00234-5 · Cited by 166 · see citing works
Scott-Little, C., Hamann, M. S., & Jurs, S. G. (2002) · American Journal of Evaluation · doi:10.1177/109821400202300403 · Cited by 166 · see citing works
Smith, S. W., Daunic, A. P., & Taylor, G. G. (2007) · Education and Treatment of Children · doi:10.1353/etc.2007.0033 · Cited by 153 · see citing works
Roth, J. L., Malone, L. M., & Brooks-Gunn, J. (2010) · American Journal of Community Psychology · doi:10.1007/s10464-010-9303-3 · Cited by 131 · see citing works
Intervention Fidelity in Special and General Education Research Journals
Swanson, E. A., Wanzek, J., Haring, C. D., Ciullo, S. P., & McCulley, L. V. (2011) · The Journal of Special Education · doi:10.1177/0022466911419516 · Cited by 127 · see citing works
Status of and Trends in Academic Intervention Research for Students with Emotional Disturbance
Mooney, P., Epstein, M. H., Reid, R. R., & Nelson, J. R. (2003) · Remedial and Special Education · doi:10.1177/07419325030240050301 · Cited by 124 · see citing works
Grant, S. P., Mayo-Wilson, E., Melendez-Torres, G. J., & Montgomery, P. (2013) · PLoS ONE · doi:10.1371/journal.pone.0065442 · Cited by 111 · see citing works
Effectiveness of cognitive therapy training
Milne, D. L., Baker, C., Blackburn, I., James, I. A., & Reichelt, K. (1999) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/s0005-7916(99)00011-7 · Cited by 93 · see citing works
Mayo‐Wilson, E. (2007) · American Journal of Public Health · doi:10.2105/ajph.2006.094169 · Cited by 66 · see citing works
Treatment Fidelity in Social Work Intervention Research: A Review of Published Studies
Naleppa, M. J., & Cagle, J. G. (2010) · Research on Social Work Practice · doi:10.1177/1049731509352088 · Cited by 65 · see citing works
Enhancing the Educational Achievement of At-Risk Youth
Schinke, S. P., Cole, K., & Poulin, S. R. (2000) · Prevention Science · doi:10.1023/a:1010076000379 · Cited by 62 · see citing works
After-school programs for adolescents: a review of evaluation research.
Apsler, R. (2009) · PubMed · Cited by 60 · see citing works
Afterschool Programs in America: Origins, Growth, Popularity, and Politics
Mahoney, J. L., Parente, M. E., & Zigler, E. (2009) · Journal of Youth Development · doi:10.5195/jyd.2009.250 · Cited by 59 · see citing works
The Growth In After-School Programs And Their Impact
Hollister, R. G. (2003) · Works - Scholarship, Research, & Creative Expression (Swarthmore College) · Cited by 54 · see citing works
Program Strength and Fidelity in Evaluation
Summerfelt, W. T. (2003) · Applied Developmental Science · doi:10.1207/s1532480xads0702_2 · Cited by 47 · see citing works
Attention to Treatment Fidelity in Social Work Outcomes: A Review of the Literature from the 1990s
Tucker, A. R., & Blythe, B. J. (2008) · Social Work Research · doi:10.1093/swr/32.3.185 · Cited by 41 · see citing works
Gottfredson, D. C., Cross, A. B., Wilson, D., Rorie, M. L., & Connell, N. M. (2009) · Prevention Science · doi:10.1007/s11121-009-0156-7 · Cited by 40 · see citing works
Treatment Implementation and Statistical Power
Cook, T. J., & Poole, W. K. (1982) · Evaluation Review · doi:10.1177/0193841x8200600305 · Cited by 39 · see citing works
Dworsky, A., & Courtney, M. (2009) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2008.10.134 · Cited by 35 · see citing works
Hester, P., Baltodano, H. M., Gable, R. A., Tonelson, S. W., & Hendrickson, J. M. (2003) · ODU Digital Commons (Old Dominion University) · Cited by 34 · see citing works
Kaye, S., & Osteen, P. J. (2011) · Children and Youth Services Review · doi:10.1016/j.childyouth.2011.06.020 · Cited by 23 · see citing works
After-School Programs at Public Elementary Schools, 2008
(2017) · Cited by 8 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Kearney, C. A., Gonzálvez, C., Graczyk, P. A., & Fornander, M. J. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02222 · Cited by 130 · see citing works
Kremer, K. P., Maynard, B. R., Polanin, J. R., Vaughn, M. G., & Sarteschi, C. M. (2015) · Journal of Youth and Adolescence · doi:10.1007/s10964-014-0226-4 · Cited by 132 · see citing works
Self-citationRixon, L., Baron, J., McGale, N., Lorencatto, F., Francis, J., & Davies, A. (2016) · BMC Health Services Research · doi:10.1186/s12913-016-1904-6 · Cited by 117 · see citing works
Singh, S. K., & Gorey, K. M. (2018) · Social Work in Mental Health · doi:10.1080/15332985.2017.1373266 · Cited by 32 · see citing works
- Authors
- Maynard, B. R., Peters, K. E., Vaughn, M. G., & Sarteschi, C. M.
- Affiliations
- Saint Louis University (US)
- Chatham University (US)
- Corresponding author
- Maynard, Brandy R.
- Venue
- Research on Social Work Practice
- Publisher
- SAGE Publications
- Published
- 2013-06-04
- Volume
- 23
- Issue
- 6
- Pages
- 613-623
- ISSN
- 1049-7315 (ISSN-L)
- 1552-7581
- OpenAlex
- W2075628460
- Cited by
- 30
- References
- 48
- Citations by year
- 2025: 1 · 2023: 3 · 2022: 8 · 2021: 3 · 2019: 2 · 2017: 5 · 2016: 4 · 2015: 2 · 2014: 2
- FWCI
- 4.83
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Research on Social Work Practice (publishedVersion)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Safety Research › Youth Development and Social Support
- Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
- Keywords
- intervention fidelity, after-school programs, at-risk students, fidelity assessment, intervention manual, employee training, supervision, reporting quality, program implementation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
