Defining, Conceptualizing, and Measuring Fidelity of Implementation and Its Relationship to Outcomes in K–12 Curriculum Intervention Research
The DOI matches its Crossref record; 97 of the 97 references it cites were found in a registry or an index.
Abstract
Education researchers are being asked to conduct rigorous, scientifically based studies of K–12 curriculum interventions; therefore, the need for measuring fidelity of implementation and empirically relating it to outcomes (the chief rationale for this review) is warranted to ensure internal and external validity. The results of this review indicate that there are too few studies to guide researchers on how fidelity of implementation to core curriculum interventions can be measured and related to outcomes, particularly within efficacy and effectiveness studies, where the requirements for fidelity measures differ. This review attempts to clarify the definition, conceptualization, and measurement of fidelity of implementation and to guide future researchers in understanding how fidelity of implementation can be used to adjust or interpret outcome measures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rogers, E. M., Singhal, A., & Quinlan, M. M. (2019) · An Integrated Approach to Communication Theory and Research · doi:10.4324/9780203710753-35 · Cited by 37,322
Valente, T. W. (2003) · Genetics in Medicine · doi:10.1097/01.gim.0000061743.67794.c4 · Cited by 13,723
From dissertation abstracts international
(1974) · ACM SIGGRAPH Computer Graphics · doi:10.1145/988616.988619 · Cited by 12,099
The Science of “Muddling Through”
Lindblom, C. E. (1959) · Public Administration Review · doi:10.2307/973677 · Cited by 8,513
Review of research in education
Shulman, L. S., Mowry, S., & Berliner, D. C. (1973) · F.E. Peacock eBooks · Cited by 7,754
Principles of Instructional Design
Gagné, R. M., & Briggs, L. J. (1974) · Cited by 4,467
The New Meaning of Educational Change
Fullan, M. (2001) · doi:10.4324/9780203986561 · Cited by 4,373
Saeverot, H. (2022) · Indirect Education · doi:10.4324/9781003193463-8 · Cited by 3,059
Utilization-Focused Evaluation
(2018) · The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation · doi:10.4135/9781506326139.n729 · Cited by 2,308
BOUNDED RATIONALITY, AMBIGUITY, AND THE ENGINEERING OF CHOICE
March, J. G. (1988) · Decision Making · doi:10.1017/cbo9780511598951.004 · Cited by 1,688
Dusenbury, L. (2003) · Health Education Research · doi:10.1093/her/18.2.237 · Cited by 1,671
Dane, A. V., & Schneider, B. H. (1998) · Clinical Psychology Review · doi:10.1016/s0272-7358(97)00043-3 · Cited by 1,557 · cited 13×
“Constructs—sometimes referred to as components (Dane & Schneider, 1998; Dumas et al., 2001), criteria (Mowbray et al., 2003), or factors (Becker, Smith, Tanzman, Drake, & Tremblay, 2001) in fidelity studies—serve as the building blocks of theory (Turner, 1989) and help researchers to identify…”
BackgroundResultInfluentialBounded rationality, ambiguity and the engineering of choice
March, J. G. (1978) · The Bell Journal of Economics · doi:10.2307/3003600 · Cited by 1,539
The Meaning of Educational Change
(2001) · The New Meaning of Educational Change · doi:10.4324/9780203986561-9 · Cited by 1,492
Dorland??s Illustrated Medical Dictionary
AREY, L. B., BURROWS, W., GREENHILL, J. P., & HEWITT, R. M. (1957) · the american journal of the medical sciences · doi:10.1097/00000441-195712000-00043 · Cited by 1,347
Examining Key Concepts in Research on Teachers’ Use of Mathematics Curricula
Remillard, J. T. (2005) · Review of Educational Research · doi:10.3102/00346543075002211 · Cited by 1,329
Ball, D. L., & Cohen, D. K. (1996) · Educational Researcher · doi:10.2307/1177151 · Cited by 1,167
Handbook of Research on Curriculum, The
(2010) · Encyclopedia of Curriculum Studies · doi:10.4135/9781412958806.n229 · Cited by 1,127
Evaluation : methods for studying programs and policies
Weiss, C. H. (1997) · Cited by 924
Fidelity Criteria: Development, Measurement, and Validation
MOWBRAY, C. (2003) · The American Journal of Evaluation · doi:10.1016/s1098-2140(03)00057-2 · Cited by 884
Treatment fidelity in outcome studies
Moncher, F. J., & Prinz, R. J. (1991) · Clinical Psychology Review · doi:10.1016/0272-7358(91)90103-2 · Cited by 797 · cited 2×
“For example, fidelity of implementation is a means of confirming “that the manipulation of the independent variable occurred as planned” (Moncher & Prinz, 1991, p. 247).”
BackgroundResearch on Curriculum and Instruction Implementation
Fullan, M., & Pomfret, A. (1977) · Review of Educational Research · doi:10.2307/1170134 · Cited by 732
Hord, S. M. (1987) · Cited by 723
Ennett, S. T., Tobler, N. S., Ringwalt, C. L., & Flewelling, R. L. (1994) · American Journal of Public Health · doi:10.2105/ajph.84.9.1394 · Cited by 655
How Can Comprehensive School Reform Models Be Successfully Implemented?
Desimone, L. (2002) · Review of Educational Research · doi:10.3102/00346543072003433 · Cited by 528
Program fidelity in assertive community treatment: Development and use of a measure.
Teague, G. B., Bond, G. R., & Drake, R. E. (1998) · American Journal of Orthopsychiatry · doi:10.1037/h0080331 · Cited by 520
Diffusion of a School-Based Substance Abuse Prevention Program: Predictors of Program Implementation
Rohrbach, L. A., Graham, J. W., & Hansen, W. B. (1993) · Preventive Medicine · doi:10.1006/pmed.1993.1020 · Cited by 431
Domitrovich, C. E., & Greenberg, M. T. (2000) · Journal of Educational and Psychological Consultation · doi:10.1207/s1532768xjepc1102_04 · Cited by 428
Implementation of Educational Innovation
Berman, P., & McLaughlin, M. W. (1976) · The Educational Forum · doi:10.1080/00131727609336469 · Cited by 410
Yeaton, W. H., & Sechrest, L. (1981) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.49.2.156 · Cited by 404
Staff Development and School Change
McLaughlin, M. W., & Marsh, D. D. (1978) · Teachers College Record: The Voice of Scholarship in Education · doi:10.1177/016146817808000107 · Cited by 400
Measuring the fidelity of implementation of a mental health program model.
McGrew, J. H., Bond, G. R., Dietzen, L., & Salyers, M. (1994) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.62.4.670 · Cited by 380
Connell, D. B., Turner, R. R., & Mason, E. F. (1985) · Journal of School Health · doi:10.1111/j.1746-1561.1985.tb05656.x · Cited by 359
A Developmental Model for Determining Whether the Treatment Is Actually Implemented
Hall, G. E., & Loucks, S. F. (1977) · American Educational Research Journal · doi:10.2307/1162291 · Cited by 353
McHugo, G. J., Drake, R. E., Teague, G. B., & Xie, H. (1999) · Psychiatric Services · doi:10.1176/ps.50.6.818 · Cited by 324
Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005) · Behavior Therapy · doi:10.1016/s0005-7894(05)80049-8 · Cited by 320
A Developmental Conceptualization of the Adoption Process Within Educational Institutions.
Hall, G. E. (1973) · Cited by 301
The fidelity‐adaptation debate: Implications for the implementation of public sector social programs
Blakely, C. H., Mayer, J. P., Gottschalk, R. G., Schmitt, N., Davidson, W. S., Roitman, D. B., & Emshoff, J. G. (1987) · American Journal of Community Psychology · doi:10.1007/bf00922697 · Cited by 297 · cited 10×
“, 2003)—an issue often raised in the fidelity-adaptation debate (Blakely et al., 1987).”
BackgroundMethodInfluentialCommunity-Based Intervention Research: Coping With the “Noise” of Real Life in Study Design
Hohmann, A. A., & Shear, M. K. (2002) · American Journal of Psychiatry · doi:10.1176/appi.ajp.159.2.201 · Cited by 276
Promoting intervention fidelity
Dumas, J. E., Lynch, A. M., Laughlin, J. E., Phillips Smith, E., & Prinz, R. J. (2001) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(00)00272-5 · Cited by 274
The teacher-curriculum encounter: freeing teachers from the tyranny of texts
(1991) · Choice Reviews Online · doi:10.5860/choice.29-0429 · Cited by 273
Harachi, T. W., Abbott, R. D., Catalano, R. F., Haggerty, K. P., & Fleming, C. B. (1999) · American Journal of Community Psychology · doi:10.1023/a:1022194005511 · Cited by 269
Avoiding type III error in program evaluation
Dobson, D., & Cook, T. J. (1980) · Evaluation and Program Planning · doi:10.1016/0149-7189(80)90042-7 · Cited by 266
Project DARE: No effects at 10-year follow-up.
Lynam, D. R., Milich, R., Zimmerman, R., Novak, S. P., Logan, T. K., Martin, C., Leukefeld, C., & Clayton, R. (1999) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.67.4.590 · Cited by 266
Economic Impacts of Assertive Community Treatment: A Review of the Literature
Latimer, E. A. (1999) · The Canadian Journal of Psychiatry · doi:10.1177/070674379904400504 · Cited by 251
Treatment Integrity of School-Based Behavioral Intervention Studies: 1980–1990
Gresham, F. M., Gansle, K. A., Noell, G. H., Cohen, S., & Rosenblum, S. (1993) · School Psychology Review · doi:10.1080/02796015.1993.12085651 · Cited by 241
Aron, D. C. (1997) · Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.82.6.1780 · Cited by 226
A model for the institutionalization of health promotion programs
Goodman, R. M., & Steckler, A. (1989) · Family & Community Health · doi:10.1097/00003727-198902000-00009 · Cited by 224
Enacting reform‐based science materials: The range of teacher enactments in reform classrooms
Schneider, R. M., Krajcik, J., & Blumenfeld, P. (2005) · Journal of Research in Science Teaching · doi:10.1002/tea.20055 · Cited by 222
Resnick, B., Bellg, A. J., Borrelli, B., De Francesco, C., Breger, R., Hecht, J., Sharp, D. L., Levesque, C., Orwig, D., Ernst, D., et al. (2005) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm2902s_8 · Cited by 215
Evaluating use of continuous treatment teams for persons with mental illness and substance abuse
(1995) · Psychiatric Services · doi:10.1176/ps.46.7.689 · Cited by 211
Dorland's Illustrated Medical Dictionary
Dirckx, J. H. (1995) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1995.03520340077042 · Cited by 210
Reinventing fidelity: The transfer of social technology among settings
Bauman, L. J., Stein, R. E. K., & Ireys, H. T. (1991) · American Journal of Community Psychology · doi:10.1007/bf00937995 · Cited by 208
On evaluating curricular effectiveness judging the quality of K-12 mathematics evaluations
Confrey, J., Stohl, V., & National Research Council (U.S.) (2004) · Cited by 206
US Department of Education; Institute of Education Sciences; National Center for Education Evaluation and Regional Assistance (2003) · PsycEXTRA Dataset · doi:10.1037/e370412004-001 · Cited by 202
Enhancing Kindergartners' Mathematical Development: Effects of Peer-Assisted Learning Strategies
Fuchs, L. S., Fuchs, D., & Karns, K. (2001) · The Elementary School Journal · doi:10.1086/499684 · Cited by 186
Hansen, W. B., Graham, J. W., Wolkenstein, B. H., & Rohrbach, L. A. (1991) · Journal of Studies on Alcohol · doi:10.15288/jsa.1991.52.568 · Cited by 179
Fidelity of Supported Employment Programs and Employment Outcomes
Becker, D. R., Smith, J., Tanzman, B., Drake, R. E., & Tremblay, T. (2001) · Psychiatric Services · doi:10.1176/appi.ps.52.6.834 · Cited by 178
Vaughn, S. R., Cirino, P. T., Linan‐Thompson, S., Mathes, P. G., Carlson, C. D., Cárdenas-Hagan, E., Pollard‐Durodola, S. D., Fletcher, J. M., & Francis, D. J. (2006) · American Educational Research Journal · doi:10.3102/00028312043003449 · Cited by 175
Metaphor in the study of teachers’ professional knowledge
Munby, H., & Russell, T. (1990) · Theory Into Practice · doi:10.1080/00405849009543441 · Cited by 173
Managing and Guiding School Reform: Leadership in Success for All Schools
Datnow, A., & Castellano, M. E. (2001) · Educational Administration Quarterly · doi:10.1177/00131610121969307 · Cited by 171
Scaling-Up Exemplary Interventions
McDonald, S.-K., Keesler, V. A., Kauffman, N. J., & Schneider, B. (2006) · Educational Researcher · doi:10.3102/0013189x035003015 · Cited by 161
Advancing Scientific Research in Education
National Research Council (2004) · National Academies Press eBooks · doi:10.17226/11112 · Cited by 155
The use of treatment manuals in cognitive therapy: Experience and isssues.
Dobson, K. S., & Shaw, B. F. (1988) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.56.5.673 · Cited by 132
Evaluating Program Implementation
Leithwood, K. A., & Montgomery, D. J. (1980) · Evaluation Review · doi:10.1177/0193841x8000400202 · Cited by 131
Sensitivity, bias, and theory in impact evaluations.
Boruch, R. F., & Gomez, H. (1977) · Professional Psychology · doi:10.1037/0735-7028.8.4.411 · Cited by 110
A Three-Year Evaluation of the Know Your Body Program in Inner-City Schoolchildren
Resnicow, K., Cohn, L., Reinhardt, J., CROSS, D. S., Futterman, R., Kirschner, E., Wynder, E. L., & Allegrante, J. P. (1992) · Health Education Quarterly · doi:10.1177/109019819201900410 · Cited by 110
Utilization-Focused Evaluation
Heilman, J. G., & Patton, M. Q. (1979) · The Western Political Quarterly · doi:10.2307/447919 · Cited by 98
Measuring the Degree of Program Implementation: A Methodological Review
Scheirer, M. A., & Rezmovic, E. L. (1983) · Evaluation Review · doi:10.1177/0193841x8300700502 · Cited by 97
Penuel, W. R., & Means, B. (2004) · Journal of Research in Science Teaching · doi:10.1002/tea.20002 · Cited by 87
Allinder, R. M., Bolling, R. M., Oats, R. G., & Gagnon, W. A. (2000) · Remedial and Special Education · doi:10.1177/074193250002100403 · Cited by 83
Meeting the Curricular Needs of Academically Low-Achieving Students in Middle Grade Mathematics
Woodward, J. P., & Brown, C. (2006) · The Journal of Special Education · doi:10.1177/00224669060400030301 · Cited by 83
The Concept of Curriculum Potential
Ben‐Peretz, M. (1975) · Curriculum Theory Network · doi:10.2307/1179278 · Cited by 80
School-based smoking prevention: The teacher training process
Tortu, S., & Botvin, G. J. (1989) · Preventive Medicine · doi:10.1016/0091-7435(89)90075-3 · Cited by 79
A tool for analyzing implementation fidelity of an integrated learning system
Mills, S. C., & Ragan, T. J. (2000) · Educational Technology Research and Development · doi:10.1007/bf02300498 · Cited by 74
Ysseldyke, J., Spicuzza, R., Kosciolek, S., Teelucksingh, E., Boys, C., & Lemkuil, A. (2003) · Journal of Education for Students Placed at Risk (JESPAR) · doi:10.1207/s15327671espr0802_4 · Cited by 70 · cited 6×
“Relationships ranged from causal (Ysseldyke et al., 2003), to associational (Hall & Loucks, 1977), to predictive (Songer & Gotwals, 2005).”
BackgroundMethodInfluentialImplementing Comprehensive Health Education: Educational Innovations and Social Change
Kolbe, L. J., & Iverson, D. C. (1981) · Health Education Quarterly · doi:10.1177/109019818100800109 · Cited by 67
Assessing program fidelity in substance abuse health services research
Orwin, R. G. (2000) · Addiction · doi:10.1080/09652140020004250 · Cited by 66
A Process Evaluation of the District of Columbia “Know Your Body” Project
TAGGART, V. S., Bush, P. J., Zuckerman, A. E., & Theiss, P. K. (1990) · Journal of School Health · doi:10.1111/j.1746-1561.1990.tb05907.x · Cited by 65
Addressing community problems : psychological research and interventions
Arriaga, X. B., & Oskamp, S. (1998) · Cited by 61
Introduction: Scaling Up School Restructuring Designs in Urban Schools
Stringfield, S., & Datnow, A. (1998) · Education and Urban Society · doi:10.1177/0013124598030003001 · Cited by 61
Comprehensive evaluation of a model alcohol education curriculum.
Hopkins, R. H., Mauss, A. L., Kearney, K. A., & Weisheit, R. A. (1988) · Journal of Studies on Alcohol · doi:10.15288/jsa.1988.49.38 · Cited by 55
Innovation Configurations: Analyzing the Adaptations of Innovations.
Hall, G. E., & Loucks, S. F. (1978) · Cited by 47
Program Strength and Fidelity in Evaluation
Summerfelt, W. T. (2003) · Applied Developmental Science · doi:10.1207/s1532480xads0702_2 · Cited by 47
Emshoff, J. G., Blakely, C. H., Gottschalk, R. G., Mayer, J., Davidson, W. S., & Erickson, S. K. (1987) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737009004300 · Cited by 46
Can the Curriculum Guide Both Emancipate and Educate Teachers?
Shkedi, A. (1998) · Curriculum Inquiry · doi:10.1111/0362-6784.00085 · Cited by 43
Sex Differences: Implications for Program Planning for the Academically Gifted
Fox, L. H. (1976) · Gifted Child Quarterly · doi:10.1177/001698627602000314 · Cited by 42
Increasing the Use of Science to Improve Child-Rearing
Biglan, A., & Taylor, T. Κ. (2000) · The Journal of Primary Prevention · doi:10.1023/a:1007083203280 · Cited by 42
Scaling Up School Restructuring Designs in Urban Schools.
Stringfield, S., & Datnow, A. (1998) · Education and Urban Society · Cited by 39
Rethinking Curriculum Implementation: Paradigms, Models, and Teachers' Work.
Cho, J. (1998) · Cited by 37
Efficacy vs. Effectiveness in Psychiatric Research
Summerfelt, W. T., & Meltzer, H. Y. (1998) · Psychiatric Services · doi:10.1176/ps.49.6.834 · Cited by 29
Fidelity of Implementation in Three Sequential Curricular Units
Songer, N. B., & Gotwals, A. W. (2005) · Cited by 24
Learning to Effectively Implement Constant Time Delay Procedures to Teach Spelling
Hughes, T. A., Fredrick, L. D., & Keel, M. C. (2002) · Learning Disability Quarterly · doi:10.2307/1511303 · Cited by 23
Ward, T., Bernier, R., Mukerji, C., et al. (2012) · Springer eBooks · doi:10.1007/978-1-4419-1698-3_100603 · Cited by 19
Cost-Effectiveness Analysis of an Educational Drug Abuse Prevention Program
Mitchel, M. E., Hu, T.-W., Donnell, N. M., & Swisher, J. D. (1984) · Journal of Drug Education · doi:10.2190/lypb-tmr5-p7h0-ngjp · Cited by 18
Improving tuberculosis therapy completion after jail: translation of research to practice
White, M. C., Tulsky, J. P., Menendez, E., Arai, S., Goldenson, J., & Kawamura, L. M. (2004) · Health Education Research · doi:10.1093/her/cyg109 · Cited by 13
O’Donnell, C. (2007) · Cited by 10
Self-citation
Cited by
(2014) · Medical Entomology and Zoology · Cited by 1,154
Challenges and Future Directions of Big Data and Artificial Intelligence in Education
Luan, H., Geczy, P., Lai, H., Gobert, J., Yang, S. J. H., Ogata, H., Baltes, J., Guerra, R., Li, P., & Tsai, C.-C. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.580820 · Cited by 667
“To bridge the gap, effective teacher education programs and continuing education programs have to be developed and offered to support the adoption of these new technologies so that they can be implemented with fidelity (O’Donnell, 2008).”
BackgroundFrom design experiments to formative interventions
Engeström, Y. (2011) · Theory & Psychology · doi:10.1177/0959354311419252 · Cited by 642
Motivation Interventions in Education: A Meta-Analytic Review
Lazowski, R. A., & Hulleman, C. S. (2016) · Review of Educational Research · doi:10.3102/0034654315617832 · Cited by 607
Improving Student Outcomes in Higher Education: The Science of Targeted Intervention
Harackiewicz, J. M., & Priniski, S. J. (2018) · Annual Review of Psychology · doi:10.1146/annurev-psych-122216-011725 · Cited by 429
Metadata
- Authors
- O’Donnell, C. L.
- Venue
- Review of Educational Research
- Published
- 2008-03
- Volume
- 78
- Issue
- 1
- Pages
- 33-84
- ISSN
- 0034-6543 (ISSN-L)
- 1935-1046
- OpenAlex
- W2114684950
- Cited by
- 1,032
- References
- 97
- FWCI
- 69.40
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.