A Mixed-Methods Approach for Embedding Cost Analysis Within Fidelity Assessment in School-Based Programs

Bradshaw, Catherine P.; Debnam, Katrina J.; Player, Daniel; Bowden, Brooks; Lindstrom Johnson, Sarah

doi:10.1177/0198742920944850

Cited by 17Behavioral and Psychological StudiesSchool Health and Nursing EducationEducation Discipline and Inequality

A Mixed-Methods Approach for Embedding Cost Analysis Within Fidelity Assessment in School-Based Programs

Bradshaw, Catherine P., Debnam, Katrina J., Player, Daniel, Bowden, Brooks, Lindstrom Johnson, Sarah·2023·Behavioral Disorders

DOI 10.1177/0198742920944850

Open this paper in CoChat — free account

Abstract

This mixed-methods study describes a framework for conducting cost analyses of school-based programs leveraging fidelity data and applying the ingredients method. We illustrate this approach by applying it to Positive Behavioral Interventions and Supports (PBIS), drawing on multiple sources of data from a sample of U.S. 77 schools that were trained in PBIS. We concluded that the average per school cost of PBIS was US$53,216.00 (median = US$36,698), with an average per-pupil cost of US$90.00 (median = US$58.00), which is considerably less than other school-based prevention models. The cost did, however, differ by implementation level, such that high-fidelity implementation tended to cost more than low-fidelity implementation. We provide a case illustration to elucidate some of the cost drivers of PBIS implementation. Specifically, these data highlight the variability in the amount of training and coaching by the specific evidence-based program implemented within the tiered PBIS framework. Through this case illustration, we demonstrate the utility of tracking costs of school-based program within the context of fidelity data collection. The findings also suggest the potential cost savings of PBIS, both when compared with other evidence-based interventions as well as the known costs of negative school outcomes like dropout.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1177/0198742920944850

Cites

40 of 40 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

null
null (2016) · Philosophy Study · doi:10.17265/2159-5313/2016.09.003 · Cited by 28,317 · see citing works
SAGE Handbook of Mixed Methods in Social & Behavioral Research
Tashakkori, A., & Teddlie, C. (2010) · doi:10.4135/9781506335193 · Cited by 6,215 · see citing works
What's wrong with Bonferroni adjustments
Perneger, T. V. (1998) · BMJ · doi:10.1136/bmj.316.7139.1236 · Cited by 5,900 · see citing works
Untitled work
(2005) · Cited by 3,748 · see citing works
A typology of mixed methods research designs
Leech, N. L., & Onwuegbuzie, A. J. (2007) · Quality & Quantity · doi:10.1007/s11135-007-9105-3 · Cited by 1,736 · see citing works
PROGRAM INTEGRITY IN PRIMARY AND EARLY SECONDARY PREVENTION: ARE IMPLEMENTATION EFFECTS OUT OF CONTROL?
Dane, A. V., & Schneider, B. H. (1998) · Clinical Psychology Review · doi:10.1016/s0272-7358(97)00043-3 · Cited by 1,557 · cited 2× · see citing works
“…there is a growing body of research demonstrating that higher fidelity of an evidence-based program (EBP) is associated with better outcomes (Dane & Schneider, 1998; Domitrovich et al., 2008), high fidelity might come at a higher cost because more elements are in place and more time is…”
Background
Examining the Effects of Schoolwide Positive Behavioral Interventions and Supports on Student Outcomes: Results From a Randomized Controlled Effectiveness Trial in Elementary Schools
Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010) · Journal of Positive Behavior Interventions · doi:10.1177/1098300709334798 · Cited by 868 · see citing works
Self-citation
A Randomized, Wait-List Controlled Effectiveness Trial Assessing School-Wide Positive Behavior Support in Elementary Schools
Horner, R. H., Sugai, G. M., Smolkowski, K., Eber, L., Nakasato, J., Todd, A. W., & Esperanza, J. (2009) · Journal of Positive Behavior Interventions · doi:10.1177/1098300709332067 · Cited by 826 · see citing works
The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry
(2015) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199933624.001.0001 · Cited by 786 · see citing works
Integrated Approaches to Preventing Antisocial Behavior Patterns among School-Age Children and Youth
Walker, H. M., Horner, R. H., Sugai, G. M., Bullis, M. D., Sprague, J. R., Bricker, D. D., & Kaufman, M. J. (1996) · Journal of Emotional and Behavioral Disorders · doi:10.1177/106342669600400401 · Cited by 780 · see citing works
Maximizing the Implementation Quality of Evidence-Based Preventive Interventions in Schools: A Conceptual Framework
Domitrovich, C. E., Bradshaw, C. P., Poduska, J. M., Hoagwood, K., Buckley, J. A., Olin, S., Romanelli, L. H., Leaf, P. J., Greenberg, M. T., & Ialongo, N. S. (2008) · Advances in School Mental Health Promotion · doi:10.1080/1754730x.2008.9715730 · Cited by 749 · see citing works
Self-citation
Handbook of Evaluation Research
Ross, L., & Cronbach, L. J. (1976) · Educational Researcher · doi:10.3102/0013189x005010009 · Cited by 699 · see citing works
Examining the Evidence Base for School-Wide Positive Behavior Support
Horner, R. H., Sugai, G. M., & Anderson, C. M. (2010) · Focus on Exceptional Children · doi:10.17161/foec.v42i8.6906 · Cited by 582 · see citing works
The School-Wide Evaluation Tool (SET)
Horner, R. H., Todd, A. W., Lewis-Palmer, T., Irvin, L. K., Sugai, G. M., & Boland, J. B. (2004) · Journal of Positive Behavior Interventions · doi:10.1177/10983007040060010201 · Cited by 563 · see citing works
The Study of Implementation: Current Findings From Effective Programs that Prevent Mental Disorders in School-Aged Children
Domitrovich, C. E., & Greenberg, M. T. (2000) · Journal of Educational and Psychological Consultation · doi:10.1207/s1532768xjepc1102_04 · Cited by 428 · see citing works
Preventing youth violence and delinquency through a universal school-based prevention approach
Botvin, G. J., Griffin, K. W., & Nichols, T. D. (2006) · Prevention Science · doi:10.1007/s11121-006-0057-y · Cited by 322 · see citing works
The economic value of social and emotional learning
Belfield, C., Bowden, A. B., Klapp, A., Levin, H., Shand, R., & Zander, S. (2015) · Journal of Benefit-Cost Analysis · doi:10.1017/bca.2015.55 · Cited by 307 · see citing works
Scaling up Evidence-Based Interventions in US Public Systems to Prevent Behavioral Health Problems: Challenges and Opportunities
Fagan, A. A., Bumbarger, B. K., Barth, R. P., Bradshaw, C. P., Cooper, B. R., Supplee, L. H., & Walker, D. K. (2019) · Prevention Science · doi:10.1007/s11121-019-01048-8 · Cited by 216 · see citing works
Self-citation
The Long-Term Effects and Cost-Effectiveness of Success for All
Borman, G. D., & Hewes, G. M. (2002) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737024004243 · Cited by 182 · see citing works
Economic Evaluation in Education: Cost-Effectiveness and Benefit-Cost Analysis
Levin, H. M., McEwan, P. J., Belfield, C., Bowden, A. B., & Shand, R. (2018) · doi:10.4135/9781483396514 · Cited by 181 · see citing works
Principles of sustainable prevention: Designing scale‐up of School‐wide Positive Behavior Support to promote durable systems
McIntosh, K., Filter, K. J., Bennett, J. L., Ryan, C., & Sugai, G. M. (2009) · Psychology in the Schools · doi:10.1002/pits.20448 · Cited by 180 · see citing works
Introduction
Onwuegbuzie, A. J. (2012) · International Journal of Multiple Research Approaches · doi:10.5172/mra.2012.6.3.192 · Cited by 131 · see citing works
Mixed-method approaches to strengthen economic evaluations in implementation research
Dopp, A. R., Mundey, P., Beasley, L. O., Silovsky, J. F., & Eisenberg, D. (2019) · Implementation Science · doi:10.1186/s13012-018-0850-6 · Cited by 118 · see citing works
Guiding the Development and Use of Cost-Effectiveness Analysis in Education
Levin, H. M., & Belfield, C. (2014) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2014.915604 · Cited by 114 · see citing works
Sustainability of Systems-Level Evidence-Based Practices in Schools: Current Knowledge and Future Directions
McIntosh, K., Horner, R. H., & Sugai, G. M. (2008) · Issues in clinical child psychology · doi:10.1007/978-0-387-09632-2_14 · Cited by 102 · see citing works
Standards of Evidence for Conducting and Reporting Economic Evaluations in Prevention Science
Crowley, D. M., Dodge, K. A., Barnett, W. S., Corso, P. S., Duffy, S. Q., Graham, P., Greenberg, M. H., Haskins, R., Hill, L. G., Jones, D. E., et al. (2018) · Prevention Science · doi:10.1007/s11121-017-0858-1 · Cited by 92 · see citing works
A state-wide quasi-experimental effectiveness study of the scale-up of school-wide Positive Behavioral Interventions and Supports
Pas, E. T., Ryoo, J. H., Musci, R. J., & Bradshaw, C. P. (2019) · Journal of School Psychology · doi:10.1016/j.jsp.2019.03.001 · Cited by 76 · see citing works
Self-citation
Effective Behavior Support: A Systems Approach to Proactive Schoolwide Management
Sarungallo, F. E., Marini, L. F., & Baisa, L. Y. (2017) · Focus on Exceptional Children · doi:10.17161/fec.v31i6.6767 · Cited by 72 · see citing works
Understanding Real-World Implementation Quality and “Active Ingredients” of PBIS
Molloy, L. E., Moore, J. E., Trail, J. B., Epps, J. J. V., & Hopfer, S. (2013) · Prevention Science · doi:10.1007/s11121-012-0343-9 · Cited by 69 · see citing works
Preliminary Validation of the Implementation Phases Inventory for Assessing Fidelity of Schoolwide Positive Behavior Supports
Bradshaw, C. P., Debnam, K. J., Koth, C. W., & Leaf, P. J. (2008) · Journal of Positive Behavior Interventions · doi:10.1177/1098300708319126 · Cited by 56 · see citing works
Self-citation
Progress and Priorities in Research to Improve Outcomes for Students With or at Risk for Emotional and Behavioral Disorders
Lloyd, B. P., Bruhn, A. L., Sutherland, K. S., & Bradshaw, C. P. (2018) · Behavioral Disorders · doi:10.1177/0198742918808485 · Cited by 56 · see citing works
Self-citation
Application of economic analysis to school-wide positive behavior support (SWPBS) programs
Blonigen, B. A., Harbaugh, W. T., Singell, L. D., Horner, R. H., Irvin, L. K., & Smolkowski, K. S. (2008) · Journal of Positive Behavior Interventions · doi:10.1177/1098300707311366 · Cited by 50 · see citing works
Secondary and Tertiary Support Systems in Schools Implementing School-Wide Positive Behavioral Interventions and Supports
Debnam, K. J., Pas, E. T., & Bradshaw, C. P. (2012) · Journal of Positive Behavior Interventions · doi:10.1177/1098300712436844 · Cited by 36 · see citing works
Self-citation
Examining the Relative Utility of PBIS Implementation Fidelity Scores in Relation to Student Outcomes
Pas, E. T., Johnson, S. R., Debnam, K. J., Hulleman, C. S., & Bradshaw, C. P. (2019) · Remedial and Special Education · doi:10.1177/0741932518805192 · Cited by 34 · see citing works
Self-citation
Tracking Time and Resources Associated with Systems Change and the Adoption of Evidence-Based Programs: The “Hidden Costs” of School-Based Coaching
Pas, E. T., Johnson, S. L., Alfonso, Y. N., & Bradshaw, C. P. (2020) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-020-01039-w · Cited by 29 · see citing works
Self-citation
Cost-Effectiveness and Educational Policy. Yearbook of the American Education Finance Association, 2002.
Levin, H. M., & McEwan, P. J. (2002) · Cited by 20 · see citing works
Evaluating Educational Interventions That Induce Service Receipt
Bowden, A. B., Shand, R., Belfield, C., Wang, A., & Levin, H. M. (2016) · American Journal of Evaluation · doi:10.1177/1098214016664983 · Cited by 17 · see citing works
Scaling up Positive Behavioral Interventions and Supports: Costs and their distribution across state, districts, and schools
Lindstrom Johnson, S., Alfonso, Y. N., Pas, E. T., Debnam, K. J., & Bradshaw, C. P. (2020) · School Psychology Review · doi:10.1080/2372966x.2020.1777831 · Cited by 17 · see citing works
Self-citation
Using Cost-Effectiveness Analysis in Mixed Methods Research: An Evaluation of an Integrated Care Program for Frequently Hospitalized Older Adults in Singapore
Penkunas, M. J., Matchar, D. B., Wong, C. H., Liu, C., & Chan, A. W.-M. (2019) · Journal of Mixed Methods Research · doi:10.1177/1558689819844838 · Cited by 9 · see citing works
Enhancing Capacity for Evidence-Based Policymaking: the Role of Economic Evaluation Standards
Webb, M. B. (2018) · Prevention Science · doi:10.1007/s11121-018-0872-y · Cited by 4 · see citing works

Flagged first, then by citations

Sorted by citations

Research recommendations for assessing potential harm from universal school-based mental health interventions
Foulkes, L., Andrews, J. L., Reardon, T. C., & Stringaris, A. K. (2024) · Nature Mental Health · doi:10.1038/s44220-024-00208-2 · Cited by 44 · see citing works
Implementation Science in School Mental Health: A 10-Year Progress Update and Development of a New Research Agenda
Lyon, A. R., Connors, E. H., Lawson, G. M., Nadeem, E., & Owens, J. S. (2024) · School Mental Health · doi:10.1007/s12310-024-09731-0 · Cited by 47 · see citing works
Tracking Time and Resources Associated with Systems Change and the Adoption of Evidence-Based Programs: The “Hidden Costs” of School-Based Coaching
Pas, E. T., Johnson, S. L., Alfonso, Y. N., & Bradshaw, C. P. (2020) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-020-01039-w · Cited by 29 · see citing works
Self-citation
An Economic Evaluation of Emerging and Ineffective Interventions: Examining the Role of Cost When Translating Research into Practice
Scheibel, G., Zane, T. L., & Zimmerman, K. N. (2022) · Exceptional Children · doi:10.1177/00144029211073522 · Cited by 24 · see citing works
Restorative Practices: Using local evidence on costs and student outcomes to inform school district decisions about behavioral interventions
Hollands, F. M., Leach, S. M., Shand, R., Head, L., Wang, Y., Dossett, D., Chang, F., Yan, B., Martin, M., Pan, Y., et al. (2022) · Journal of School Psychology · doi:10.1016/j.jsp.2022.03.007 · Cited by 19 · see citing works
Scaling up Positive Behavioral Interventions and Supports: Costs and their distribution across state, districts, and schools
Lindstrom Johnson, S., Alfonso, Y. N., Pas, E. T., Debnam, K. J., & Bradshaw, C. P. (2020) · School Psychology Review · doi:10.1080/2372966x.2020.1777831 · Cited by 17 · see citing works
Self-citation
Translating sophisticated data analytic strategies regarding school attendance and absenteeism into targeted educational policy
Kearney, C. A., & Childs, J. (2023) · Improving Schools · doi:10.1177/13654802231174986 · Cited by 17 · see citing works
Scaling Up Behavioral Health Promotion Efforts in Maryland: The Economic Benefit of Positive Behavioral Interventions and Supports
Bradshaw, C. P., Johnson, S. L., Zhu, Y., & Pas, E. T. (2020) · School Psychology Review · doi:10.1080/2372966x.2020.1823797 · Cited by 19 · see citing works
Self-citation
Study protocol for a cluster-randomized trial of a bundle of implementation support strategies to improve the fidelity of implementation of schoolwide Positive Behavioral Interventions and Supports in rural schools
Turner, L. R., Calvert, H. G., Fleming, C. M., Lewis, T., Siebert, C. F., Anderson, N., Castleton, T., Havlicak, A., & McQuilkin, M. (2022) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2022.100949 · Cited by 17 · see citing works
Implementation Preparation Costs of Virtual Interview Training in Pre-Employment Transition Services: A Budget Impact Analysis
Danielson, E. C., Smith, M. J., Ross, B., Sherwood, K. L., Smith, J. D., Atkins, M. S., & Jordan, N. (2023) · Journal of Special Education Technology · doi:10.1177/01626434231175372 · Cited by 9 · see citing works
The Need for Technology-Aided Instruction and Intervention Policy for Autistic Youth
Sherwood, K. L., Smith, M. J., & Eldredge, M. A. (2023) · Journal of Disability Policy Studies · doi:10.1177/10442073221150603 · Cited by 6 · see citing works
Estimating the Cost of School Mental Health Programming to Increase Adoption and Scale-up of Evidence-Based Programs and Practices
Johnson, S. L., Bowden, A. B., & Bradshaw, C. P. (2023) · Issues in clinical child psychology · doi:10.1007/978-3-031-20006-9_36 · Cited by 5 · see citing works
Self-citation
Designing Field Experiments to Integrate Research on Costs
Bowden, A. B. (2023) · AERA Open · doi:10.1177/23328584231171536 · Cited by 6 · see citing works
Telling Better Stories: An Introduction to the Special Issue Using Mixed Methods Research in the Field of Behavioral Disorders
Anderson, J. A. (2023) · Behavioral Disorders · doi:10.1177/01987429231159015 · Cited by 3 · see citing works
Improving the implementation of KiVa antibullying program with tailored support: Study protocol for a cluster randomized controlled trial
Herkama, S., Larose, M., Harjuniemi, I., Pöyhönen, V., Yanagida, T., Kankaanpää, E., Rissanen, E., & Salmivalli, C. (2023) · Contemporary Clinical Trials · doi:10.1016/j.cct.2023.107407 · Cited by 2 · see citing works
Prevention Frameworks: The Path from Public Health to Multi-Tiered System of Support
Simonsen, B. M., Coyne, M. D., & Goodman, S. (2025) · Issues in clinical child psychology · doi:10.1007/978-3-031-96102-1_3 · Cited by 1 · see citing works
Measuring the Impact of PBIS Through Multiple Metrics: Mapping Outcomes to Cost Savings
Pérez, P., Beahm, L. A., Pandey, T., Johnson, S. L., & Bradshaw, C. P. (2025) · Issues in clinical child psychology · doi:10.1007/978-3-031-96102-1_24
Self-citation

Scaling up Positive Behavioral Interventions and Supports: Costs and their distribution across state, districts, and schools
Lindstrom Johnson, S., Alfonso, Y. N., Pas, E. T., Debnam, K. J., & Bradshaw, C. P. (2020) · School Psychology Review · doi:10.1080/2372966x.2020.1777831 · Cited by 17 · see citing works
Self-citation
Application of economic analysis to school-wide positive behavior support (SWPBS) programs
Blonigen, B. A., Harbaugh, W. T., Singell, L. D., Horner, R. H., Irvin, L. K., & Smolkowski, K. S. (2008) · Journal of Positive Behavior Interventions · doi:10.1177/1098300707311366 · Cited by 50 · see citing works
A mixed methods partner-focused cost and budget impact analysis to deploy implementation strategies for school-based prevention
Eisman, A. B., Whitman, J., Palinkas, L. A., Fridline, J., Harvey, C., Kilbourne, A. M., & Hutton, D. W. (2023) · Implementation Science Communications · doi:10.1186/s43058-023-00511-6 · Cited by 9 · see citing works
Comprehensive Cost Analysis of First Step Next for Preschoolers with Disruptive Behavior Disorder: Using Real-World Intervention Data to Estimate Costs at Scale
Frey, A. J., Kuklinski, M. R., Bills, K., Small, J. W., Forness, S. R., Walker, H. M., Feil, E. G., & Seeley, J. R. (2019) · Prevention Science · doi:10.1007/s11121-019-01035-z · Cited by 15 · see citing works
Using Staff and Student Time Engaged in Disciplinary Procedures to Evaluate the Impact of School-Wide PBS
Scott, T. M., & Barrett, S. B. (2004) · Journal of Positive Behavior Interventions · doi:10.1177/10983007040060010401 · Cited by 202 · see citing works

Authors: Bradshaw, C. P., Debnam, K. J., Player, D., Bowden, B., & Lindstrom Johnson, S.
Affiliations: University of Virginia (US)
University of Pennsylvania (US)
Arizona State University, Tempe, USA
Corresponding author: Bradshaw, Catherine P.
ORCID: Bradshaw, Catherine P.
Debnam, Katrina J.
Player, Daniel
Venue: Behavioral Disorders
Publisher: SAGE Publications
Published: 2020-08-11
Volume: 48
Issue: 3
Pages: 174-186
ISSN: 0198-7429 (ISSN-L)
2163-5307
DOI: 10.1177/0198742920944850
OpenAlex: W3048800194
Cited by: 17
References: 40
Citations by year: 2025: 2 · 2024: 2 · 2023: 7 · 2022: 3 · 2020: 3
FWCI: 4.91
Type: journal article
Language: en
Open access: closed
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Behavioral Disorders (publishedVersion)
Funders: Institute of Education Sciences (R305H150027)
National Institute of Justice (2014-CK-BX-0005)
Topics: Social Sciences › Psychology › Developmental and Educational Psychology › Behavioral and Psychological Studies
Health Sciences › Health Professions › Speech and Hearing › School Health and Nursing Education
Social Sciences › Education › Education Discipline and Inequality
Keywords: cost analysis, implementation fidelity, school programs, Positive Behavioral Interventions and Supports, cost per student, cost drivers, school dropout, evidence-based interventions, school prevention programs

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A Mixed-Methods Approach for Embedding Cost Analysis Within Fidelity Assessment in School-Based Programs

Abstract

Integrity signals

Claims the paper makes

Self-check

Cites

Cited by

Related

Record

About these checks