Cost analysis in implementation studies of evidence-based practices for mental health and substance use disorders: a systematic review
Open accessThe DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Abstract
Background: This study is a systematic literature review of cost analyses conducted within implementation studies on behavioral health services. Cost analysis of implementing evidence-based practices (EBP) has become important within implementation science and is critical for bridging the research to practice gap to improve access to quality healthcare services. Costing studies in this area are rare but necessary since cost can be a barrier to implementation and sustainment of EBP. Methods: We followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology and applied the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist. Key search terms included: (1) economics, (2) implementation, (3) EBP, and (4) behavioral health. Terms were searched within article title and abstracts in: EconLit, SocINDEX, Medline, and PsychINFO. A total of 464 abstracts were screened independently by two authors and reduced to 37 articles using inclusion and exclusion criteria. After a full-text review, 18 articles were included. Results: Findings were used to classify costs into direct implementation, direct services, and indirect implementation. While all studies included phases of implementation as part of their design, only five studies examined resources across multiple phases of an implementation framework. Most studies reported direct service costs associated with adopting a new practice, usually summarized as total EBP cost, cost per client, cost per clinician, and/or cost per agency. For studies with detailed analysis, there were eleven direct cost categories represented. For five studies that reported costs per child served, direct implementation costs varied from $886 to $9470 per child, while indirect implementation costs ranged from $897 to $3805 per child. Conclusions: This is the first systematic literature review to examine costs of implementing EBP in behavioral healthcare settings. Since 2000, 18 studies were identified that included a cost analysis. Given a wide variation in the study designs and economic methods, comparison across studies was challenging, which is a major limitation in the field, as it becomes difficult to replicate studies or to estimate future costs to inform policy decisions related to budgeting. We recommend future economic implementation studies to consider standard economic costing methods capturing costs across implementation framework phases to support comparisons and replicability.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“There has been a recent surge of publications related to the collection of costs in implementation science [9, 14, 31, 36], and emerging consensus regarding the importance of economic influences on implementation outcomes (see other articles in this collection).”
From Implementation science issues in understanding, collecting, and using cost estimates: A multi-stakeholder perspective (2021) · cites this paper as [9]
In the citing paper
“Implementation costs and effects accrue at different levels and are variable across different parties.”
From Implementation science issues in understanding, collecting, and using cost estimates: A multi-stakeholder perspective (2021) · cites this paper as [9]
In the citing paper
“Stakeholder engagement across levels is especially critical when organizations face overwhelming competing demands and scarce resources and/or stakeholders experience disproportionate costs related to implementation, such as opportunity costs for frontline staff engaging in implementation efforts in lieu of billable patient care [9].”
From Implementation science issues in understanding, collecting, and using cost estimates: A multi-stakeholder perspective (2021) · cites this paper as [9]
In the citing paper
“This is consistent with conclusions from a recent systematic review whereby researchers recommended standardized guidelines around costing perspectives, instrumentation, and reporting as critical next steps to advance the field [9].”
From Implementation science issues in understanding, collecting, and using cost estimates: A multi-stakeholder perspective (2021) · cites this paper as [9]
In the citing paper
“Tools to support standardization of multi-level costing approaches and reporting will also aid in building the business case for implementation science; this will facilitate harmonizing data across implementation studies and provide needed empirical evidence for the costs and benefits of implementation efforts for all stakeholders [9].”
From Implementation science issues in understanding, collecting, and using cost estimates: A multi-stakeholder perspective (2021) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223
Wang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,657
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496 · cited 1×
“The Consolidated Framework for Implementation Research (CFIR) provides a point of reference as it includes a comprehensive list of constructs (intervention characteristics, agency/staff characteristics, outer and inner contexts within which the agency operates, and the implementation process itself) [37].”
MethodMethods for the Economic Evaluation of Health Care Programmes
Drummond, M. E., Sculpher, M. J., Torrance, G. W., O’Brien, B. J., & Stoddart, G. L. (2005) · doi:10.1093/oso/9780198529446.001.0001 · Cited by 13,648
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0319-7 · Cited by 8,721 · cited 1×
“specifically called for implementation research to include costs as an outcome [4].”
BackgroundEffectiveness and efficiency of guideline dissemination and implementation strategies
Grimshaw, J., Thomas, R., MacLennan, G., Fraser, C., Ramsay, C., Vale, L., Whitty, P., Eccles, M., Matowe, L., Shirran, L., et al. (2004) · Health Technology Assessment · doi:10.3310/hta8060 · Cited by 3,044
Siddaway, A. P., Wood, A. M., & Hedges, L. V. (2019) · Annual Review of Psychology · doi:10.1146/annurev-psych-010418-102803 · Cited by 2,550 · cited 1×
“We based our search strategy on suggestions from psychology and economics published best practices for systematic literature reviews [14, 15].”
MethodAn introduction to implementation science for the non-specialist
Bauer, M. S., Damschroder, L., Hagedorn, H., Smith, J., & Kilbourne, A. M. (2015) · BMC Psychology · doi:10.1186/s40359-015-0089-9 · Cited by 1,611
Cost-Effectiveness in Health and Medicine
(2016) · doi:10.1093/acprof:oso/9780190492939.001.0001 · Cited by 1,552
Effectiveness and efficiency of guideline dissemination and implementation strategies
Grimshaw, J. M., Thomas, R. E., MacLennan, G., Fraser, C., Ramsay, C. R., Vale, L., Whitty, P., Eccles, M. P., Matowe, L., Shirran, L., et al. (2005) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462305290190 · Cited by 864
Knowledge translation in health: how implementation science could contribute more
Wensing, M., & Grol, R. (2019) · BMC Medicine · doi:10.1186/s12916-019-1322-9 · Cited by 413
Five types of OECD healthcare systems: Empirical results of a deductive classification
Böhm, K., Schmid, A., Götze, R., Landwehr, C., & Rothgang, H. (2013) · Health Policy · doi:10.1016/j.healthpol.2013.09.003 · Cited by 396
Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement
Husereau, D., Drummond, M., Petrou, S., Carswell, C., Moher, D., Greenberg, D., Augustovski, F., Briggs, A. H., Mauskopf, J., Loder, E., et al. (2013) · Cost Effectiveness and Resource Allocation · doi:10.1186/1478-7547-11-6 · Cited by 381
The Sustainability of Evidence-Based Practices in Routine Mental Health Agencies
Swain, K., Whitley, R., McHugo, G. J., & Drake, R. E. (2010) · Community Mental Health Journal · doi:10.1007/s10597-009-9202-y · Cited by 193
Saldana, L. (2014) · Implementation Science · doi:10.1186/1748-5908-9-43 · Cited by 188 · cited 4×
“Several studies focused on specific demographic groups [children with substance use disorders, mental health disorders, or trauma (six articles [19, 23–27]), adolescents/youth with mental health disorders (two articles [21, 28]), justice-involved youth (two articles [29, 30]), geriatric mental health (one article [20]…”
BackgroundInfluentialEvaluation of query expansion using MeSH in PubMed
Lu, Z., Kim, W., & Wilbur, W. J. (2009) · Information Retrieval · doi:10.1007/s10791-008-9074-8 · Cited by 177
Roberts, S. L. E., Healey, A., & Sevdalis, N. (2019) · Implementation Science · doi:10.1186/s13012-019-0901-7 · Cited by 161
What Is Implementation Research?: Rationale, Concepts, and Practices
Bhattacharyya, O., Reeves, S., & Zwarenstein, M. (2009) · Research on Social Work Practice · doi:10.1177/1049731509335528 · Cited by 159
The Economic Analysis of Prevention in Mental Health Programs
Mihalopoulos, C., Vos, T., Pirkis, J., & Carter, R. (2011) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-032210-104601 · Cited by 159
Thielen, F., Van Mastrigt, G., Burgers, L., Bramer, W., Majoie, H., Evers, S., & Kleijnen, J. (2016) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1080/14737167.2016.1246962 · Cited by 132
Stewart, R. E., Adams, D. R., Mandell, D. S., Hadley, T. R., Evans, A. C., Rubin, R., Erney, J., Neimark, G., Hurford, M. O., & Beidas, R. S. (2016) · Psychiatric Services · doi:10.1176/appi.ps.201500392 · Cited by 86
Economic evaluations of interventions designed to prevent mental disorders: a systematic review
Mihalopoulos, C., & Chatterton, M. L. (2015) · Early Intervention in Psychiatry · doi:10.1111/eip.12156 · Cited by 68
The Deepening Opioid Crisis in North America: Historical Context and Current Solutions
Donroe, J. H., Socias, M. E., & Marshall, B. D. L. (2018) · Current Addiction Reports · doi:10.1007/s40429-018-0228-5 · Cited by 66
Costs to Community Mental Health Agencies to Sustain an Evidence-Based Practice
Roundfield, K. D., & Lang, J. M. (2017) · Psychiatric Services · doi:10.1176/appi.ps.201600193 · Cited by 60
Telementoring Primary Care Clinicians to Improve Geriatric Mental Health Care
Fisher, E. M., Hasselberg, M. J., Conwell, Y., Weiss, L., Padrón, N. A., Tiernan, E., Karuza, J., Donath, J., & Pagán, J. A. (2017) · Population Health Management · doi:10.1089/pop.2016.0087 · Cited by 57
The Role of Economic Evaluation in Dissemination and Implementation Research
Raghavan, R. (2012) · Dissemination and Implementation Research in HealthTranslating Science to Practice · doi:10.1093/acprof:oso/9780199751877.003.0005 · Cited by 51
Measuring Costs to Community-Based Agencies for Implementation of an Evidence-Based Practice
Lang, J. M., & Connell, C. M. (2017) · The Journal of Behavioral Health Services & Research · doi:10.1007/s11414-016-9541-8 · Cited by 41
Thorpe, K., Jain, S., & Joski, P. (2017) · Health Affairs · doi:10.1377/hlthaff.2016.0875 · Cited by 41
Dopp, A. R., Hanson, R. F., Saunders, B. E., Dismuke, C. E., & Moreland, A. D. (2017) · Psychological Services · doi:10.1037/ser0000131 · Cited by 41
Palinkas, L. A., Um, M. Y., Jeong, C. H., Chor, K. H. B., Olin, S., Horwitz, S. M., & Hoagwood, K. E. (2017) · Health Research Policy and Systems · doi:10.1186/s12961-017-0190-z · Cited by 36
Trauma-Focused Treatment in a State System of Care: Is It Worth the Cost?
Greer, D., Grasso, D. J., Cohen, A., & Webb, C. P. M. (2013) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-013-0468-6 · Cited by 34
Dopp, A. R., Coen, A. S., Smith, A. B., Reno, J., Bernstein, D. H., Kerns, S. E. U., & Altschul, D. (2018) · Behavior Therapy · doi:10.1016/j.beth.2017.12.003 · Cited by 33
Knudsen, H. K., & Roman, P. M. (2014) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2014.08.003 · Cited by 31
H, D. B. M. P., Henry, B. F., Wasserman, G. A., Knight, D. K., Gardner, S. K., Krupka, K. E., Grossi, B., Cawood, M., Wiley, T., & Robertson, A. A. (2018) · Journal of Applied Juvenile Justice Services · doi:10.52935/18.21523.01 · Cited by 27
Self-citationThe Role of Economic Evaluation in Dissemination and Implementation Research
Raghavan, R. (2017) · Oxford University Press eBooks · doi:10.1093/oso/9780190683214.003.0006 · Cited by 25
Suarez, E., Jackson, D. S., Slavin, L. A., Michels, S., & McGeehan, K. M. (2014) · Europe PMC (PubMed Central) · Cited by 24
Lee, T. G., Walker, S. C., & Bishop, A. S. (2015) · Psychiatric Services · doi:10.1176/appi.ps.201400544 · Cited by 22
Economic analysis in behavioral health: Toward application of standardized methodologies.
Kaplan, R. M., Gold, M. R., Duffy, S. Q., Miller, N. E., Glassman, J. R., Chambers, D., Ganiats, T. G., Berndt, S., & Wilson, D. K. (2019) · Health Psychology · doi:10.1037/hea0000769 · Cited by 21
Implementing Residential Integrated Treatment for Co-occurring Disorders
McKee, S. A., Harris, G. T., & Cormier, C. A. (2013) · Journal of Dual Diagnosis · doi:10.1080/15504263.2013.807073 · Cited by 20
King, D. K., Gonzalez, S. J., Hartje, J. A., Hanson, B. L., Edney, C., Snell, H., Zoorob, R. J., & Roget, N. A. (2018) · Translational Behavioral Medicine · doi:10.1093/tbm/ibx020 · Cited by 17 · cited 7×
“Table 5 CHEERS quality rating CHEERS item [29] [28] [20] [25] [32] [33] [26] [30] [34] [21] [31] [27] [24] [22] [35] [23] [36] [19] Mean 1 Title 1 1 N/A 1 N/A N/A 1 1 N/A N/A 1 1 N/A 1 1 N/A N/A 1 100% 2 Abstract 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0.”
BackgroundInfluentialGeographic differences in substance use screening for justice-involved youth
Marks, K. R., Leukefeld, C. G., Dennis, M. L., Scott, C. K., & Funk, R. R. (2019) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2019.04.005 · Cited by 16
The Outcomes and Costs of an Interagency Partnership to Serve Maltreated Children in State Custody
Swenson, C. C., Randall, J., Henggeler, S. W., & Ward, D. M. (2000) · Children s Services · doi:10.1207/s15326918cs0304_1 · Cited by 15
H, D. B. M. P., Henry, B. F., & McCollister, K. E. (2019) · Health Services Insights · doi:10.1177/1178632919855037 · Cited by 15
Self-citationRollins, A. L., Kukla, M., Salyers, M. P., McGrew, J. H., Flanagan, M., LESLIE, D. L., Hunt, M. G., & McGuire, A. B. (2017) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-016-0785-7 · Cited by 14
Robertson, A. A., Hiller, M. L., Dembo, R., Dennis, M. L., Scott, C. K., Henry, B. F., & Elkington, K. S. (2019) · Journal of Child and Family Studies · doi:10.1007/s10826-019-01488-4 · Cited by 14
Self-citationThe Need for Quality Improvement in Behavioral Health
Gaudiano, B. A., Dalrymple, K. L., D’Avanzato, C., & Bunaciu, L. (2016) · Springer eBooks · doi:10.1007/978-3-319-26209-3_3 · Cited by 5
Shi, Y., Scanlon, D. P., Kang, R. H., McHugh, M., Greene, J., Christianson, J. B., Jean‐Jacques, M., Mahmud, Y., & Alexander, J. A. (2016) · PubMed · Cited by 5
Rochefort, D. A. (2020) · International Journal of Health Services · doi:10.1177/0020731420906746 · Cited by 4
Cited by
Cost data in implementation science: categories and approaches to costing
Gold, H. T., McDermott, C., Hoomans, T., & Wagner, T. H. (2022) · Implementation Science · doi:10.1186/s13012-021-01172-6 · Cited by 152
Eisman, A. B., Quanbeck, A., Bounthavong, M., Panattoni, L., & Glasgow, R. E. (2021) · Implementation Science · doi:10.1186/s13012-021-01143-x · Cited by 126
Saldana, L., Ritzwoller, D. P., Campbell, M. R., & Block, E. P. (2022) · Implementation Science Communications · doi:10.1186/s43058-022-00295-1 · Cited by 86
Implementation costs of hospital-based computerised decision support systems: a systematic review
Donovan, T., Abell, B., Fernando, M., McPhail, S., & Carter, H. E. (2023) · Implementation Science · doi:10.1186/s13012-023-01261-8 · Cited by 55
Tracing the Evolution of Agency Theory in Corporate Governance
Gwala, R. S., & Mashau, P. (2023) · Advances in public policy and administration (APPA) book series · doi:10.4018/978-1-6684-6966-8.ch013 · Cited by 42
Metadata
- Authors
- Bowser, D. M., Henry, B. F., & McCollister, K. E.
- Affiliations
- Brandeis University (US)
- University of Miami (US)
- Corresponding author
- Henry, Brandy F.
- Venue
- Implementation Science
- Published
- 2021-03-12
- Volume
- 16
- Issue
- 1
- Pages
- 26-26
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC7953634
- OpenAlex
- W3134013756
- Cited by
- 61
- References
- 51
- Citations by year
- 2026: 12 · 2025: 13 · 2024: 7 · 2023: 18 · 2022: 8 · 2021: 3
- FWCI
- 1.92
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0/ (tdm)
- creativecommons.org/licenses/by/4.0/ (vor)
- Locations
- Implementation Science (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institute on Drug Abuse (T32DA37801, R21DA044378, P30 DA040500, R01DA035808)
- National Institute on Alcohol Abuse and Alcoholism (T32AA007567)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Health Sciences › Medicine › Psychiatry and Mental health › Mental Health Treatment and Access
- Keywords
- cost analysis, evidence-based practice, implementation costs, behavioral health services, mental health services, substance use disorders, research-practice gap
- MeSH terms
- Child, Cost-Benefit Analysis, Delivery of Health Care, Humans, Mental Health, Substance-Related Disorders, Evidence-Based Practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
