Conceptualizing and measuring sustainability of prevention programs, policies, and practices
Open accessAbstract
A large knowledge gap exists regarding the measurement of sustainability of evidence-based prevention programs for mental and behavioral health. We interviewed 45 representatives of 10 grantees and 9 program officers within 4 Substance Abuse and Mental Health Services Administration prevention grant initiatives to identify experiences with implementation and sustainability barriers and facilitators; what “sustainability” means and what it will take to sustain their programs; and which Consolidated Framework for Implementation Research (CFIR) elements are important for sustainability. Lists of sustainability determinants and outcomes were then compiled from each data set and compared with one another. Analysis of themes from interviews and free lists revealed considerable overlap between sustainability determinants and outcomes. Four sustainability elements were identified by all three data sets (ongoing coalitions, collaborations, and networks and partnerships; infrastructure and capacity to support sustainability; community need for program; and ongoing evaluation of performance and outcomes), and 11 elements were identified by two of three data sets (availability of funding; consistency with organizational culture; evidence of positive outcomes; development of a plan for implementation and sustainment; presence of a champion; institutionalization and integration of program; institutional support and commitment; community buy-in and support; program continuity; supportive leadership; and opportunities for staff training). All but one of the CFIR domain elements (pressure from other states, tribes, or communities) were endorsed as important to sustainability by 50% or more of participants. It may be more important to implement a standardized process of eliciting determinants and outcomes of sustainability than to implement a single standardized instrument.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496 · see citing works
Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-013-0528-y · Cited by 12,038 · see citing works
Self-citationDiffusion of Innovations in Service Organizations: Systematic Review and Recommendations
GREENHALGH, T., ROBERT, G., MACFARLANE, F., BATE, P., & KYRIAKIDOU, O. (2004) · The Milbank Quarterly · doi:10.1111/j.0887-378x.2004.00325.x · Cited by 7,448 · see citing works
Making sense of implementation theories, models and frameworks
Nilsen, P. (2015) · Implementation Science · doi:10.1186/s13012-015-0242-0 · Cited by 4,447 · see citing works
Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0327-7 · Cited by 3,240 · cited 3× · see citing works
“This finding supports the premise of the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework that features of the inner or outer setting or intervention that predict earlier stages of implementation may not necessarily predict for sustainability, the final stage [3].”
BackgroundResultThe dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change
Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-117 · Cited by 1,724 · see citing works
Wiltsey Stirman, S., Kimberly, J., Cook, N., Calloway, A., Castro, F., & Charns, M. (2012) · Implementation Science · doi:10.1186/1748-5908-7-17 · Cited by 1,355 · see citing works
The Sustainability of Evidence-Based Interventions and Practices in Public Health and Health Care
Shelton, R. C., Cooper, B. R., & Stirman, S. W. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-014731 · Cited by 806 · see citing works
An Agenda for Research on the Sustainability of Public Health Programs
Scheirer, M. A., & Dearing, J. W. (2011) · American Journal of Public Health · doi:10.2105/ajph.2011.300193 · Cited by 775 · see citing works
Sustainability of Teacher Implementation of School-Based Mental Health Programs
Han, S. S., & Weiss, B. (2005) · Journal of Abnormal Child Psychology · doi:10.1007/s10802-005-7646-2 · Cited by 641 · see citing works
Sustaining Community‐Based Programs for Families: Conceptualization and Measurement*
Mancini, J. A., & Marek, L. I. (2004) · Family Relations · doi:10.1111/j.0197-6664.2004.00040.x · Cited by 199 · see citing works
Saldana, L. (2014) · Implementation Science · doi:10.1186/1748-5908-9-43 · Cited by 188 · see citing works
Predicting program start-up using the stages of implementation measure
Saldana, L., Chamberlain, P., Wang, W., & Hendricks Brown, C. (2012) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-011-0363-y · Cited by 148 · see citing works
Johnson, A. M., Moore, J. E., Chambers, D. A., Rup, J., Dinyarian, C., & Straus, S. E. (2019) · Implementation Science · doi:10.1186/s13012-019-0895-1 · Cited by 53 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Advancing the pragmatic measurement of sustainment: a narrative review of measures
Moullin, J. C., Sklar, M., Green, A., Dickson, K. S., Stadnick, N. A., Reeder, K., & Aarons, G. A. (2020) · Implementation Science Communications · doi:10.1186/s43058-020-00068-8 · Cited by 72 · see citing works
Palinkas, L. A., Chou, C.-P., Spear, S. E., Mendon, S. J., Villamar, J., & Brown, C. H. (2020) · Implementation Science · doi:10.1186/s13012-020-01030-x · Cited by 70 · see citing works
Self-citationCabassa, L. J., Stefancic, A., Bochicchio, L., Tuda, D., Weatherly, C., & Lengnick-Hall, R. (2021) · Translational Behavioral Medicine · doi:10.1093/tbm/ibaa089 · Cited by 5 · see citing works
- Authors
- Palinkas, L. A., Spear, S. E., Mendon, S. J., Villamar, J., Reynolds, C., Green, C. D., Olson, C., Adade, A., & Brown, C. H.
- Affiliations
- University of Southern California (US)
- California State University, Northridge (US)
- Northwestern University (US)
- Substance Abuse and Mental Health Services Administration (US)
- Corresponding author
- Palinkas, Lawrence A
- Venue
- Translational Behavioral Medicine
- Publisher
- Oxford University Press (OUP)
- Published
- 2019-11-25
- Volume
- 10
- Issue
- 1
- Pages
- 136-145
- ISSN
- 1613-9860 (ISSN-L)
- 1869-6716
- PMC
- PMC7020391
- OpenAlex
- W2990046995
- Cited by
- 66
- References
- 14
- Citations by year
- 2026: 4 · 2025: 6 · 2024: 19 · 2023: 11 · 2022: 10 · 2021: 9 · 2020: 7
- FWCI
- 8.54
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model (vor)
- Locations
- Translational Behavioral Medicine (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute on Drug Abuse (1 R34 DA037516-01A1)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Medicine › Epidemiology › Substance Abuse Treatment and Outcomes
- Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
- Keywords
- program sustainability, prevention programs, mental health prevention, Consolidated Framework for Implementation Research, implementation barriers and facilitators, sustainability determinants, sustainability outcomes, SAMHSA, organizational culture, community buy-in, leadership, employee training, program evaluation
- MeSH terms
- Humans, Program Evaluation, Policy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
