Using the Program Sustainability Assessment Tool to Assess and Plan for Sustainability
Open accessAbstract
Implementing and growing a public health program that benefits society takes considerable time and effort. To ensure that positive outcomes are maintained over time, program managers and stakeholders should plan and implement activities to build sustainability capacity within their programs. We describe a 3-part sustainability planning process that programs can follow to build their sustainability capacity. First, program staff and stakeholders take the Program Sustainability Assessment Tool to measure their program's sustainability across 8 domains. Next, managers and stakeholders use results from the assessment to inform and prioritize sustainability action planning. Lastly, staff members implement the plan and keep track of progress toward their sustainability goals. Through this process, staff can more holistically address the internal and external challenges and pressures associated with sustaining a program. We include a case example of a chronic disease program that completed the Program Sustainability Assessment Tool and engaged in program sustainability planning.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The PSAT has been found to be easy to use, requiring no or minimal training.
“In addition, the PSAT has been found to be easy to use, requiring no or minimal training (34).”
From Assessing the sustainability capacity of evidence-based programs in community and health settings (2022) · cites this paper as [34]
The Limitations and Implications for Practice section calls the PSAT a “simple, user-friendly tool,” supporting the claim that it is easy to use. Our copy does not say that using it requires no or minimal training; searches for training did not establish that part of the claim. The supplied passages have no page numbers. (LLM, conf=0.86)
Experimental — accuracy not yet measuredThe PSAT and CSAT have been completed online over 13,000 times to measure sustainability capacity across a variety of settings.
“Together, the PSAT and CSAT have been completed online over 13,000 times to measure the sustainability capacity of programs and practices across a variety of settings, including public health [10, 13], clinical care [9, 14], behavioral health [15], and education [16].”
From Lowering the burden: Shorter versions of the Program Sustainability Assessment Tool (PSAT) and Clinical Sustainability Assessment Tool (CSAT) (2024) · cites this paper as [10]
The “Complete the Program Sustainability Assessment Tool” section says the PSAT can be completed online, and “Limitations and Implications for Practice” reports its use by more than 550 public health practitioners and managers to assess programs. The paper does not report over 13,000 online completions or discuss the CSAT, so it supports only the PSAT-in-public-health portion of the claim. No page numbers were available in our copy. (LLM, conf=0.88)
Experimental — accuracy not yet measuredThere is emerging evidence for the reliability and validity of a small number of sustainability-specific assessments and tools designed to determine an organization's capacity in key factors linked to sustainment success and to develop strategic action plans.
“There is emerging evidence for the reliability and validity of a small number of sustainability-specific assessments and tools, which are designed to determine an organization's capacity in key factors (e.g., partnerships, strategic planning) that have been shown to be linked to sustainment success (61, 76) and to develop a strategic action plan for building capacity in areas of weakness (19, 35,”
From The Sustainability of Evidence-Based Interventions and Practices in Public Health and Health Care (2018) · cites this paper as [19]
These sustainability assessment and planning strategies have only begun to be evaluated in small pilot studies.
“However, these strategies have only begun to be evaluated in small pilot studies (19, 65).”
From The Sustainability of Evidence-Based Interventions and Practices in Public Health and Health Care (2018) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Shediac-Rizkallah, M. C., & Bone, L. R. (1998) · Health Education Research · doi:10.1093/her/13.1.87 · Cited by 1,129 · see citing works
Chaskin, R. J. (2001) · Urban Affairs Review · doi:10.1177/10780870122184876 · Cited by 825 · see citing works
An Agenda for Research on the Sustainability of Public Health Programs
Scheirer, M. A., & Dearing, J. W. (2011) · American Journal of Public Health · doi:10.2105/ajph.2011.300193 · Cited by 775 · see citing works
Is Sustainability Possible? A Review and Commentary on Empirical Studies of Program Sustainability
Scheirer, M. A. (2005) · American Journal of Evaluation · doi:10.1177/1098214005278752 · Cited by 710 · see citing works
Public health program capacity for sustainability: a new framework
Schell, S. F., Luke, D. A., Schooley, M. W., Elliott, M. B., Herbers, S. H., Mueller, N. B., & Bunger, A. C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-15 · Cited by 563 · see citing works
Self-citationBuilding capacity and sustainable prevention innovations: a sustainability planning model
Johnson, K., Hays, C., Center, H., & Daley, C. (2004) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2004.01.002 · Cited by 411 · see citing works
The Program Sustainability Assessment Tool: A New Instrument for Public Health Programs
Luke, D. A., Calhoun, A., Robichaux, C. B., Elliott, M. B., & Moreland-Russell, S. (2014) · Preventing Chronic Disease · doi:10.5888/pcd11.130184 · Cited by 364 · see citing works
Self-citationFactors Enhancing Sustained Use of Research-Based Instructional Practices
Gersten, R. M., Chard, D. J., & Baker, S. (2000) · Journal of Learning Disabilities · doi:10.1177/002221940003300505 · Cited by 208 · see citing works
Sustaining Community‐Based Programs for Families: Conceptualization and Measurement*
Mancini, J. A., & Marek, L. I. (2004) · Family Relations · doi:10.1111/j.0197-6664.2004.00040.x · Cited by 199 · cited 2× · see citing works
“Consider completing the PSAT annually as part of yearly progress reporting and compare the scores to monitor improvements (11).”
The failure of success: Challenges of disseminating effective substance abuse prevention programs
Backer, T. E. (2000) · Journal of Community Psychology · doi:10.1002/(sici)1520-6629(200005)28:3 · Cited by 155 · see citing works
Predictors and Level of Sustainability of Community Prevention Coalitions
Feinberg, M. E., Bontempo, D. E., & Greenberg, M. T. (2008) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2008.01.030 · Cited by 97 · see citing works
Leadership and Management Competencies Defined by Practicing Social Work Managers
Wimpfheimer, S. (2004) · Administration in Social Work · doi:10.1300/j147v28n01_04 · Cited by 58 · see citing works
Coalition Sustainability: Long-Term Successes & Lessons Learned
Lodl, K. A., & Stevens, G. T. (2002) · Journal of Extension · doi:10.66752/1077-5315.5092 · Cited by 14 · see citing works
Allies Against Asthma: A Midstream Comment on Sustainability
Friedman, A. R., & Wicklund, K. (2006) · Health Promotion Practice · doi:10.1177/1524839906287058 · Cited by 12 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [16]Herman RD. Executive leadership. In: Renz DO, editor. The Jossey-Bass handbook of nonprofit leadership and management. 3rd revised edition. San Francisco (CA): Jossey-Bass; 2010. p. 157–77.
- [18]Goodman RM. Bridging the gap in effective program implementation: from concept to application. J Community Psychol 2000; 28(3): 309– 21. 10.1002/(SICI)1520-6629(200005)28:3<309::AID-JCOP6>3.0.CO;2-O · DOI
Cited by
Sorted by citations
The Sustainability of Evidence-Based Interventions and Practices in Public Health and Health Care
Shelton, R. C., Cooper, B. R., & Stirman, S. W. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-014731 · Cited by 806 · see citing works
The Program Sustainability Assessment Tool: A New Instrument for Public Health Programs
Luke, D. A., Calhoun, A., Robichaux, C. B., Elliott, M. B., & Moreland-Russell, S. (2014) · Preventing Chronic Disease · doi:10.5888/pcd11.130184 · Cited by 364 · see citing works
Self-citationPalinkas, L. A., Chou, C.-P., Spear, S. E., Mendon, S. J., Villamar, J., & Brown, C. H. (2020) · Implementation Science · doi:10.1186/s13012-020-01030-x · Cited by 70 · see citing works
Assessing capacity for sustainability of effective programs and policies in local health departments
Tabak, R. G., Duggan, K., Smith, C., Aisaka, K., Moreland-Russell, S., & Brownson, R. C. (2016) · Journal of Public Health Management and Practice · doi:10.1097/phh.0000000000000254 · Cited by 40 · see citing works
Self-citationStoll, S., Janevic, M., Lara, M., Ramos-Valencia, G., Stephens, T. B., Persky, V., Uyeda, K., Ohadike, Y., & Malveaux, F. (2015) · Preventing Chronic Disease · doi:10.5888/pcd12.150133 · Cited by 32 · see citing works
Hall, A., Shoesmith, A., Shelton, R. C., Lane, C., Wolfenden, L., & Nathan, N. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph182111414 · Cited by 20 · see citing works
Assessing the sustainability capacity of evidence-based programs in community and health settings
Bacon, C., Malone, S., Prewitt, K., Hackett, R., Hastings, M., Dexter, S., & Luke, D. A. (2022) · Frontiers in Health Services · doi:10.3389/frhs.2022.1004167 · Cited by 22 · see citing works
“In addition, the PSAT has been found to be easy to use, requiring no or minimal training (34).”
MethodSelf-citationFrancis, L., Dunt, D., & Cadilhac, D. A. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010944 · Cited by 24 · see citing works
Examination of sustainability indicators for fall prevention strategies in three states
Smith, M. L., Durrett, N. K., Schneider, E. C., Byers, I. N., Shubert, T. E., Wilson, A. D., Towne, S. D., & Ory, M. G. (2018) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2018.02.001 · Cited by 22 · see citing works
King, D. K., Gonzalez, S. J., Hartje, J. A., Hanson, B. L., Edney, C., Snell, H., Zoorob, R. J., & Roget, N. A. (2018) · Translational Behavioral Medicine · doi:10.1093/tbm/ibx020 · Cited by 17 · see citing works
Malone, S., Prewitt, K., McKay, V., Zabotka, L., Bacon, C., & Luke, D. A. (2024) · Implementation Science Communications · doi:10.1186/s43058-024-00656-y · Cited by 12 · see citing works
“The PSAT and CSAT have been used in organizational evaluation as well as within public health and implementation science research [10–12].”
BackgroundMethodSelf-citation
- Authors
- Calhoun, A., Mainor, A., Moreland-Russell, S., Maier, R. C., Brossart, L., & Luke, D. A.
- Affiliations
- Washington University in St. Louis (US)
- University of North Carolina at Chapel Hill (US)
- Corresponding author
- Calhoun, Annaliese
- Venue
- Preventing Chronic Disease
- Published
- 2014-01-23
- Volume
- 11
- Pages
- 130185-130185
- ISSN
- 1545-1151 (ISSN-L)
- PMC
- PMC3900327
- OpenAlex
- W2055071990
- Cited by
- 100
- References
- 14
- Citations by year
- 2026: 3 · 2025: 5 · 2024: 20 · 2023: 16 · 2022: 12 · 2021: 7 · 2020: 16 · 2019: 3 · 2018: 8 · 2017: 1 · 2016: 3 · 2015: 5 · 2014: 1
- FWCI
- 6.02
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- public-domain
- Locations
- Preventing Chronic Disease (publishedVersion, public-domain)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Carolina Digital Repository (University of North Carolina at Chapel Hill) (submittedVersion)
- Funders
- Centers for Disease Control and Prevention
- U.S. Public Health Service (200-2011-42065)
- National Institute of Diabetes and Digestive and Kidney Diseases (P30DK092950)
- Topics
- Health Sciences › Health Professions › General Health Professions › Public Health Policies and Education
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
- Keywords
- program sustainability, sustainability planning, public health programs, stakeholder engagement, program evaluation
- MeSH terms
- Community Health Services, Factor Analysis, Statistical, Health Promotion, Humans, Public Health, Reproducibility of Results, Program Evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review