Measuring Sustainability Within the Veterans Administration Mental Health System Redesign Initiative
Open accessAbstract
OBJECTIVE: To examine how attributes affecting sustainability differ across Veterans Health Administration organizational components and by staff characteristics. SUBJECTS: Surveys of 870 change team members and 50 staff interviews within the Veterans Affairs' Mental Health System Redesign initiative. METHODS: A 1-way ANOVA with a Tukey post hoc test examined differences in sustainability by Veteran Integrated Service Networks, job classification, and tenure from staff survey data of the Sustainability Index. Qualitative interviews used an iterative process to identify "a priori" and "in vivo" themes. A simple stepwise linear regression explored predictors of sustainability. RESULTS: Sustainability differed across Veteran Integrated Service Networks and staff tenure. Job classification differences existed for the following: (1) benefits and credibility of the change and (2) staff involvement and attitudes toward change. Sustainability barriers were staff and institutional resistance and nonsupportive leadership. Facilitators were commitment to veterans, strong leadership, and use of quality improvement tools. Sustainability predictors were outcomes tracking, regular reporting, and use of Plan, Do, Study, Adjust cycles. CONCLUSIONS: Creating homogeneous implementation and sustainability processes across a national health system is difficult. Despite the Veterans Affairs' best evidence-based implementation efforts, there was significant variance. Locally tailored interventions might better support sustainability than "one-size-fits-all" approaches. Further research is needed to understand how participation in a quality improvement collaborative affects sustainability.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Leaders in agencies that successfully sustained A-CHESS took steps to ensure that their strategic decisions and plans for A-CHESS created a coherent vision for organizational transformation; empowered employees to participate in the change process, including providing training; and allocated resources, change policies, procedures, and job descriptions.
“These leaders took steps to ensure that their strategic decisions and plans for A-CHESS created a coherent vision for organizational transformation; empowered employees to participate in the change process, including providing training; and allocated resources, change policies, procedures, and job descriptions [31,41,42].”
From Successful Organizational Strategies to Sustain Use of A-CHESS: A Mobile Intervention for Individuals With Alcohol Use Disorders (2015) · cites this paper as [41]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Qualitative Analysis for Social Scientists
Strauss, A. L. (1987) · doi:10.1017/cbo9780511557842 · Cited by 12,277 · see citing works
Diffusion of Innovations in Service Organizations: Systematic Review and Recommendations
GREENHALGH, T., ROBERT, G., MACFARLANE, F., BATE, P., & KYRIAKIDOU, O. (2004) · The Milbank Quarterly · doi:10.1111/j.0887-378x.2004.00325.x · Cited by 7,448 · see citing works
The Department of Veterans Affairs' NSQIP
Khuri, S. F., Daley, J., Henderson, W. G., Hur, K., Demakis, J. G., Aust, J. B., Chong, V., Fabri, P. J., Gibbs, J. O., Grover, F. L., et al. (1998) · Annals of Surgery · doi:10.1097/00000658-199810000-00006 · Cited by 1,534 · see citing works
Choosing Strategies for Change
Kotter, J. P., & Schlesinger, L. A. (1989) · doi:10.1007/978-1-349-20317-8_21 · Cited by 1,391 · see citing works
Ferlie, E. B., & Shortell, S. M. (2001) · The Milbank Quarterly · doi:10.1111/1468-0009.00206 · Cited by 1,335 · see citing works
Beer, M., & Nohria, N. (2000) · PubMed · Cited by 1,182 · see citing works
Shediac-Rizkallah, M. C., & Bone, L. R. (1998) · Health Education Research · doi:10.1093/her/13.1.87 · Cited by 1,129 · see citing works
Association Between Implementation of a Medical Team Training Program and Surgical Mortality
Neily, J. B., Mills, P. D., Young‐Xu, Y., Carney, B. T., West, P., Berger, D. H., Mazzia, L. M., Paull, D. E., & Bagian, J. P. (2010) · JAMA · doi:10.1001/jama.2010.1506 · Cited by 1,039 · see citing works
Cognitive, Emotional, and Language Processes in Disclosure
Pennebaker, J. W., & Francis, M. E. (1996) · Cognition & Emotion · doi:10.1080/026999396380079 · Cited by 986 · see citing works
Is Sustainability Possible? A Review and Commentary on Empirical Studies of Program Sustainability
Scheirer, M. A. (2005) · American Journal of Evaluation · doi:10.1177/1098214005278752 · Cited by 710 · see citing works
The heart of change: real-life stories of how people change their organizations
Kotter, J. P., & Cohen, D. S. (2002) · Choice Reviews Online · doi:10.5860/choice.40-1642 · Cited by 695 · see citing works
Knowledge Translation in Health Care: Moving from Evidence to Practice
(2009) · International Journal of Health Care Quality Assurance · doi:10.1108/ijhcqa.2009.06222fae.002 · Cited by 689 · see citing works
No going back: A review of the literature on sustaining organizational change
Buchanan, D., Fitzgerald, L., Ketley, D., Gollop, R., Jones, J. L., Lamont, S. S., Neath, A., & Whitby, E. (2005) · International Journal of Management Reviews · doi:10.1111/j.1468-2370.2005.00111.x · Cited by 475 · see citing works
Telling Stories: The Health Benefits of Narrative
Pennebaker, J. W. (2000) · Literature and medicine · doi:10.1353/lm.2000.0011 · Cited by 435 · see citing works
Building capacity and sustainable prevention innovations: a sustainability planning model
Johnson, K., Hays, C., Center, H., & Daley, C. (2004) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2004.01.002 · Cited by 411 · see citing works
Piette, J. D., Weinberger, M., Kraemer, F. B., & McPhee, S. J. (2001) · Diabetes Care · doi:10.2337/diacare.24.2.202 · Cited by 346 · see citing works
Sustainability: the elusive element of process improvement
Bateman, N. (2005) · International Journal of Operations & Production Management · doi:10.1108/01443570510581862 · Cited by 268 · see citing works
Weiner, B. J., Shortell, S. M., & Alexander, J. A. (1997) · PubMed · Cited by 251 · see citing works
What Do Collaborative Improvement Projects Do? Experience from Seven Countries
Wilson, T., Berwick, D. M., & Cleary, P. D. (2003) · The Joint Commission Journal on Quality and Safety · doi:10.1016/s1549-3741(03)29011-0 · Cited by 210 · see citing works
Correlates of the Sustainability of Community-Based Heart Health Promotion Interventions
O’Loughlin, J. L., Rénaud, L., Richard, L., Gomez, L. S., & Paradis, G. (1998) · Preventive Medicine · doi:10.1006/pmed.1998.0348 · Cited by 175 · see citing works
Nembhard, I. M. (2008) · Health Services Research · doi:10.1111/j.1475-6773.2008.00923.x · Cited by 153 · see citing works
Stetler, C. B., McQueen, L., Demakis, J. G., & Mittman, B. S. (2008) · Implementation Science · doi:10.1186/1748-5908-3-30 · Cited by 142 · see citing works
Mills, P. D., & Weeks, W. B. (2004) · The Joint Commission Journal on Quality and Safety · doi:10.1016/s1549-3741(04)30017-1 · Cited by 128 · see citing works
Hammermeister, K. E., Johnson, R. R., Marshall, G. R., et al. (1994) · Annals of Surgery · doi:10.1097/00000658-199403000-00008 · Cited by 123 · see citing works
Redesigning Work Processes in Health Care: Lessons from the National Health Service
Ham, C., Kipping, R., & McLeod, H. (2003) · Milbank Quarterly · doi:10.1111/1468-0009.t01-3-00062 · Cited by 118 · see citing works
Workforce Cultural Factors in TQM/CQI Implementation in Hospitals
Huq, Z., & Martin, T. N. (2000) · Health Care Management Review · doi:10.1097/00004010-200007000-00009 · Cited by 104 · see citing works
Measuring persistence of implementation: QUERI Series
Initiative, T. H. Q. E. R., Bowman, C. C., Sobo, E. J., Asch, S. M., & Gifford, A. L. (2008) · Implementation Science · doi:10.1186/1748-5908-3-21 · Cited by 100 · see citing works
Why do employees resist change?
Strebel, P. J. (2009) · IEEE Engineering Management Review · doi:10.1109/emr.2009.5235497 · Cited by 99 · see citing works
Sustainability of Social Programs: A Comparative Case Study Analysis
Savaya, R., Spiro, S., & Elran-Barak, R. (2008) · American Journal of Evaluation · doi:10.1177/1098214008325126 · Cited by 98 · see citing works
Gollop, R. (2004) · BMJ Quality & Safety · doi:10.1136/qshc.2003.007450 · Cited by 95 · see citing works
Bonello, R. S., Fletcher, C. E., Becker, W. K., Clutter, K., Arjes, S. L., Cook, J. J., & Petzel, R. A. (2008) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(08)34081-1 · Cited by 88 · see citing works
Cleeland, C. S., Reyes‐Gibby, C. C., Schall, M. W., Nolan, K. M., Paice, J. A., Rosenberg, J. M., Tollett, J. H., & Kerns, R. D. (2003) · Clinical Journal of Pain · doi:10.1097/00002508-200309000-00003 · Cited by 84 · see citing works
Institutionalizing organizational change through cascade training
Jacobs, R. L. (2002) · Journal of European Industrial Training · doi:10.1108/03090590210422058 · Cited by 77 · see citing works
Lessons Learned about Implementing Research Evidence into Clinical Practice.
Hagedorn, H., Hogan, M. M., Smith, J. L., Bowman, C., Curran, G. M., Espadas, D., Kimmel, B., Kochevar, L. K., Legro, M. W., & Sales, A. E. (2006) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2006.00358.x · Cited by 76 · see citing works
Parker, L. E., Kirchner, J. E., Bonner, L. M., Fickel, J. J., Ritchie, M. J., Simons, C. E., & Yano, E. M. (2009) · Qualitative Health Research · doi:10.1177/1049732308329481 · Cited by 71 · see citing works
Using a Framework for Spread: The Case of Patient Access in the Veterans Health Administration
Nolan, K. M., Schall, M. W., Erb, F., & Nolan, T. W. (2005) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(05)31045-2 · Cited by 63 · see citing works
Parker, L. E., de Pillis, E., Altschuler, A., Rubenstein, L. V., & Meredith, L. S. (2007) · Qualitative Health Research · doi:10.1177/1049732307307447 · Cited by 63 · see citing works
Projected Sustainability of Innovative Social Programs
Savaya, R., Elsworth, G., & Rogers, P. (2009) · Evaluation Review · doi:10.1177/0193841x08322860 · Cited by 53 · see citing works
What Do Collaborative Improvement Projects Do? Experience from Seven Countries
Wilson, T., Berwick, D. M., & Cleary, P. D. (2004) · The Joint Commission Journal on Quality and Safety · doi:10.1016/s1549-3741(04)30106-1 · Cited by 49 · see citing works
Sridharan, S., Go, S., Zinzow, H., Gray, A., & Gutierrez Barrett, M. (2007) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2006.10.006 · Cited by 46 · see citing works
Evaluating the Legacy of Community Health Initiatives
Beery, W. L., Senter, S., Cheadle, A. D., Greenwald, H. P., Pearson, D. C., Brousseau, R., & Nelson, G. D. (2005) · American Journal of Evaluation · doi:10.1177/1098214005275627 · Cited by 45 · see citing works
Implementation of a Clinical Innovation
Lukas, C. V., Meterko, M. M., Mohr, D. C., Seibert, M. N., Parlier, R., Levesque, O., & Petzel, R. A. (2008) · Journal of Ambulatory Care Management · doi:10.1097/01.jac.0000314699.04301.3e · Cited by 45 · see citing works
Leading strategic change : breaking through the brain barrier
Black, J. S., & Gregersen, H. B. (2002) · Financial Times Prentice Hall eBooks · Cited by 42 · see citing works
Bray, P., Cummings, D. M., Wolf, M., Massing, M. W., & Reaves, J. (2009) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(09)35069-2 · Cited by 41 · see citing works
The North Carolina Experience with the Diabetes Health Disparities Collaboratives
Wang, A., Wolf, M., Carlyle, R., Wilkerson, J., Porterfield, D. S., & Reaves, J. (2004) · The Joint Commission Journal on Quality and Safety · doi:10.1016/s1549-3741(04)30045-6 · Cited by 31 · see citing works
Using a NIATx based local learning collaborative for performance improvement
Roosa, M. R., Scripa, J. S., Zastowny, T. R., & Ford, J. H. (2011) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2011.02.006 · Cited by 30 · see citing works
Self-citationSolberg, L. (2009) · Implementation Science · doi:10.1186/1748-5908-4-9 · Cited by 28 · see citing works
Sustainable Change in the Public Sector: The Longitudinal Benefits of Organization Development
Wayne Boss, R., Dunford, B. B., Boss, A. D., & McConkie, M. L. (2010) · The Journal of Applied Behavioral Science · doi:10.1177/0021886310387087 · Cited by 22 · see citing works
Fremont, A. M., Joyce, G., Anaya, H. D., et al. (2006) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(06)32042-9 · Cited by 18 · see citing works
Achieving Large System Change in Health Care
Oldham, J. M. (2009) · JAMA · doi:10.1001/jama.2009.228 · Cited by 16 · see citing works
Weeks, W. B., Mills, P., Waldron, J., et al. (2003) · Journal of Healthcare Management · doi:10.1097/00115514-200307000-00009 · Cited by 16 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [25]Kotter JP Leading Change 1996 Boston, MA Harvard Business Press
- [42]Davies B Tremblay D Edwards N Bick D Graham ID Sustaining evidence based practice systems and measuring the impacts Evaluating the Impact of Implementing Evidence Based Practice 2010 Oxford Wiley-Blackwell 165 188
Cited by
Sorted by citations
The Sustainability of Evidence-Based Interventions and Practices in Public Health and Health Care
Shelton, R. C., Cooper, B. R., & Stirman, S. W. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-014731 · Cited by 806 · see citing works
Doyle, C., Howe, C., Woodcock, T., Myron, R., Phekoo, K., McNicholas, C., Saffer, J., & Bell, D. (2013) · Implementation Science · doi:10.1186/1748-5908-8-127 · Cited by 166 · see citing works
Braithwaite, J., Ludlow, K., Testa, L., Herkes, J., Augustsson, H., Lamprell, G., McPherson, E., & Zurynski, Y. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-036453 · Cited by 147 · see citing works
Meza, R. D., Triplett, N. S., Woodard, G. S., Martin, P., Khairuzzaman, A. N., Jamora, G., & Dorsey, S. (2021) · Implementation Science · doi:10.1186/s13012-021-01104-4 · Cited by 76 · see citing works
Ford II, J. H., Alagoz, E., Dinauer, S., Johnson, K. A., Pe-Romashko, K., & Gustafson, D. H. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3965 · Cited by 73 · see citing works
Sayer, N. A., Rosen, C. S., Bernardy, N. C., Cook, J. M., Orazem, R. J., Chard, K. M., Mohr, D. C., Kehle-Forbes, S. M., Eftekhari, A., Crowley, J., et al. (2017) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-017-0809-y · Cited by 68 · see citing works
Mohr, D. C., Rosen, C. S., Schnurr, P. P., Orazem, R. J., Noorbaloochi, S., Clothier, B. A., Eftekhari, A., Bernardy, N. C., Chard, K. M., Crowley, J. J., et al. (2018) · Psychiatric Services · doi:10.1176/appi.ps.201700432 · Cited by 26 · see citing works
Family care map: Sustaining family-centered care in Polytrauma Rehabilitation Centers
Ford, J. H., Wise, M., Krahn, D., Oliver, K. A., Hall, C., & Sayer, N. (2014) · Journal of Rehabilitation Research and Development · doi:10.1682/jrrd.2014.03.0066 · Cited by 24 · see citing works
Self-citation
- Authors
- Ford, J. H., Krahn, D., Wise, M., & Oliver, K. A.
- Affiliations
- University of Wisconsin–Madison (US)
- Corresponding author
- Ford, James H.
- ORCID
- Ford, James H.
- Venue
- Quality Management in Health Care
- Published
- 2011-10
- Volume
- 20
- Issue
- 4
- Pages
- 263-279
- ISSN
- 1063-8628 (ISSN-L)
- PMC
- PMC3188394
- OpenAlex
- W2050871368
- Cited by
- 60
- References
- 51
- Citations by year
- 2025: 3 · 2024: 5 · 2023: 3 · 2022: 7 · 2021: 13 · 2020: 2 · 2019: 2 · 2018: 5 · 2017: 3 · 2016: 3 · 2015: 3 · 2014: 2 · 2013: 6 · 2012: 3
- FWCI
- 6.53
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Quality Management in Health Care (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute on Drug Abuse (R01DA020832)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Health Professions › Health Information Management › Healthcare Quality and Management
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Organizational Change and Leadership
- Keywords
- sustainability, implementation, Veterans Health Administration, organizational change, staff characteristics, occupational classification, resistance to change, leadership, quality improvement, veterans' mental health services
- MeSH terms
- Female, Health Personnel, Humans, Male, Mental Health Services, Occupations, Time Factors, United States, United States Department of Veterans Affairs, Delivery of Health Care, Integrated
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
