After the Collaborative Is Over: What Sustains Quality Improvement Initiatives in Primary Care Practices?
Abstract
Background Multisite quality improvement (QI) initiatives, often known as collaboratives, involving primary care practices such as community health centers, academic practices, and managed care groups have been reported. Yet relatively little is known about the sustainability of these QI initiatives after the initial project, and frequently its funding, has ended. A series of practice characteristics that constitute critical elements for QI sustainability activities, as described in a Sustainability Pyramid Model, were proposed.Methods Approximately five months after the cessation of formalized activities of the final collaborative, 25 in-person interviews were conducted in 13 primary care practices that had participated in the three North Carolina Chronic Disease Management collaboratives, which initially involved 33 practices. Clinical outcomes were not considered.Findings Twelve of the 13 practices stated that the collaborative work resulted in improvement in one or more process and/or outcome clinical measures and those improvements have been continued. Five of the 13 practices reported that sustaining improvements had been a challenge since the collaboratives ended. Content analysis of the interviews indicated that the practices variously cited the practice characteristics, as included in the Sustainability Pyramid Model: regular meetings to study practice population data, leadership commitment, availability of infrastructure/staff support, pursuit of additional funding, publicity, and strategic partnerships.Discussion Although the improvement activities initiated during the collaborative were sustained, the process of developing and implementing new QI activities appeared to be more challenging for almost half of the practices. The practices that could accomplish this ongoing new QI process had "institutionalized" their QI strategies--a finding with important implications for sustainability.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Involved and committed leaders can help facilitate sustainability.
“Involved and committed leaders can help facilitate sustainability [38-40].”
From Successful Organizational Strategies to Sustain Use of A-CHESS: A Mobile Intervention for Individuals With Alcohol Use Disorders (2015) · cites this paper as [39]
Developing an ongoing measurement and feedback system facilitates and supports organizational sustainability efforts.
“Developing an ongoing measurement and feedback system facilitates and supports organizational sustainability efforts [39,50,52-54].”
From Successful Organizational Strategies to Sustain Use of A-CHESS: A Mobile Intervention for Individuals With Alcohol Use Disorders (2015) · cites this paper as [39]
This local participatory approach enhances sustainability efforts.
“This local participatory approach enhances sustainability efforts [39,49-51].”
From Successful Organizational Strategies to Sustain Use of A-CHESS: A Mobile Intervention for Individuals With Alcohol Use Disorders (2015) · cites this paper as [39]
Cites
Chronic disease management: what will it take to improve care for chronic illness?
Eh, W. (1998) · PubMed · Cited by 2,095 · see citing works
Improvements in Diabetes Processes of Care and Intermediate Outcomes: United States, 1988–2002
SAADDINE, J. B., Cadwell, B. L., Gregg, E. W., Engelgau, M. M., Vinicor, F., Imperatore, G., & Narayan, K. M. V. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-7-200604040-00005 · Cited by 561 · see citing works
Chin, M. H., Cook, S., Drum, M. L., Jin, L., Guillen, M., Humikowski, C. A., Koppert, J., Harrison, J. F., Lippold, S. A., & Schaefer, C. T. (2004) · Diabetes Care · doi:10.2337/diacare.27.1.2 · Cited by 208 · see citing works
The distinctiveness of case-oriented research.
Ragin, C. C. (1999) · PubMed · Cited by 181 · see citing works
Landon, B. E., Wilson, I. B., McInnes, D. K., Landrum, M. B., Hirschhorn, L. R., Marsden, P. V., Gustafson, D. H., & Cleary, P. D. (2004) · Annals of Internal Medicine · doi:10.7326/0003-4819-140-11-200406010-00010 · Cited by 173 · see citing works
Challenges of Change: A Qualitative Study of Chronic Care Model Implementation
Hroscikoski, M. C. (2006) · The Annals of Family Medicine · doi:10.1370/afm.570 · Cited by 165 · see citing works
Does the Collaborative Model Improve Care for Chronic Heart Failure?
Asch, S. M., Baker, D. W., KEESEY, J. W., Broder, M. S., Schonlau, M., Rosen, M., Wallace, P., & Keeler, E. B. (2005) · Medical Care · doi:10.1097/01.mlr.0000167182.72251.a1 · Cited by 95 · see citing works
Collaborative Clinical Quality Improvement for Pressure Ulcers in Nursing Homes
Lynn, J., West, J., Hausmann, S. M., Gifford, D. R., Nelson, R., McGann, P. E., Bergstrom, N. I., & Ryan, J. A. (2007) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2007.01380.x · Cited by 66 · see citing works
Stoeckle-Roberts, S., Reeves, M. J., Jacobs, B. S., Maddox, K., Choate, L., Wehner, S., & Mullard, A. J. (2006) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(06)32067-3 · Cited by 43 · see citing works
Seow, H. Y., Phillips, C. O., Rich, M. W., Spertus, J. A., Krumholz, H. M., & Lynn, J. (2006) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2005.00638.x · Cited by 43 · see citing works
Neily, J. B., Howard, K., Quigley, P. A., & Mills, P. D. (2005) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(05)31035-x · Cited by 38 · see citing works
Obstacles to collaborative quality improvement: the case of ambulatory general medical care
Gandhi, T. K., Puopolo, A. L., Dasse, P., Haas, J. S., Burstin, H. R., Cook, E. F., & Brennan, T. A. (2000) · International Journal for Quality in Health Care · doi:10.1093/intqhc/12.2.115 · Cited by 37 · see citing works
Sustainability of Partnership Projects: A Conceptual Framework and Checklist
Edwards, J. C., Feldman, P. H., Sangl, J., Polakoff, D., Stern, G., & Casey, D. (2007) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(07)33122-x · Cited by 36 · see citing works
The North Carolina Experience with the Diabetes Health Disparities Collaboratives
Wang, A., Wolf, M., Carlyle, R., Wilkerson, J., Porterfield, D. S., & Reaves, J. (2004) · The Joint Commission Journal on Quality and Safety · doi:10.1016/s1549-3741(04)30045-6 · Cited by 31 · see citing works
Self-citationInducing Sustainable Improvement in Depression Care in Primary Care Practices
Nease, D. E., Nutting, P. A., Dickinson, W. P., Bonham, A. J., Graham, D. G., Gallagher, K. M., & Main, D. S. (2008) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(08)34031-8 · Cited by 24 · see citing works
A statewide collaboration to monitor diabetes quality improvement among Wisconsin health plans.
Siomos, E. E., Newsom, R. S., Camponeschi, J. L., & Remington, P. (2005) · PubMed · Cited by 6 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Explaining Michigan: Developing an Ex Post Theory of a Quality Improvement Program
DIXON-WOODS, M., BOSK, C. L., AVELING, E. L., GOESCHEL, C. A., & PRONOVOST, P. J. (2011) · Milbank Quarterly · doi:10.1111/j.1468-0009.2011.00625.x · Cited by 603 · see citing works
Lennox, L., Maher, L., & Reed, J. (2018) · Implementation Science · doi:10.1186/s13012-017-0707-4 · Cited by 421 · see citing works
Implementation of the CALM intervention for anxiety disorders: a qualitative study
Curran, G. M., Sullivan, G., Mendel, P. J., Craske, M. G., Sherbourne, C. D., Stein, M. B., McDaniel, A., & Roy‐Byrne, P. (2012) · Implementation Science · doi:10.1186/1748-5908-7-14 · Cited by 349 · see citing works
Ford II, J. H., Alagoz, E., Dinauer, S., Johnson, K. A., Pe-Romashko, K., & Gustafson, D. H. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3965 · Cited by 73 · see citing works
Measuring Sustainability Within the Veterans Administration Mental Health System Redesign Initiative
Ford, J. H., Krahn, D., Wise, M., & Oliver, K. A. (2011) · Quality Management in Health Care · doi:10.1097/qmh.0b013e3182314b20 · Cited by 60 · see citing works
Strategies for sustaining a quality improvement collaborative and its patient safety gains
Parand, A., Benn, J., Burnett, S., Pinto, A., & Vincent, C. (2012) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzs030 · Cited by 61 · see citing works
Facilitators of Transforming Primary Care: A Look Under the Hood at Practice Leadership
Donahue, K. E., Halladay, J. R., Wise, A., Reiter, K. L., Lee, S.-Y. D., Ward, K., Mitchell, M., & Qaqish, B. F. (2013) · The Annals of Family Medicine · doi:10.1370/afm.1492 · Cited by 58 · see citing works
Crespo-González, C., Benrimoj, S. I., Scerri, M., & García‐Cárdenas, V. (2020) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2020.01.015 · Cited by 51 · see citing works
Breckenridge, J. P., Gray, N., Toma, M., Ashmore, S., Glassborow, R., Stark, C. R., & Renfrew, M. J. (2019) · BMJ Open Quality · doi:10.1136/bmjoq-2018-000553 · Cited by 49 · see citing works
Knight, A. W., Caesar, C., Ford, D., Coughlin, A., & Frick, C. (2012) · BMJ Quality & Safety · doi:10.1136/bmjqs-2011-000165 · Cited by 46 · see citing works
Brewster, L., Tarrant, C., & Dixon‐Woods, M. (2016) · Journal of Hospital Infection · doi:10.1016/j.jhin.2016.01.021 · Cited by 46 · see citing works
The role of chief executive officers in a quality improvement initiative: a qualitative study
Parand, A., Dopson, S., & Vincent, C. (2013) · BMJ Open · doi:10.1136/bmjopen-2012-001731 · Cited by 37 · see citing works
Gardner, K., Sibthorpe, B. M., Chan, M., Sargent, G. M., Dowden, M., & McAullay, D. R. (2018) · BMC Health Services Research · doi:10.1186/s12913-018-3308-2 · Cited by 33 · see citing works
Best-practice care pathway for improving management of mastitis and breast abscess
Patani, N., MacAskill, F. E. N., Eshelby, S., Omar, A., Kaura, A., Contractor, K. B., Thiruchelvam, P. T. R., Curtis, S., Main, J., Cunningham, D., et al. (2018) · British journal of surgery · doi:10.1002/bjs.10919 · Cited by 30 · see citing works
Flynn, R., Stevens, B., Bains, A., Kennedy, M., & Scott, S. D. (2022) · Systematic Reviews · doi:10.1186/s13643-022-02093-1 · Cited by 30 · see citing works
Health Districts as Quality Improvement Collaboratives and Multijurisdictional Entities
Livingood, W. C., Marshall, N. A., Peden, A. H., Gonzalez, K., Shah, G. H., Alexander, D. S., Penix, K. O., Lawrence, R. H., Toal, R. B., & Woodhouse, L. D. (2012) · Journal of Public Health Management and Practice · doi:10.1097/phh.0b013e31825b89fd · Cited by 26 · see citing works
Backman, R., Foy, R., Michael, B. D., Defres, S., Kneen, R., & Solomon, T. (2015) · Implementation Science · doi:10.1186/s13012-015-0224-2 · Cited by 26 · see citing works
Family care map: Sustaining family-centered care in Polytrauma Rehabilitation Centers
Ford, J. H., Wise, M., Krahn, D., Oliver, K. A., Hall, C., & Sayer, N. (2014) · Journal of Rehabilitation Research and Development · doi:10.1682/jrrd.2014.03.0066 · Cited by 24 · see citing works
Flexible implementation and integration of new team members to support patient-centered care
Grace, S. M., Rich, J., Chin, W., & Rodríguez, H. P. (2014) · Healthcare · doi:10.1016/j.hjdsi.2014.02.003 · Cited by 21 · see citing works
The role of quality improvement collaboratives in general practice: a qualitative systematic review
Knight, A. W., Tam, C. W. M., Dennis, S. M., Fraser, J. D., & Pond, D. C. (2022) · BMJ Open Quality · doi:10.1136/bmjoq-2021-001800 · Cited by 21 · see citing works
Donahue, K. E., Newton, W. P., Lefebvre, A., & Plescia, M. (2013) · The Annals of Family Medicine · doi:10.1370/afm.1497 · Cited by 20 · see citing works
Stone, S., Lee, H. C., & Sharek, P. J. (2016) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(16)42042-8 · Cited by 20 · see citing works
Katz, D. A., Wu, C., Jaske, E. L., Stewart, G. L., & Mohr, D. C. (2020) · The Annals of Family Medicine · doi:10.1370/afm.2569 · Cited by 19 · see citing works
Balicer, R. D., Hoshen, M. B., Cohen-Stavi, C. J., Spitzer, S., Kay, C., Bitterman, H., Lieberman, N., Jacobson, O., & Shadmi, E. (2015) · Health Services Research · doi:10.1111/1475-6773.12300 · Cited by 17 · see citing works
Reducing wrong intraocular lens implants in cataract surgery
Loh, H. P., de Korne, D. F., Chee, S.-P., & Mathur, R. (2017) · International Journal of Health Care Quality Assurance · doi:10.1108/ijhcqa-06-2016-0095 · Cited by 17 · see citing works
Croff, R. L., Hoffman, K. A., Alanis‐Hirsch, K., Ford, J., McCarty, D. W., & Schmidt, L. A. (2018) · The Journal of Behavioral Health Services & Research · doi:10.1007/s11414-018-9616-9 · Cited by 15 · see citing works
Implementation of M-CHAT Screening for Autism in Primary Care in Saudi Arabia
Alawami, A. H., Perrin, E. C., & Sakai, C. (2019) · Global Pediatric Health · doi:10.1177/2333794x19852021 · Cited by 14 · see citing works
Sustainment of Lean Redesigns for Primary Care Teams
Hung, D. Y., Gray, C. P., Truong, Q. A., & Harrison, M. I. (2019) · Quality Management in Health Care · doi:10.1097/qmh.0000000000000200 · Cited by 13 · see citing works
Sathe, N. A., Nocon, R. S., Hughes, B. L., Peek, M. E., Chin, M. H., & Huang, E. S. (2016) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(16)42002-7 · Cited by 11 · see citing works
Ford, J. H., Krahn, D. D., Oliver, K. A., & Kirchner, J. E. (2012) · Quality Management in Health Care · doi:10.1097/qmh.0b013e31826d1d1a · Cited by 9 · see citing works
How Eight Primary Care Practices Initiated and Maintained Quality Monitoring and Reporting
Sloane, P. D., Wroth, T. H., Halladay, J. R., Bray, P., Spragens, L. H., Stearns, S. C., & Zimmerman, S. (2011) · The Journal of the American Board of Family Medicine · doi:10.3122/jabfm.2011.04.100101 · Cited by 9 · see citing works
Self-citationLessons Learned From Implementing Health Coaching in The Heart Healthy Lenoir Hypertension Study
Donahue, K. E., Tillman, J., Halladay, J. R., Cené, C. W., Hinderliter, A. L., Cummings, D. M., Miller, C. L., Gizlice, Z., Garcia, B. A., Wu, J. R., et al. (2016) · Progress in community health partnerships · doi:10.1353/cpr.2016.0064 · Cited by 8 · see citing works
Self-citationSustaining Improvement Initiatives: Challenges and Potential Tools
Rolnitsky, A., & Bell, C. M. (2023) · Clinics in Perinatology · doi:10.1016/j.clp.2023.01.001 · Cited by 6 · see citing works
Lasting change: sustaining improvements in oral health care
Demko, C. A. (2011) · Journal of Public Health Dentistry · doi:10.1111/j.1752-7325.2011.00231.x · Cited by 4 · see citing works
Ayeleke, R. O., Tenbensel, T., Silwal, P. R., & Walton, L. (2020) · Journal of Primary Health Care · doi:10.1071/hc20012 · Cited by 4 · see citing works
Aurelio, M. G., Singh, S. A., Chitewe, A., Mckenna, C., Marufu, E., Bridle, D., Sunduza, L., & Shah, A. (2025) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzaf070 · Cited by 4 · see citing works
Bray, P., Cummings, D. M., & Thompson, D. (2011) · North Carolina Medical Journal · doi:10.18043/ncm.72.5.390 · Cited by 3 · see citing works
Self-citationImproving Blood Pressure Control in Individuals with Diabetes: A Quality Improvement Collaborative
Powers, M. A., Cuddihy, R. M., Bergenstal, R. M., Tompos, P., Pearson, J., & Morgan, B. (2011) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(11)37013-4 · Cited by 3 · see citing works
Murphy, F., O’Brien, B., & Barry, M. (2016) · Arrow@dit (Dublin Institute of Technology) · doi:10.34961/10238
Runge, T. M. (2019) · Carolina Digital Repository (University of North Carolina at Chapel Hill) · doi:10.17615/zjak-fj02
Health Management, Health Administration, and Quality Improvement
(2013) · Elsevier eBooks · doi:10.1016/b978-1-4557-0658-7.00028-1
- Authors
- Bray, P., Cummings, D. M., Wolf, M., Massing, M. W., & Reaves, J.
- Affiliations
- East Carolina University (US)
- NC Department of Health and Human Services (US)
- Corresponding author
- Bray, Paul
- Venue
- The Joint Commission Journal on Quality and Patient Safety
- Publisher
- Elsevier BV
- Published
- 2009-10
- Volume
- 35
- Issue
- 10
- Pages
- 502-AP3
- ISSN
- 1553-7250 (ISSN-L)
- OpenAlex
- W2419603663
- Cited by
- 41
- References
- 16
- Citations by year
- 2025: 1 · 2023: 1 · 2022: 2 · 2020: 3 · 2019: 4 · 2018: 4 · 2017: 1 · 2016: 5 · 2015: 3 · 2014: 2 · 2013: 4 · 2012: 5
- FWCI
- 6.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Joint Commission Journal on Quality and Patient Safety (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › General Health Professions › Primary Care and Health Outcomes
- Health Sciences › Health Professions › Health Information Management › Healthcare Quality and Management
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Keywords
- sustainability of quality improvement, primary care practices, chronic disease management, practice characteristics, leadership commitment, strategic partnership, community health centers
- MeSH terms
- Attitude of Health Personnel, Chronic Disease, Cooperative Behavior, Humans, Interinstitutional Relations, Primary Health Care, Quality Assurance, Health Care, Program Evaluation, Community Networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review