What makes a sustainability tool valuable, practical and useful in real-world healthcare practice? A mixed-methods study on the development of the Long Term Success Tool in Northwest London
Open accessAbstract
Objectives: Although improvement initiatives show benefits to patient care, they often fail to sustain. Models and frameworks exist to address this challenge, but issues with design, clarity and usability have been barriers to use in healthcare settings. This work aimed to collaborate with stakeholders to develop a sustainability tool relevant to people in healthcare settings and practical for use in improvement initiatives. Design: Tool development was conducted in six stages. A scoping literature review, group discussions and a stakeholder engagement event explored literature findings and their resonance with stakeholders in healthcare settings. Interviews, small-scale trialling and piloting explored the design and tested the practicality of the tool in improvement initiatives. Setting: National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care for Northwest London (CLAHRC NWL). Participants: CLAHRC NWL improvement initiative teams and staff. Results: The iterative design process and engagement of stakeholders informed the articulation of the sustainability factors identified from the literature and guided tool design for practical application. Key iterations of factors and tool design are discussed. From the development process, the Long Term Success Tool (LTST) has been designed. The Tool supports those implementing improvements to reflect on 12 sustainability factors to identify risks to increase chances of achieving sustainability over time. The Tool is designed to provide a platform for improvement teams to share their own views on sustainability as well as learn about the different views held within their team to prompt discussion and actions. Conclusion: The development of the LTST has reinforced the importance of working with stakeholders to design strategies which respond to their needs and preferences and can practically be implemented in real-world settings. Further research is required to study the use and effectiveness of the tool in practice and assess engagement with the method over time.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Many approaches presented in this review recommend that they be used and evaluated further within other healthcare initiatives and settings to explore applicability and further development needed.
“Many approaches presented in this review recommend that they be used and evaluated further within other healthcare initiatives and settings to explore applicability and further development needed [11, 36, 56, 79].”
From Navigating the sustainability landscape: a systematic review of sustainability approaches in healthcare (2018) · cites this paper as [56]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Scoping studies: towards a methodological framework
Arksey, H., & O'Malley, L. (2005) · International Journal of Social Research Methodology · doi:10.1080/1364557032000119616 · Cited by 36,742 · see citing works
The qualitative content analysis process
Elo, S., & Kyngäs, H. (2008) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2007.04569.x · Cited by 22,617 · see citing works
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496 · see citing works
Evaluating the public health impact of health promotion interventions: the RE-AIM framework.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999) · American Journal of Public Health · doi:10.2105/ajph.89.9.1322 · Cited by 7,217 · see citing works
Leading change: why transformation efforts fail
Kotter, J. P. (2009) · IEEE Engineering Management Review · doi:10.1109/emr.2009.5235501 · Cited by 4,513 · see citing works
Taylor, M. J., McNicholas, C., Nicolay, C., Darzi, A., Bell, D., & Reed, J. E. (2014) · BMJ Quality & Safety · doi:10.1136/bmjqs-2013-001862 · Cited by 1,871 · see citing works
Self-citationRoles in Sociological Field Observations
Gold, R. L. (1958) · Social Forces · doi:10.2307/2573808 · Cited by 1,768 · see citing works
The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change
Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-117 · Cited by 1,724 · see citing works
Wiltsey Stirman, S., Kimberly, J., Cook, N., Calloway, A., Castro, F., & Charns, M. (2012) · Implementation Science · doi:10.1186/1748-5908-7-17 · Cited by 1,355 · see citing works
Shediac-Rizkallah, M. C., & Bone, L. R. (1998) · Health Education Research · doi:10.1093/her/13.1.87 · Cited by 1,129 · see citing works
Feldstein, A. C., & Glasgow, R. E. (2008) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(08)34030-6 · Cited by 987 · see citing works
Planning and Studying Improvement in Patient Care: The Use of Theoretical Perspectives
GROL, R. P. T. M., BOSCH, M. C., HULSCHER, M. E. J. L., ECCLES, M. P., & WENSING, M. (2007) · The Milbank Quarterly · doi:10.1111/j.1468-0009.2007.00478.x · Cited by 943 · see citing works
An Agenda for Research on the Sustainability of Public Health Programs
Scheirer, M. A., & Dearing, J. W. (2011) · American Journal of Public Health · doi:10.2105/ajph.2011.300193 · Cited by 775 · see citing works
Is Sustainability Possible? A Review and Commentary on Empirical Studies of Program Sustainability
Scheirer, M. A. (2005) · American Journal of Evaluation · doi:10.1177/1098214005278752 · Cited by 710 · see citing works
Sampling Designs in Qualitative Research: Making the Sampling Process More Public
Onwuegbuzie, A. J., & Leech, N. L. (2015) · The Qualitative Report · doi:10.46743/2160-3715/2007.1636 · Cited by 707 · see citing works
Kaplan, H. C., Provost, L., Froehle, C. M., & Margolis, P. A. (2011) · BMJ Quality & Safety · doi:10.1136/bmjqs-2011-000010 · Cited by 572 · see citing works
No going back: A review of the literature on sustaining organizational change
Buchanan, D., Fitzgerald, L., Ketley, D., Gollop, R., Jones, J. L., Lamont, S. S., Neath, A., & Whitby, E. (2005) · International Journal of Management Reviews · doi:10.1111/j.1468-2370.2005.00111.x · Cited by 475 · see citing works
Lean healthcare: rhetoric, ritual and resistance.
Waring, J., & Bishop, S. (2010) · Social Science & Medicine (1967) · doi:10.1016/j.socscimed.2010.06.028 · Cited by 434 · see citing works
Wiek, A., Ness, B., Schweizer-Ries, P., Brand, F. S., & Farioli, F. (2012) · Sustainability Science · doi:10.1007/s11625-011-0148-y · Cited by 412 · see citing works
Building capacity and sustainable prevention innovations: a sustainability planning model
Johnson, K., Hays, C., Center, H., & Daley, C. (2004) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2004.01.002 · Cited by 411 · see citing works
Sustainability science: an integrated approach for health-programme planning
Gruen, R. L., Elliott, J. H., Nolan, M. L., Lawton, P. D., Parkhill, A., McLaren, C. J., & Lavis, J. N. (2008) · The Lancet · doi:10.1016/s0140-6736(08)61659-1 · Cited by 395 · see citing works
Moullin, J. C., Sabater-Hernández, D., Fernandez-Llimos, F., & Benrimoj, S. I. (2015) · Health Research Policy and Systems · doi:10.1186/s12961-015-0005-z · Cited by 392 · see citing works
Hanney, S. R., Castle‐Clarke, S., Grant, J., Guthrie, S., Henshall, C., Mestre‐Ferrandiz, J., Pistollato, M., Pollitt, A., Sussex, J., & Wooding, S. (2015) · Health Research Policy and Systems · doi:10.1186/1478-4505-13-1 · Cited by 366 · see citing works
The Program Sustainability Assessment Tool: A New Instrument for Public Health Programs
Luke, D. A., Calhoun, A., Robichaux, C. B., Elliott, M. B., & Moreland-Russell, S. (2014) · Preventing Chronic Disease · doi:10.5888/pcd11.130184 · Cited by 364 · see citing works
The problem with plan-do-study-act cycles
Reed, J. E., & Card, A. J. (2016) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-005076 · Cited by 352 · see citing works
Self-citationThe hard side of change management
Sirkin, H. L., Keenan, P., & Jackson, A. (2014) · IEEE Engineering Management Review · doi:10.1109/emr.2014.6966953 · Cited by 313 · see citing works
The Effect of the Question on Survey Responses: A Review
Kalton, G., & Schuman, H. (1982) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2981421 · Cited by 259 · see citing works
Development of Level of Institutionalization Scales for Health Promotion Programs
Goodman, R. M., McLeroy, K. R., Steckler, A., & Hoyle, R. H. (1993) · Health Education Quarterly · doi:10.1177/109019819302000208 · Cited by 181 · see citing works
Doyle, C., Howe, C., Woodcock, T., Myron, R., Phekoo, K., McNicholas, C., Saffer, J., & Bell, D. (2013) · Implementation Science · doi:10.1186/1748-5908-8-127 · Cited by 166 · see citing works
Self-citationHooker, L., Small, R., Humphreys, C., Hegarty, K., & Taft, A. (2015) · Implementation Science · doi:10.1186/s13012-015-0230-4 · Cited by 132 · see citing works
Martin, G. P., Weaver, S., Currie, G., Finn, R., & McDonald, R. (2012) · Health Services Management Research · doi:10.1177/0951484812474246 · Cited by 116 · see citing works
Caldwell, S. E., & Mays, N. (2012) · Health Research Policy and Systems · doi:10.1186/1478-4505-10-32 · Cited by 105 · see citing works
Reed, J. E., McNicholas, C., Woodcock, T. M., Issen, L., & Bell, D. (2014) · BMJ Quality & Safety · doi:10.1136/bmjqs-2014-003103 · Cited by 102 · see citing works
Self-citationMeasuring persistence of implementation: QUERI Series
Initiative, T. H. Q. E. R., Bowman, C. C., Sobo, E. J., Asch, S. M., & Gifford, A. L. (2008) · Implementation Science · doi:10.1186/1748-5908-3-21 · Cited by 100 · see citing works
Leon, N., Lewin, S. A., & Mathews, C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-97 · Cited by 65 · see citing works
Chilundo, B., Cliff, J., Mariano, A. R., Rodríguez, D., & George, A. S. (2015) · Health Policy and Planning · doi:10.1093/heapol/czv036 · Cited by 51 · see citing works
Use of Implementation Theory: A Focus on PARIHS
Ullrich, P. M., Sahay, A. P., & Stetler, C. B. (2013) · Worldviews on Evidence-Based Nursing · doi:10.1111/wvn.12016 · Cited by 43 · see citing works
The REFReSH study group, Drew, S., Judge, A., May, C., Farmer, A., Cooper, C., Javaid, M. K., & Gooberman-Hill, R. (2015) · Implementation Science · doi:10.1186/s13012-015-0243-z · Cited by 43 · see citing works
A cluster-randomised quality improvement study to improve two inpatient stroke quality indicators
Williams, L., Daggett, V., Slaven, J. E., Yu, Z., Sager, D., Myers, J., Plue, L., Woodward-Hagg, H., & Damush, T. M. (2016) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-004188 · Cited by 42 · see citing works
A ‘work smarter, not harder’ approach to improving healthcare quality
Hayes, C. W., Batalden, P. B., & Goldmann, D. A. (2014) · BMJ Quality & Safety · doi:10.1136/bmjqs-2014-003673 · Cited by 39 · see citing works
Development of Level of Institutionalization Scales for Health Promotion Programs [and] Commentary.
Goodman, R. M. (1993) · Health Education Quarterly · Cited by 36 · see citing works
Model-driven approach to data collection and reporting for quality improvement
Ćurčin, V., Woodcock, T. M., Poots, A. J., Majeed, A., & Bell, D. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.04.014 · Cited by 28 · see citing works
Self-citationOnline emergency department ratings, patient satisfaction and the age-old issue of communication
Ranney, M. L., & Peimer, C. A. (2015) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-004806 · Cited by 26 · see citing works
Total quality management sustaining audit tool: Description and use
Dale, B. G., Boaden, R., Wilcox, M., et al. (1997) · Total Quality Management · doi:10.1080/0954412979406 · Cited by 24 · see citing works
Supporting improvement in a quality collaborative
Howe, C., Randall, K., Chalkley, S. R., & Bell, D. (2013) · British Journal of Healthcare Management · doi:10.12968/bjhc.2013.19.9.434 · Cited by 21 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- [47]Lennox L, Maher L, Bell D, et al The sustainability landscape: a systematic review of sustainability strategies and their impact in healthcare settings. PROSPERO Int Prospect Regist Syst Rev 2016; CRD4201604: 1– 4.
Cited by
Sorted by citations
Lennox, L., Maher, L., & Reed, J. (2018) · Implementation Science · doi:10.1186/s13012-017-0707-4 · Cited by 421 · see citing works
Self-citationCôté-Boileau, É., Denis, J.-L., Callery, B., & Sabean, M. (2019) · Health Research Policy and Systems · doi:10.1186/s12961-019-0482-6 · Cited by 133 · see citing works
Hall, A., Shoesmith, A., Doherty, E., McEvoy, B., Mettert, K., Lewis, C. C., Wolfenden, L., Yoong, S., Kingsland, M., Shelton, R. C., et al. (2022) · Implementation Science · doi:10.1186/s13012-022-01252-1 · Cited by 81 · see citing works
- Authors
- Lennox, L., Doyle, C., Reed, J. E., & Bell, D.
- Affiliations
- Imperial College London (GB)
- National Institute for Health and Care Research (GB)
- Corresponding author
- Lennox, Laura
- Venue
- BMJ Open
- Publisher
- BMJ
- Published
- 2017-09
- Volume
- 7
- Issue
- 9
- Pages
- e014417
- ISSN
- 2044-6055 (ISSN-L)
- PMC
- PMC5623390
- OpenAlex
- W2756638226
- Cited by
- 59
- References
- 45
- Citations by year
- 2026: 4 · 2025: 4 · 2024: 2 · 2023: 11 · 2022: 13 · 2021: 5 · 2020: 12 · 2019: 4 · 2018: 3
- FWCI
- 8.42
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- BMJ Open (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- HighWire Press Open Archive (submittedVersion)
- Spiral (Imperial College London) (submittedVersion, other-oa)
- Funders
- National Institute for Health Research
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Health Professions › Health Information Management › Healthcare Quality and Management
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Appreciative Inquiry and Organizational Change
- Keywords
- quality improvement, sustainability factors, stakeholder engagement, tool development, usability, improvement teams, patient care
- MeSH terms
- Diffusion of Innovation, Humans, Leadership, London, Quality Assurance, Health Care, State Medicine, Program Evaluation, Models, Organizational, Quality Improvement
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
