Cited by 147Health Policy Implementation ScienceEvaluation and Performance AssessmentHealthcare Quality and Management

Built to last? The sustainability of healthcare system improvements, programmes and interventions: a systematic integrative review

Open access
Braithwaite, Jeffrey, Ludlow, Kristiana, Testa, Luke, Herkes, Jessica, Augustsson, Hanna, Lamprell, Gina, McPherson, Elise, Zurynski, Yvonne·2020·BMJ Open

Abstract

INTRODUCTION: The sustainability of healthcare delivery systems is challenged by ageing populations, complex systems, increasing rates of chronic disease, increasing costs associated with new medical technologies and growing expectations by healthcare consumers. Healthcare programmes, innovations and interventions are increasingly implemented at the front lines of care to increase effectiveness and efficiency; however, little is known about how sustainability is conceptualised and measured in programme evaluations. OBJECTIVES: We aimed to describe theoretical frameworks, definitions and measures of sustainability, as applied in published evaluations of healthcare improvement programmes and interventions. DESIGN: Systematic integrative review. METHODS: We searched six academic databases, CINAHL, Embase, Ovid MEDLINE, Emerald Management, Scopus and Web of Science, for peer-reviewed English journal articles (July 2011-March 2018). Articles were included if they assessed programme sustainability or sustained outcomes of a programme at the healthcare system level. Six reviewers conducted the abstract and full-text review. Data were extracted on study characteristics, definitions, terminology, theoretical frameworks, methods and tools. Hawker's Quality Assessment Tool was applied to included studies. RESULTS: Of the 92 included studies, 75.0% were classified as high quality. Twenty-seven (29.3%) studies provided 32 different definitions of sustainability. Terms used interchangeably for sustainability included continuation, maintenance, follow-up or long term. Eighty studies (87.0%) clearly reported the timepoints at which sustainability was evaluated: 43.0% at 1-2 years and 11.3% at <12 months. Eighteen studies (19.6%) used a theoretical framework to conceptualise or assess programme sustainability, including frameworks that were not specifically designed to assess sustainability. CONCLUSIONS: The body of literature is limited by the use of inconsistent definitions and measures of programme sustainability. Evaluations of service improvement programmes and interventions seldom used theoretical frameworks. Embedding implementation science and healthcare service researchers into the healthcare system is a promising strategy to improve the rigour of programme sustainability evaluations.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/bmjopen-2019-036453

Cites

72 of 72 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. [15]Ferlie E, Montgomery K, Reff A, et al Patient safety and quality: The Oxford handbook of health care management. Oxford. UK: Oxford University Press, 2016: 325– 51.

Cited by

Sorted by citations

Authors
Braithwaite, J., Ludlow, K., Testa, L., Herkes, J., Augustsson, H., Lamprell, G., McPherson, E., & Zurynski, Y.
Affiliations
  • National Health and Medical Research Council (AU)
  • Macquarie University (AU)
Corresponding author
Braithwaite, Jeffrey
Venue
BMJ Open
Publisher
BMJ
Published
2020-06
Volume
10
Issue
6
Pages
e036453
ISSN
2044-6055 (ISSN-L)
OpenAlex
W3031326207
Cited by
147
References
72
Citations by year
2026: 19 · 2025: 29 · 2024: 31 · 2023: 18 · 2022: 26 · 2021: 24
FWCI
3.24
Type
journal article
Language
en
Open access
Locations
  • BMJ Open (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • HighWire Press Open Archive (acceptedVersion)
Funders
National Health and Medical Research Council (9100002, APP1176620)
Topics
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
  • Health Sciences › Health Professions › Health Information Management › Healthcare Quality and Management
Keywords
sustainability, health interventions, program evaluation, definition of sustainability, sustainability measurement, theoretical framework, implementation science, health care system, service improvement
MeSH terms
Chronic Disease, Delivery of Health Care, Health Services, Humans, Program Evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review