Navigating the sustainability landscape: a systematic review of sustainability approaches in healthcare

In the paper

“Although multiple definitions were found (continuation of the health benefits from an initiative, capacity built in the workforce or community, further development or adaptation and the ability to recover costs), there was a clear dependence on this one measure which has been previously observed in the literature [76].”

What the sources say

• Supported

  The Discussion (page not available in our copy) explicitly identifies a trend toward using maintenance of programme activities as the primary sustainability outcome. It also notes that only about one-quarter of studies measured patient-level health outcomes as a primary outcome, supporting the citing sentence’s contrast between multiple definitions and reliance on this measure. (LLM, conf=0.91)

  cites [76] Francis L, Dunt D, Cadilhac DA. How is the sustainability of chronic disease health programmes empirically measured in hospital and related healthcare services?-a scoping review. BMJ Open. 2016;6:e010944. source

In the paper

“A smaller number of approaches were designed for use in acute, 3% (2/62), and e-health settings, 3% (2/62).”

In the paper

“This was followed by 26% (16/62) using ‘professional expertise’ such as an advisory panel and 24% (15/62) using interviews.”

In the paper

“Public health settings were specified in 31% (19/62) of the approaches, followed by community healthcare in 26% of the approaches (16/62).”

In the paper

“Six constructs were included in over 75% of the approaches: ‘General resources’ (90%), ‘Demonstrating effectiveness’ (89%), ‘Monitoring progress over time’ (84%), ‘Stakeholder participation’ (79%), ‘Integration with existing programs and policies’ (79%) and ‘Training and capacity building’ (76%).”

In the paper

“Although 37% (23/62) did not have an explicit link to theory, 15 different theories were identified within the other approaches.”

In the paper

“The majority of these approaches have been designed for use by multiple groups of professionals or practitioners (e.g. researchers as well as nurses).”

In the paper

“5.Recovering costs (included in 3% (2/62))”

In the paper

“Public health settings were specified in 31% (19/62) of the approaches, followed by community healthcare in 26% of the approaches (16/62).”

In the paper

“Suggested users were specified in 55% (34/62) of the approaches (Fig. 4).”

In the paper

“Nine approaches viewed sustainability as a linear process with sustainability being studied retrospectively after implementation has been ‘completed’.”

In the paper

“Continued programme activities (included in 86% (53/62) of the approaches)”

In the paper

“Sixty-one percent (38/62) of the development processes included a literature review or systematic review.”

In the paper

“While numerous theories were found, 4 theories were common across multiple sustainability approaches covering 45% of papers: diffusion of innovations theory, complexity theory, ecological theory and open systems theory.”

In the paper

“This was followed by 26% (16/62) using ‘professional expertise’ such as an advisory panel and 24% (15/62) using interviews.”

In the paper

“The highest proportion of approaches, 39% (24/62), aimed to evaluate sustainability, followed by 23% (14/62) of the approaches which aimed to support planning for sustainability.”

In the paper

“Eleven approaches aimed to examine sustainability at an organisational or systems level (e.g. a long-term care organisation) [32].”

What the sources say

• Not checked yetcites [32] Slaghuis SS, Strating MMH, Bal RA, Nieboer AP. A framework and a measurement instrument for sustainability of work practices in long-term care. BMC Health Serv Res. 2011;11:314. BioMed Central Ltd source

In the paper

“The highest proportion, 66% (41/62) of approaches, viewed the sustainability as a prospective process to be explored throughout implementation.”

In the paper

“Thirty-seven percent (23/62) of the approaches were designed for use in general healthcare settings and did not specify a specific healthcare setting for use.”

In the paper

“Definitions for sustainability were explicitly stated in 76% (47/62) of approaches and implicitly deduced from the remaining 24%.”

In the paper

“The sustainability approaches were developed through several techniques often using a mixed-method approach (e.g. literature review and interviews) (Fig. 3).”

In the paper

“The highest proportion of approaches, 39% (24/62), aimed to evaluate sustainability, followed by 23% (14/62) of the approaches which aimed to support planning for sustainability.”

In the paper

“The remaining 12 approaches specified they could be used both prospectively and retrospectively, during implementation or following implementation.”

In the paper

“Many approaches highlighted the importance of the ability of an initiative to garner and maintain resources [15, 27, 36–41] through stable sources [19, 36, 39, 42–45].”

What the sources say

• Not checked yetcites [15] Shigayeva A, Coker RJ. Communicable disease control programmes and health systems: an analytical approach to sustainability. Health Policy Plan. 2015;30:368–85. source
• Not checked yetcites [27] Sivaram S, Celentano DD. Training outreach workers for AIDS prevention in rural India: is it sustainable? Health Policy Plan. 2003;18:411–20. source
• Not checked yetcites [36] Luke D a, Calhoun A, Robichaux CB, Elliott MB, Moreland-Russell S. The program sustainability assessment tool: a new instrument for public health programs. Prev Chronic Dis. 2014;11:130184. source
• Not checked yetcites [37] Mancini JA, Marek LI. Sustaining community-based programs for families: conceptualization and measurement. Fam Relat. 2004;53:339–47. source
• Not checked yetcites [38] Parand A, Benn J, Burnett S, Pinto A, Vincent C. Strategies for sustaining a quality improvement collaborative and its patient safety gains. Int J Qual Heal. Care. (C) International Society for Quality in Health Care and Oxford University Press 2012. Published by Oxford University Press. All rights reserved.: Department of Surgery and Cancer, Imperial College London, Rm 503, 5th Floor Wright Fleming Building, St Mary’s Campus, Norfolk Place, London W2 1PG, UK; 2012. p. 380–90. source
• Not checked yetcites [39] Nelson DE, Reynolds JH, Luke D a, Mueller NB, Eischen MH, Jordan J, et al. Successfully maintaining program funding during trying times: lessons from tobacco control programs in five states. J Public Health Manag Pract. 2007;13:612–20. source
• Not checked yetcites [40] Tuyet Hanh TT, Hill PS, Kay BH, Tran MQ. Development of a framework for evaluating the sustainability of community-based dengue control projects. Am J Trop Med Hyg. 2009;80:312–8. source
• Not checked yetcites [41] Stefanini A, Ruck N. Managing externally-assisted health projects for sustainability in developing countries. Int J Health Plan Manag. 1992;7:199–210. source
• Not checked yetcites [19] Johnson K, Hays C, Center H, Daley C. Building capacity and sustainable prevention innovations: a sustainability planning model. Eval Program Plann. 2004;27:135–49. source
• Not checked yetcites [42] Edwards JC, Feldman PH, Sangl J, Polakoff D, Stern G, Casey D. Sustainability of partnership projects: a conceptual framework and checklist. Jt Comm J Qual Patient Saf. 2007;33:37–47. source

In the paper

“The constructs have been organised under the following six emergent themes: the initiative design and delivery, negotiating initiative processes, the people involved, resources, the external environment and the organisational setting.”

In the paper

“From these results, a consolidated framework for sustainability constructs in healthcare was developed.”

In the paper

“Although 37% (23/62) did not have an explicit link to theory, 15 different theories were identified within the other approaches.”

In the paper

“2.Continued health benefits (included in 44% (27/62))”

In the paper

“The highest proportion identified themselves as frameworks.”

In the paper

“The majority, 82% (51/62), of approaches have been designed to examine or influence sustainability at a specific intervention or programme level (e.g. a single improvement project) [31].”

What the sources say

• Not checked yetcites [31] Sarriot EG, Winch PJ, Ryan LJ, Bowie J, Kouletio M, Swedberg E, et al. A methodological approach and framework for sustainability assessment in NGO-implemented primary health care programs. Int J Health Plann Manag. 2004:23–41. source

In the paper

“4.Further development (adaptation) (included in 16% (10/62))”

In the paper

“3.Capacity built (included in 19% (12/62))”

In the paper

“Definitions for sustainability were explicitly stated in 76% (47/62) of approaches and implicitly deduced from the remaining 24%.”

In the paper

“In total, 62 publications identifying a sustainability approach were included in this review (32 frameworks, 16 models, 8 tools, 4 strategies, 1 checklist and 1 process).”

• Also stated in: § Results · plain

  “In total, 62 papers which identified sustainability approaches were identified for inclusion in this review Fig. 1.”

In the paper

“Although variation was seen, results show that there are consistent constructs across approaches regardless of proposed interventions, settings or application types.”

In the paper

“Although common constructs were found across approaches, each approach reported diverse means to investigating and defining individual constructs.”

In the paper

“Prospective approaches are used for a combination of planning, guidance and evaluation.”

In the paper

“A selection of approaches took a wider perspective and looked at whether the initiative benefits were perceived by staff and other stakeholders as valuable [8, 30, 42, 51, 61-63].”

What the sources say

• Not checked yetcites [8] Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ Res. 1998;13:87–108. source
• Not checked yetcites [30] Schalock RL, Verdugo M, Lee T. A systematic approach to an organization’s sustainability. Eval Program Plann. 2016;56:56–63. source
• Not checked yetcites [42] Edwards JC, Feldman PH, Sangl J, Polakoff D, Stern G, Casey D. Sustainability of partnership projects: a conceptual framework and checklist. Jt Comm J Qual Patient Saf. 2007;33:37–47. source
• Not checked yetcites [51] May C. A rational model for assessing and evaluating complex interventions in health care. BMC Health Serv Res. 2006;6:11. source
• Not checked yetcites [61] Maher L, Gustafson D, Evans A. Sustainability model and guide. Coventry: NHS Institute for Innovation and Improvement; 2010.
• Not checked yetcites [62] Alexander JA, Weiner BJ, Metzger ME, Shortell SM, Bazzoli GJ, Hasnain-Wynia R, et al. Sustainability of collaborative capacity in community health partnerships. Med Care Res Rev. 2003;60:130S–60S. source
• Not checked yetcites [63] Dauphinee WD, Dauphinee WD, Reznick R, Reznick R. A framework for designing, implementing, and sustaining a national simulation network. Simul Healthc. 2011;6:94–100. source

In the paper

“Monitoring progress over-time (the ability to monitor the initiative using standardised systems or mechanisms over-time) appeared in 84% of the approaches.”

In the paper

“Approaches assessing an intervention’s sustainability emphasised the need to consider how an initiative becomes integrated into current programmes and policies specifically looking what intervention adaption may be needed.”

In the paper

“Approach characteristics were wide-ranging with diverse settings, interventions and designs.”

In the paper

“These approaches were also more likely to assess how training and capacity building were conducted to ensure staff were able to undertake the initiative tasks.”

In the paper

“Within the six constructs included in the majority of approaches, diverse views and different assessment mechanisms were taken, highlighting the complexity within each construct.”

In the paper

“Prospective approaches show a greater emphasis on building relationships and getting stakeholder buy-in throughout an initiative.”

In the paper

“Results have demonstrated that sustainability is most often defined and assessed as the maintenance of programme activities.”

In the paper

“These approaches also highlighted the role of initiative adaptation to ensure initiatives align with stakeholder and setting needs.”

In the paper

“Our comparison across approaches demonstrated consistent constructs were seen regardless of proposed interventions, setting or level of application.”

In the paper

“The ability of an initiative to share resources with partners and other organisations [41, 46], seek out alternative and supplemental resources [18, 47–49] and/or uncover multiple funding sources [8, 36, 49, 50] were also highlighted across some approaches as important to overall sustainability.”

What the sources say

• Not checked yetcites [41] Stefanini A, Ruck N. Managing externally-assisted health projects for sustainability in developing countries. Int J Health Plan Manag. 1992;7:199–210. source
• Not checked yetcites [46] Dorsey SG, Schiffman R, Redeker NS, Heitkemper M, McCloskey DJ, Weglicki LS, et al. National Institute of Nursing Research Centers of Excellence: a logic model for sustainability, leveraging resources, and collaboration to accelerate cross-disciplinary science. Nurs Outlook U S. 2014;62:384–93. source
• Not checked yetcites [18] Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101:2059–67. source
• Not checked yetcites [47] Ford JH, Alagoz E, Dinauer S, Johnson KA, Pe-Romashko K, Gustafson DH. Successful organizational strategies to sustain use of A-CHESS: a mobile intervention for individuals with alcohol use disorders. J Med Internet Res. 2015;17:e201. source
• Not checked yetcites [48] Roy M, Czaicki N, Holmes C, Chavan S, Tsitsi A, Odeny T, et al. Understanding sustained retention in HIV/AIDS care and treatment: a synthetic review. Curr HIV/AIDS Rep. 2016;13:177–85. source
• Not checked yetcites [49] Savaya R, Elsworth G, Rogers P. Projected sustainability of innovative social programs. Eval Rev. 2009;33:189–205. source
• Not checked yetcites [8] Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ Res. 1998;13:87–108. source
• Not checked yetcites [36] Luke D a, Calhoun A, Robichaux CB, Elliott MB, Moreland-Russell S. The program sustainability assessment tool: a new instrument for public health programs. Prev Chronic Dis. 2014;11:130184. source
• Not checked yetcites [50] Song B, Sun Q, Li Y, Que C. Evaluating the sustainability of community-based long-term care programmes: a hybrid multi-criteria decision making approach. Sustainability. 2016;8:1–19.

In the paper

“Retrospective approaches were more often designed for evaluation purposes emphasising the need for a shared vision and accountability to deliver the initiative.”

In the paper

“Regardless of level of use, 5 of the top 10 constructs are found across both types of approaches.”

In the paper

“Approaches assessing organisational sustainability were also much more likely to prioritise defining overall aims for the programme and garnering belief in initiatives from stakeholders.”

In the paper

“many approaches (24%) were informed by professional expertise”

In the paper

“with six constructs included in over 75% of approaches”

In the paper

“With little overarching direction for this area of research, new definitions, factors and methods for assessing sustainability have been produced by individual studies [18].”

What the sources say

• Not checked yetcites [18] Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101:2059–67. source

In the paper

“Large variation in the practices and care can be seen across similar services when initiatives which initially demonstrate improved patient outcomes fail to maintain their gains [7, 8].”

What the sources say

• Not checked yetcites [7] Gruen RL, Elliott JH, Nolan ML, Lawton PD, Parkhill A, McLaren CJ, et al. Sustainability science: an integrated approach for health-programme planning. Lancet. 2008;372:1579–89. Elsevier Ltd source
• Not checked yetcites [8] Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ Res. 1998;13:87–108. source

In the paper

“Further complicating this research area is the lack of consensus on how to define sustainability.”

What the sources say

• Not checked yetcites [10] Martin GP, Weaver S, Currie G, Finn R, Mcdonald R. Innovation sustainability in challenging health-care contexts: embedding clinically led change in routine practice. Heal Serv Manage Res. 2012;25:190–9. source
• Not checked yetcites [13] Pluye P, Potvin L, Denis J-L. Making public health programs last: conceptualizing sustainability. Eval Program Plann. 2004;27:121–33. source

In the paper

“This has also been shown to be detrimental to improvement efforts in general as staff, patient and public opinion of improvement initiatives declines and enthusiasm for engaging in future programmes is lost [9, 10].”

What the sources say

• Not checked yetcites [9] Hovlid E, Bukve O, Haug K, Aslaksen AB, von Plessen C. Sustainability of healthcare improvement: what can we learn from learning theory? BMC Health Serv Res. 2012;12:235. source
• Not checked yetcites [10] Martin GP, Weaver S, Currie G, Finn R, Mcdonald R. Innovation sustainability in challenging health-care contexts: embedding clinically led change in routine practice. Heal Serv Manage Res. 2012;25:190–9. source

In the paper

“With no clear consensus on how to define or influence sustainability, many researchers and healthcare practitioners have developed frameworks, models and tools to support and monitor sustainability in healthcare settings [12, 18].”

What the sources say

• Not checked yetcites [12] Greenhalgh T, Robert G, Bate P, Kyriakidou O, Macfarlane F, Peacock R. How to spread good ideas: a systematic review of the literature on diffusion, dissemination and sustainability of innovations in health service delivery and organisation. 2004.
• Not checked yetcites [18] Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101:2059–67. source

In the paper

“Similar results were found in a review on the continuation of programme activities where only 60% of sites reported sustaining at least one programme component [5].”

What the sources say

• Not checked yetcites [5] Scheirer MA. Is sustainability possible? A review and commentary on empirical studies of program sustainability. Am J Eval. 2005;26:320–47. source

In the paper

“Initiatives that fail to sustain are extremely wasteful of human and monetary investments [7, 8].”

What the sources say

• Not checked yetcites [7] Gruen RL, Elliott JH, Nolan ML, Lawton PD, Parkhill A, McLaren CJ, et al. Sustainability science: an integrated approach for health-programme planning. Lancet. 2008;372:1579–89. Elsevier Ltd source
• Not checked yetcites [8] Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ Res. 1998;13:87–108. source

In the paper

“Existing quality assessments were explored, but it has been noted that available quality assessment approaches often fail to consider the rationale and context of studies [22, 23].”

What the sources say

• Not checked yetcites [22] Barbour RS. Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? BMJ Br Med J. 2001;322:1115–7. source
• Not checked yetcites [23] Li LC, Grimshaw JM, Nielsen C, Judd M, Coyte PC, Graham ID. Use of communities of practice in business and health care sectors: a systematic review. Implement Sci. 2009;4:27. source

In the paper

“Choosing a sustainability method can pose a challenge because of the diverse approaches reported in the literature.”

In the paper

“Our exploration indicates there is very little consensus between approaches on what constitutes a ‘framework’, ‘model’ or ‘tool’.”

In the paper

“The test showed a high level of agreement between scorers with an inter-rater reliability score of 0.94 [34, 35].”

What the sources say

• Not checked yetcites [34] Viera AJ, Garrett JM. Understanding interobserver agreement: Fam. Med. 2005;37:360–3.
• Not checked yetcites [35] Nvivo. Run a coding comparison query. Explor. your coding using queries. 2017. p. 1.

In the paper

“The available assessments were not sensitive to the aim of our study which was exploratory in nature.”

In the paper

“Despite recognition of this challenge and considerable research conducted in this area, relatively little is known about how to translate this evidence into action to support the long-term impact of improvement efforts [12].”

What the sources say

• Not checked yetcites [12] Greenhalgh T, Robert G, Bate P, Kyriakidou O, Macfarlane F, Peacock R. How to spread good ideas: a systematic review of the literature on diffusion, dissemination and sustainability of innovations in health service delivery and organisation. 2004.

In the paper

“As this review focused on published sustainability approaches, publication bias may have affected the results of this study.”

In the paper

“Sustaining worthwhile changes poses a challenge to those undertaking an improvement initiative.”

In the paper

“Differences between these types of approaches demonstrate how the ‘level of use’ of an approach changes the potential constructs to be explored (shown in italics).”

In the paper

“It is important to note that while measuring continuation of programme activities is important to assessing sustainability, relying solely on this measure may risk other key sustainability variables being missed [18, 76].”

What the sources say

• Not checked yetcites [18] Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101:2059–67. source
• Not checked yetcites [76] Francis L, Dunt D, Cadilhac DA. How is the sustainability of chronic disease health programmes empirically measured in hospital and related healthcare services?-a scoping review. BMJ Open. 2016;6:e010944. source

In the paper

“The reviewed sustainability approaches made connections to many different theoretical perspectives which highlighted the complexity of measuring and planning for sustainable initiatives.”

In the paper

“This was observed in the Drug Assistance Resistance Education programme in America which continued to be implemented in schools despite studies showing that it had little effect on prevention or reduction of drug use by students [77].”

What the sources say

• Not checked yetcites [77] Frumkin P, Reingold D. Evaluation research and institutional pressures: challenges in public-nonprofit contracting. 2004. Report No.: 23.

In the paper

“Using continuation of programme activities as the sole measure of sustainability also risks initiative being unfairly judged as failing to sustain if activities are adapted.”

In the paper

“For example, it may result in the continuation of ineffective or undesirable practices if health benefits are not taken into account.”

In the paper

“This may have resulted in bias in inclusion or exclusion or resulted in missing or erroneous information being collected.”

In the paper

“The use of one author to conduct of the majority of screening, data extraction and coding is also a limitation of this work.”

In the paper

“indicating general learning can be gathered from across settings to inform sustainability processes and research.”

In the paper

“sustainability approaches which have until now remained isolated across healthcare fields and disciplines [18]”

What the sources say

• Not checked yetcites [18] Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101:2059–67. source

In the paper

“These areas tend to dominate this area of research”

What the sources say

• Not checked yetcites [79] Fox A, Gardner G, Osborne S. A theoretical framework to support research of health service innovation. Aust Health Rev. 2014;39:70–5. source

In the paper

“Another limitation of this work is the disproportionate number of frameworks from the community health and public health settings.”