Systematic, Multimethod Assessment of Adaptations Across Four Diverse Health Systems Interventions
Open accessAbstract
Background Many health outcomes and implementation science studies have demonstrated the importance of tailoring evidence-based care interventions to local context to improve fit. By adapting to local culture, history, resources, characteristics, and priorities, interventions are more likely to lead to improved outcomes. However, it is unclear how best to adapt evidence-based programs and promising innovations. There are few guides or examples of how to best categorize or assess healthcare adaptations, and even fewer that are brief and practical for use by non-researchers. Materials and methods This paper describes the importance and potential of assessing adaptations before, during, and after the implementation of health systems interventions. We present a promising multi-level and multi-method approach developed and being applied across four different health systems interventions. Finally, we discuss implications and opportunities for future research. Results The four case studies are diverse in the conditions addressed, interventions, and implementation strategies. They include two nurse coordinator-based transition of care interventions, a data and training-driven multimodal pain management project, and a cardiovascular patient-reported outcomes project, all of which are using audit and feedback. We used the same modified adaptation framework to document changes made to the interventions and implementation strategies. To create the modified framework, we started with the adaptation and modification model developed by Stirman and colleagues and expanded it by adding concepts from the RE-AIM framework. Our assessments address the intuitive domains of Who, How, When, What, and Why to classify and organize adaptations. For each case study, we discuss how the modified framework was operationalized, the multiple methods used to collect data, results to date and approaches utilized for data analysis. These methods include a real-time tracking system and structured interviews at key times during the intervention. We provide descriptive data on the types and categories of adaptations made and discuss lessons learned. Conclusions The multi-method approaches demonstrate utility across diverse health systems interventions. The modified adaptations model adequately captures adaptations across the various projects and content areas. We recommend systematic documentation of adaptations in future clinical and public health research, and have made our assessment materials publicly available.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Other aspects of reporting that may be important for improving understanding of modification and its impact, such as when in the implementation process the modification was made, were also not included in the prior frameworks.
“Further, we recognized that other aspects of reporting that may be important for improving understanding of modification and its impact, such as when in the implementation process the modification was made, were also not included [28–31].”
From The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions (2019) · cites this paper as [28]
The Methods > Adaptation Framework section says the original Stirman framework was expanded to include when adaptations were made. But the framework presented in this cited paper itself includes a WHEN category for the stage of the project at which an adaptation occurred. Thus, this source does not support the claim that this aspect was absent from the prior frameworks cited; it documents one that included it. (LLM, conf=0.87)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The RE-AIM framework: a systematic review of use over time
Gaglio, B., Shoup, J. A., & Glasgow, R. E. (2013) · American Journal of Public Health · doi:10.2105/ajph.2013.301299 · Cited by 1,029 · see citing works
Self-citation“Scaling-out” evidence-based interventions to new populations or new health care delivery systems
Aarons, G. A., Sklar, M., Mustanski, B., Benbow, N., & Brown, C. H. (2017) · Implementation Science · doi:10.1186/s13012-017-0640-6 · Cited by 399 · see citing works
Dissemination and Implementation Research in Health
(2017) · Oxford Scholarship Online · doi:10.1093/oso/9780190683214.001.0001 · Cited by 396 · see citing works
Glasgow, R. E., & Riley, W. T. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.03.010 · Cited by 318 · see citing works
Self-citationLewis, C. C., Stanick, C. F., Martinez, R. G., Weiner, B. J., Kim, M., Barwick, M., & Comtois, K. A. (2016) · Implementation Science · doi:10.1186/s13012-014-0193-x · Cited by 155 · see citing works
Adaptation in Dissemination and Implementation Science
Baumann, A. A., Cabassa, L. J., & Stirman, S. W. (2017) · Oxford Scholarship Online · doi:10.1093/oso/9780190683214.003.0017 · Cited by 151 · see citing works
Demakis, J. G., McQueen, L., Kizer, K. W., & Feussner, J. R. (2000) · Medical Care · doi:10.1097/00005650-200006001-00003 · Cited by 131 · see citing works
Kilbourne, A. M., Almirall, D., Eisenberg, D., Waxmonsky, J., Goodrich, D. E., Fortney, J. C., Kirchner, J. E., Solberg, L. I., Main, D., Bauer, M. S., et al. (2014) · Implementation Science · doi:10.1186/s13012-014-0132-x · Cited by 130 · see citing works
Advancing the application, quality and harmonization of implementation science measures
Rabin, B. A., Purcell, P., Naveed, S., Moser, R. P., Henton, M. D., Proctor, E. K., Brownson, R. C., & Glasgow, R. E. (2012) · Implementation Science · doi:10.1186/1748-5908-7-119 · Cited by 106 · see citing works
Self-citationMixed Methods Evaluation in Dissemination and Implementation Science
Palinkas, L. A., & Cooper, B. R. (2017) · Oxford Scholarship Online · doi:10.1093/oso/9780190683214.003.0020 · Cited by 42 · see citing works
Jones Rhodes, W. C., Ritzwoller, D. P., & Glasgow, R. E. (2018) · Translational Behavioral Medicine · doi:10.1093/tbm/ibx003 · Cited by 31 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 22
- [1]Bauman LJ Stein RE Ireys HT Reinventing fidelity: The transfer of social technology among settings. Am J Community Psychol (1991) 19(4): 619– 39. 1755439
- [3]Chambers DA Norton WE. The adaptome: advancing the science of intervention adaptation. Am J Prev Med (2016) 51(4 Suppl 2): S124– 31. 10.1016/j.amepre.2016.05.011 27371105 PMC5030159 · DOI · Paper page
- [4]Stirman SW Miller CJ Toder K Calloway A. Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implement Sci (2013) 8: 65. 10.1186/1748-5908-8-65 23758995 PMC3686699 · DOI · Paper page
- [5]Castro FG Barrera M Jr Martinez CR Jr. The cultural adaptation of prevention interventions: resolving tensions between fidelity and fit. Prev Sci (2004) 5(1): 41– 5. 10.1023/B:PREV.0000013980.12412.cd 15058911 · DOI · Paper page
- [8]Kilbourne AM Elwy AR Sales AE Atkins D. Accelerating research impact in a learning health care system: VA’s quality enhancement research initiative in the choice act era. Med Care (2017) 55(Suppl 7 Suppl 1): S4– 12. 10.1097/MLR.0000000000000683 27997456 PMC5472006 · DOI
- [9]Bradley SM Rumsfeld JS Ho PM Incorporating health status in routine care to improve health care value: the VA patient reported health status assessment (PROST) system. JAMA (2016) 316(5): 487– 8. 10.1001/jama.2016.6495 27483062 · DOI
- [11]Ayele RA Lawrence E McCreight M Fehling K Peterson J Glasgow RE Study protocol: improving the transition of care from a non-network hospital back to the patient’s medical home. BMC Health Serv Res (2017) 17(1): 123. 10.1186/s12913-017-2048-z 28183346 PMC5301366 · DOI
- [12]Leonard C Lawrence E McCreight M Lippmann B Kelley L Mayberry A Implementation and dissemination of a transition of care program for rural veterans: a controlled before and after study. Implement Sci (2017) 12(1): 123. 10.1186/s13012-017-0653-1 29058640 PMC5651587 · DOI
- [13]Hall TL Holtrop JS Dickinson LM Glasgow RE. Understanding adaptations to patient-centered medical home activities: the PCMH adaptations model. Transl Behav Med (2017) 7(4): 861– 72. 10.1007/s13142-017-0511-3 28620725 PMC5684081 · DOI
- [14]Glasgow RE Klesges LM Dzewaltowski DA Estabrooks PA Vogt TM. Evaluating the impact of health promotion programs: using the RE-AIM framework to form summary measures for decision making involving complex issues. Health Educ Res (2006) 21(5): 688– 94. 10.1093/her/cyl081 16945984 · DOI · Paper page
- [16]Glasgow RE Chambers DA Cynkin L News from the NIH: highlights in implementation science from the National Cancer Institute and the National Institute of Mental Health. Transl Behav Med (2013) 3(4): 335– 7. 10.1007/s13142-013-0218-z 24294320 PMC3830018 · DOI
- [17]Glasgow RE What does it mean to be pragmatic? Opportunities and challenges for pragmatic approaches. Health Educ Behav (2013) 40(3): 257– 65. 10.1177/1090198113486805 23709579 · DOI
- [20]Glasgow RE Kessler RS Ory MG Roby D Gorin SS Krist A. Conducting rapid, relevant research: lessons learned from the My Own Health Report project. Am J Prev Med (2014) 47(2): 212– 9. 10.1016/j.amepre.2014.03.007 24953520 PMC4609529 · DOI
- [21]Rabin BA Lewis CC Norton WE Neta G Chambers D Tobin JN Measurement resources for dissemination and implementation research in health. Implement Sci (2016) 11: 42. 10.1186/s13012-016-0401-y 27000147 PMC4802882 · DOI · Paper page
- [23]Powell BJ Stanick CF Halko HM Dorsey CN Weiner BJ Barwick MA Toward criteria for pragmatic measurement in implementation research and practice: a stakeholder-driven approach using concept mapping. Implement Sci (2017) 12(1): 118. 10.1186/s13012-017-0649-x 28974248 PMC5627503 · DOI · Paper page
- [24]Ovretveit J Mittman B Rubenstein L Ganz DA. Using implementation tools to design and conduct quality improvement projects for faster and more effective improvement. Int J Health Care Qual Assur (2017) 30(8): 755– 68. 10.1108/IJHCQA-01-2017-0019 28958203 · DOI
- [25]Gillman MW Hammond RA Precision treatment and precision prevention: integrating “below and above the skin”. JAMA Pediatr (2016) 170(1): 9– 10. 10.1001/jamapediatrics.2015.2786 26595371 PMC4705446 · DOI · Paper page
- [27]Riley WT Rivera DE Methodologies for optimizing behavioral interventions: introduction to special section. Transl Behav Med (2014) 4(3): 234– 7. 10.1007/s13142-014-0281-0 25264463 PMC4167896 · DOI
- [28]Kirchner JE Waltz TJ Powell BJ Smith JL Proctor EK Implementation strategies. In: Brownson RC Colditz G Proctor EK, editors. Dissemination and Implementation Research in Health. New York: Oxford University Press (2017). p. 245– 66.
- [30]Gaglio B Glasgow RE Evaluation approaches for dissemination and implementation research. In: Brownson RC Colditz G Proctor EK, editors. Dissemination and Implementation Research in Health. New York: Oxford University Press (2017). p. 317– 34.
- [31]Cresswell JW Qualitative Inquiry and Research Design: Choosing Among Five Approaches. 2nd ed Thousand Oaks, CA: SAGE (2007).
- [33]Holtrop JS Rabin BA Glasgow RE. Qualitative approaches to use of the RE-AIM framework: rationale and methods. BMC Health Serv Res (2018) 18: 177. 10.1186/s12913-018-2938-8 29534729 PMC5851243 · DOI
Cited by
Sorted by citations
RE-AIM Planning and Evaluation Framework: Adapting to New Science and Practice With a 20-Year Review
Glasgow, R. E., Harden, S. M., Gaglio, B., Rabin, B., Smith, M. L., Porter, G. C., Ory, M. G., & Estabrooks, P. A. (2019) · Frontiers in Public Health · doi:10.3389/fpubh.2019.00064 · Cited by 2,528 · see citing works
Self-citationWiltsey Stirman, S., Baumann, A. A., & Miller, C. J. (2019) · Implementation Science · doi:10.1186/s13012-019-0898-y · Cited by 1,329 · see citing works
“implementation or health outcomes [20].”
MethodInfluentialUnderstanding and applying the RE-AIM framework: Clarifications and resources
Holtrop, J. S., Estabrooks, P. A., Gaglio, B., Harden, S. M., Kessler, R. S., King, D. K., Kwan, B. M., Ory, M. G., Rabin, B. A., Shelton, R. C., et al. (2021) · Journal of Clinical and Translational Science · doi:10.1017/cts.2021.789 · Cited by 549 · see citing works
Self-citationvon Thiele Schwarz, U., Aarons, G. A., & Hasson, H. (2019) · BMC Health Services Research · doi:10.1186/s12913-019-4668-y · Cited by 134 · see citing works
Reilly, K. L., Kennedy, S., Porter, G., & Estabrooks, P. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.00430 · Cited by 102 · see citing works
Estimating Costs of an Implementation Intervention
Wagner, T. H., Yoon, J., Jacobs, J. C., So, A., Kilbourne, A. M., Yu, W., & Goodrich, D. E. (2020) · Medical Decision Making · doi:10.1177/0272989x20960455 · Cited by 59 · see citing works
- Authors
- Rabin, B. A., McCreight, M., Battaglia, C., Ayele, R., Burke, R. E., Hess, P. L., Frank, J. W., & Glasgow, R. E.
- Affiliations
- University of California San Diego (US)
- Denver VA Medical Center (US)
- Colorado School of Public Health (US)
- Corresponding author
- Rabin, Borsika A.
- ORCID
- Venue
- Frontiers in Public Health
- Publisher
- Frontiers Media SA
- Published
- 2018-04-09
- Volume
- 6
- Pages
- 102-102
- ISSN
- 2296-2565 (ISSN-L)
- PMC
- PMC5900443
- OpenAlex
- W2797788286
- Cited by
- 147
- References
- 11
- Citations by year
- 2026: 16 · 2025: 12 · 2024: 15 · 2023: 28 · 2022: 24 · 2021: 24 · 2020: 16 · 2019: 9 · 2018: 2 · 2017: 1
- FWCI
- 19.47
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Frontiers in Public Health (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Quality Enhancement Research Initiative (Triple Aim QUERI Program, QUERI 15-268)
- U.S. Department of Veterans Affairs (Office of Rural Health in partnership with the Office of Nursing Services through the Enterprise-Wide Initiative (EWI) (Rural Transition Project))
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- intervention adaptation, health system interventions, adaptation assessment, implementation science, adaptation framework, RE-AIM framework, audit and feedback, care transitions, nurse coordinator, multimodal pain management, real-time tracking system, implementation strategies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review