Development of a framework and coding system for modifications and adaptations of evidence-based interventions
Open accessThe DOI matches its Crossref record; 59 of the 60 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Evidence-based interventions are frequently modified or adapted during the implementation process. Changes may be made to protocols to meet the needs of the target population or address differences between the context in which the intervention was originally designed and the one into which it is implemented [Addict Behav 2011, 36(6):630-635]. However, whether modification compromises or enhances the desired benefits of the intervention is not well understood. A challenge to understanding the impact of specific types of modifications is a lack of attention to characterizing the different types of changes that may occur. A system for classifying the types of modifications that are made when interventions and programs are implemented can facilitate efforts to understand the nature of modifications that are made in particular contexts as well as the impact of these modifications on outcomes of interest. METHODS: We developed a system for classifying modifications made to interventions and programs across a variety of fields and settings. We then coded 258 modifications identified in 32 published articles that described interventions implemented in routine care or community settings. RESULTS: We identified modifications made to the content of interventions, as well as to the context in which interventions are delivered. We identified 12 different types of content modifications, and our coding scheme also included ratings for the level at which these modifications were made (ranging from the individual patient level up to a hospital network or community). We identified five types of contextual modifications (changes to the format, setting, or patient population that do not in and of themselves alter the actual content of the intervention). We also developed codes to indicate who made the modifications and identified a smaller subset of modifications made to the ways that training or evaluations occur when evidence-based interventions are implemented. Rater agreement analyses indicated that the coding scheme can be used to reliably classify modifications described in research articles without overly burdensome training. CONCLUSIONS: This coding system can complement research on fidelity and may advance research with the goal of understanding the impact of modifications made when evidence-based interventions are implemented. Such findings can further inform efforts to implement such interventions while preserving desired levels of program or intervention effectiveness.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“which commonly involve a series of non-systematic adaptations (Stirman, Miller, Toder, & Calloway, 2013).”
From User‐centered design for psychosocial intervention development and implementation. (2016) · cites this paper as Stirman, Miller, Toder, & Calloway, 2013
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Reported influences on sustainability include characteristics of the innovation and the individuals involved, leadership, capacity, processes, and context [2, 9, 13, 14].”
From Factors influencing sustainability and scale-up of rural primary healthcare memory clinics: perspectives of clinic team members (2022) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For example, while the 2013 and 2017 frameworks [15, 16] laid out distinct forms of modification that could be made to the content or mode of delivery, they did not specify when those modifications were designed to address important differences between the original population and the stakeholders in the current implementation effort (i.e., cultural adaptations).”
From The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions (2019) · cites this paper as [16]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“More work is needed to determine what types of adaptation may specifically enhance sustainability and what types may erode the benefits or delivery of the intervention over time (135, 137).”
From The Sustainability of Evidence-Based Interventions and Practices in Public Health and Health Care (2018) · cites this paper as [137]
The cited source says what the claim attributes to it.
In the citing paper
“However, new approaches to identifying and coding EBI modifications and adaptations are promising in regard to rigorous study of adaptations and their impacts [63].”
From “Scaling-out” evidence-based interventions to new populations or new health care delivery systems (2017) · cites this paper as [63]
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148 · cited 2×
“stantial agreement’ and ‘almost perfect’ agreement for the level and nature of modifications, respectively [52].”
BackgroundThe Discovery of Grounded Theory: Strategies for Qualitative Research.
Wagner, H. R., Glaser, B. G., & Strauss, A. L. (1968) · Social Forces · doi:10.2307/2575405 · Cited by 46,361
The Discovery of Grounded Theory; Strategies for Qualitative Research
Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968) · Nursing Research · doi:10.1097/00006199-196807000-00014 · Cited by 39,449
(1998) · Sage eBooks · Cited by 38,631
(1990) · Cited by 11,110
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736 · cited 1×
“The two raters overlapped on 20% of the identified segments, and we computed Cohen’s kappa coefficients [31] to determine interrater reliability for each rating category.”
MethodDusenbury, L. (2003) · Health Education Research · doi:10.1093/her/18.2.237 · Cited by 1,671 · cited 1×
“cessary [4,6-10], few studies have examined the impact”
BackgroundWiltsey Stirman, S., Kimberly, J., Cook, N., Calloway, A., Castro, F., & Charns, M. (2012) · Implementation Science · doi:10.1186/1748-5908-7-17 · Cited by 1,355 · cited 3×
“The tension between modification and fidelity is a critical issue in implementation science [4,54,55].”
BackgroundSelf-citationThe Cultural Adaptation of Prevention Interventions: Resolving Tensions Between Fidelity and Fit
Castro, F. G., Barrera, M., & Martinez, C. R. (2004) · Prevention Science · doi:10.1023/b:prev.0000013980.12412.cd · Cited by 1,228 · cited 4×
“terms of their complexity and comprehensiveness [27].”
BackgroundInfluentialShediac-Rizkallah, M. C., & Bone, L. R. (1998) · Health Education Research · doi:10.1093/her/13.1.87 · Cited by 1,129
Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence
Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. S. (1993) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.61.4.620 · Cited by 873 · cited 1×
“While the recommended considerations for fidelity include unique and essential elements, necessary but not unique, acceptable but not necessary, and proscribed elements [3], most fidelity instruments do not contain an exhaustive listing of acceptable and proscribed behaviors.”
BackgroundTechniques and Procedures for Developing Grounded Theory
Straus, A. (1998) · Medical Entomology and Zoology · Cited by 841
Issues in Disseminating and Replicating Effective Prevention Programs
Elliott, D. S., & Mihalic, S. (2004) · Prevention Science · doi:10.1023/b:prev.0000013981.28071.52 · Cited by 794 · cited 1×
“cessary [4,6-10], few studies have examined the impact”
BackgroundAn Agenda for Research on the Sustainability of Public Health Programs
Scheirer, M. A., & Dearing, J. W. (2011) · American Journal of Public Health · doi:10.2105/ajph.2011.300193 · Cited by 775 · cited 1×
“The tension between modification and fidelity is a critical issue in implementation science [4,54,55].”
BackgroundCultural Sensitivity and Adaptation in Family-Based Prevention Interventions
Kumpfer, K. L., Alvarado, R., Smith, P., & Bellamy, N. (2002) · Prevention Science · doi:10.1023/a:1019902902119 · Cited by 688 · cited 3×
“Kumpfer, Smith & Bellamy, [16] Prevention Author description”
BackgroundResultLau, A. S. (2006) · Clinical Psychology: Science and Practice · doi:10.1111/j.1468-2850.2006.00042.x · Cited by 624 · cited 1×
“[24-26] have also made recommendations regarding specific processes for adapting mental health interventions”
BackgroundDynamic adaptation process to implement an evidence-based child maltreatment intervention
Aarons, G. A., Green, A. E., Palinkas, L. A., Self-Brown, S., Whitaker, D. J., Lutzker, J. R., Silovsky, J. F., Hecht, D. B., & Chaffin, M. J. (2012) · Implementation Science · doi:10.1186/1748-5908-7-32 · Cited by 449 · cited 1×
“[24-26] have also made recommendations regarding specific processes for adapting mental health interventions”
BackgroundToward the Effective and Efficient Measurement of Implementation Fidelity
Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0321-0 · Cited by 422 · cited 1×
“Assessing fidelity, which comprises adherence to the intervention components, competence or skill with which the intervention is delivered, and differentiation from other treatments [2], by itself may fail to capture certain types of modifications (e.”
BackgroundManualized therapy for PTSD: Flexing the structure of cognitive processing therapy.
Galovski, T. E., Blain, L. M., Mott, J. M., Elwood, L., & Houle, T. (2012) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0030600 · Cited by 286 · cited 1×
“Galovski and colleagues also found positive outcomes when a highly specified set of adaptations were used in a different PTSD treatment [12].”
BackgroundCBT for Vietnamese refugees with treatment‐resistant PTSD and panic attacks: A pilot study
Hinton, D. E., Pham, T., Tran, M., Safren, S. A., Otto, M. W., & Pollack, M. H. (2004) · Journal of Traumatic Stress · doi:10.1023/b:jots.0000048956.03529.fa · Cited by 252
Webster-Stratton, C. H., & Reid, M. J. (2010) · Journal of Children s Services · doi:10.5042/jcs.2010.0115 · Cited by 216 · cited 2×
“A total of 32 articles were identified [9,11,13,15-18,20,32-51], which described 258 unique modifications (see Table 1).”
BackgroundReinventing fidelity: The transfer of social technology among settings
Bauman, L. J., Stein, R. E. K., & Ireys, H. T. (1991) · American Journal of Community Psychology · doi:10.1007/bf00937995 · Cited by 208 · cited 3×
“For example, Devieux and colleagues [23] described a process of operationalizing the adaptation process based on Bauman and colleagues’ framework for adaptation [8], which includes efforts to retain the integrity of an intervention’s causal/conceptual model.”
BackgroundMethodHasson, H., Blomberg, S., & Dunér, A. (2012) · Implementation Science · doi:10.1186/1748-5908-7-23 · Cited by 194 · cited 2×
“A total of 32 articles were identified [9,11,13,15-18,20,32-51], which described 258 unique modifications (see Table 1).”
BackgroundNastasi, B. K., Varjas, K., Schensul, S. L., Silva, K. T., Schensul, J. J., & Ratnayake, P. (2000) · School Psychology Quarterly · doi:10.1037/h0088785 · Cited by 189 · cited 1×
“, [21] Sexual risk prevention Author description”
McCabe, K. M., Yeh, M., Garland, A. F., Lau, A. S., & Chavez, G. S. (2005) · University Libraries (University of Maryland) · doi:10.13016/eman-s23m · Cited by 187 · cited 2×
“A total of 32 articles were identified [9,11,13,15-18,20,32-51], which described 258 unique modifications (see Table 1).”
BackgroundKalichman, S. C., Kelly, J. A., Hunter, T. L., & Murphy, D. A. (1993) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.61.2.291 · Cited by 164 · cited 3×
“For example, an enhanced outcome was demonstrated after modifying a brief HIV risk-reduction video intervention to match presenter and participant ethnicity and sex [13]; effectiveness was also retained after modifying an HIV risk-reduction intervention to meet the needs of five different communities [14].”
BackgroundKelly, J. A., Heckman, T. G., Stevenson, L. Y., Williams, P. N., Ertl, T., Rb, H., Nr, L., O’Donnell, L. N., Ma, T., Sogolow, E. D., et al. (2000) · PubMed · Cited by 163
Linking Sustainability Research to Intervention Types
Scheirer, M. A. (2013) · American Journal of Public Health · doi:10.2105/ajph.2012.300976 · Cited by 161 · cited 4×
“We intended this framework to apply in particular to three types of programs and interventions outlined by Scheirer [28]: those implemented by individual providers; programs requiring coordination among multiple staff; and new procedures aimed at targeting individual behaviors or behavioral health conditions.”
BackgroundInfluentialMcIntyre, L. L. (2008) · Journal of Intellectual Disability Research · doi:10.1111/j.1365-2788.2008.01108.x · Cited by 147 · cited 1×
“A total of 32 articles were identified [9,11,13,15-18,20,32-51], which described 258 unique modifications (see Table 1).”
BackgroundA Focused Approach to Assessing Program Fidelity
Hill, L. G., Maucione, K., & Hood, B. (2006) · Prevention Science · doi:10.1007/s11121-006-0051-4 · Cited by 134 · cited 6×
“Hill, Maucione & Hood, [20] Substance abuse prevention Clinician interview”
BackgroundInfluentialProject IMPACT: A Report on Barriers and Facilitators to Sustainability
Blasinsky, M., Goldman, H. H., & Unützer, J. (2006) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-006-0086-7 · Cited by 121 · cited 2×
“A total of 32 articles were identified [9,11,13,15-18,20,32-51], which described 258 unique modifications (see Table 1).”
BackgroundGoldstein, N. E. S., Kemp, K. A., Leff, S. S., & Lochman, J. E. (2012) · Clinical Psychology Science and Practice · doi:10.1111/cpsp.12011 · Cited by 119 · cited 1×
“[24-26] have also made recommendations regarding specific processes for adapting mental health interventions”
BackgroundLevitt, J. T., Malta, L. S., Martin, A. L., Davis, L. L., & Cloître, M. (2007) · Behaviour Research and Therapy · doi:10.1016/j.brat.2007.01.004 · Cited by 116
Easier Said than Done: Intervention Sustainability in an Urban After-School Program
Lyon, A. R., Frazier, S. L., Mehta, T. G., Atkins, M. S., & Weisbach, J. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-011-0339-y · Cited by 94 · cited 3×
“A total of 32 articles were identified [9,11,13,15-18,20,32-51], which described 258 unique modifications (see Table 1).”
BackgroundMelde, C., Esbensen, F., & Tusinski, K. E. (2006) · Evaluation Review · doi:10.1177/0193841x06293412 · Cited by 83 · cited 2×
“A total of 32 articles were identified [9,11,13,15-18,20,32-51], which described 258 unique modifications (see Table 1).”
BackgroundTortolero, S. R., Markham, C. M., Parcel, G. S., Peters, R. J., Escobar‐Chaves, S. L., Basen-Engquist, K. M., & Lewis, H. L. (2005) · Health Promotion Practice · doi:10.1177/1524839904266472 · Cited by 81 · cited 1×
“, [22] Sexual risk prevention Author description”
Kaysen, D. L., Lindgren, K. P., Zangana, G. A. S., Murray, L. K., Bass, J., & Bolton, P. A. (2011) · Psychological Trauma Theory Research Practice and Policy · doi:10.1037/a0026053 · Cited by 79
Participatory Model of Mental Health Programming: Lessons Learned from Work in a Developing Country
Nastasi, B. K., Sarkar, S., Varjas, K. M., & Jayasena, A. (1998) · School Psychology Review · doi:10.1080/02796015.1998.12085913 · Cited by 78 · cited 1×
“A total of 32 articles were identified [9,11,13,15-18,20,32-51], which described 258 unique modifications (see Table 1).”
BackgroundAssociation Between HIV and Subpreputial Penile Wetness in Uncircumcised Men in South Africa
O’Farrell, N., Morison, L. A., Moodley, P., Pillay, K., Vanmali, T., QUIGLEY, M. A., Hayes, R., & Sturm, A. W. (2006) · JAIDS Journal of Acquired Immune Deficiency Syndromes · doi:10.1097/01.qai.0000225014.61192.98 · Cited by 73
Stanton, B., Guo, J., Cottrell, L., Galbraith, J., Li, X., Gibson, C., Pack, R., Cole, M., Marshall, S., & Harris, C. (2005) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2004.10.005 · Cited by 69 · cited 3×
“A total of 32 articles were identified [9,11,13,15-18,20,32-51], which described 258 unique modifications (see Table 1).”
BackgroundCommunity Mental Health Provider Modifications to Cognitive Therapy: Implications for Sustainability
Wiltsey Stirman, S., Calloway, A., Toder, K., Miller, C. J., DeVito, A. K., Meisel, S. N., Xhezo, R., Evans, A. C., Beck, A. T., & Crits-Christoph, P. (2013) · Psychiatric Services · doi:10.1176/appi.ps.201200456 · Cited by 63
Self-citationZvoch, K. (2009) · American Journal of Evaluation · doi:10.1177/1098214008329523 · Cited by 59 · cited 1×
“Thus, fidelity monitoring alone will not facilitate an understanding of whether different types of modifications are detrimental, non-detrimental or enhancements [4,5].”
BackgroundLundgren, L. M., Amodeo, M., Cohen, A., Chassler, D., & Horowitz, A. (2011) · Addictive Behaviors · doi:10.1016/j.addbeh.2011.01.003 · Cited by 55 · cited 3×
“, [1] Substance abuse Clinician interview”
BackgroundDushay, R. A., Singer, M. C., Weeks, M. R., Rohena, L., & Gruber, R. (2001) · The American Journal of Drug and Alcohol Abuse · doi:10.1081/ada-100104515 · Cited by 52 · cited 2×
“A total of 32 articles were identified [9,11,13,15-18,20,32-51], which described 258 unique modifications (see Table 1).”
BackgroundRemien, R. H., Stirratt, M. J., Dognin, J. S., Day, E., El‐Bassel, N., & Warne, P. A. (2006) · JAIDS Journal of Acquired Immune Deficiency Syndromes · doi:10.1097/01.qai.0000248340.20685.7d · Cited by 52 · cited 2×
“A total of 32 articles were identified [9,11,13,15-18,20,32-51], which described 258 unique modifications (see Table 1).”
BackgroundAarons, G. A., Miller, E. A., Green, A. E., Perrott, J. A., & Bradway, R. (2012) · Journal of Children's Services · doi:10.1108/17466661211286463 · Cited by 51
Adoption, Adaptation, and Fidelity of Implementation of Sexual Violence Prevention Programs
Noonan, R. K., Emshoff, J. G., Mooss, A. D., Armstrong, M., Weinberg, J. K., & Ball, B. (2009) · Health Promotion Practice · doi:10.1177/1524839908329374 · Cited by 50 · cited 2×
“, [49] Sexual violence prevention Clinician interview”
BackgroundThe effect of tailoring a model HIV prevention program for local adolescent target audiences.
Kennedy, M., Mizuno, Y., Hoffman, R., Baume, C., & Strand, J. (2000) · AIDS Education and Prevention · doi:10.1521/aeap.2000.12.3.225 · Cited by 46 · cited 2×
“For example, an enhanced outcome was demonstrated after modifying a brief HIV risk-reduction video intervention to match presenter and participant ethnicity and sex [13]; effectiveness was also retained after modifying an HIV risk-reduction intervention to meet the needs of five different communities [14].”
BackgroundMeasuring Fidelity and Adaptation: Reliability of a Instrument for School-Based Prevention Programs
Bishop, D. C., Pankratz, M. M., Hansen, W. B., Albritton, J., Albritton, L. P., & Strack, J. (2013) · Evaluation & the Health Professions · doi:10.1177/0163278713476882 · Cited by 45
Holliday, J., Audrey, S., Moore, L., Parry‐Langdon, N., & Campbell, R. M. (2009) · Health Education Journal · doi:10.1177/0017896908100448 · Cited by 43 · cited 2×
“, [51] Health promotion Observation and interview”
BackgroundWebster-Stratton, C. H., & Reid, M. J. (2008) · Journal of Children s Services · doi:10.1108/17466660200800016 · Cited by 42 · cited 2×
“A total of 32 articles were identified [9,11,13,15-18,20,32-51], which described 258 unique modifications (see Table 1).”
BackgroundCultural Adaptation in Translational Research: Field Experiences
Dévieux, J. G. (2005) · Journal of Urban Health · doi:10.1093/jurban/jti066 · Cited by 39 · cited 2×
“For example, Devieux and colleagues [23] described a process of operationalizing the adaptation process based on Bauman and colleagues’ framework for adaptation [8], which includes efforts to retain the integrity of an intervention’s causal/conceptual model.”
MethodLeerlooijer, J. N., Ruiter, R. A. C., Reinders, J., Darwisyah, W., Kok, G., & Bartholomew, L. K. (2011) · Translational Behavioral Medicine · doi:10.1007/s13142-011-0041-3 · Cited by 36 · cited 2×
“[17-19]), but there have been relatively few efforts to systematically categorize them.”
BackgroundOwczarzak, J., & Dickson‐Gómez, J. (2011) · AIDS Patient Care and STDs · doi:10.1089/apc.2010.0322 · Cited by 35 · cited 2×
“Owczarzak & Dickson-Gomez, [37] HIV Prevention Clinician interview”
BackgroundSalerno, A. J., Margolies, P. J., Cleek, A. F., Pollock, M., Gopalan, G., & Jackson, C. T. (2011) · Psychiatric Services · doi:10.1176/ps.62.5.pss6205_0456 · Cited by 27 · cited 3×
“A total of 32 articles were identified [9,11,13,15-18,20,32-51], which described 258 unique modifications (see Table 1).”
Dévieux, J. G., Malow, R. M., Rosenberg, R., & Dyer, J. G. (2004) · PubMed · Cited by 22
Adapting an Evidence-Based Intervention: Tales of the Hustler Project
Miller, R. L. (2003) · AIDS Education and Prevention · doi:10.1521/aeap.15.1.5.127.23612 · Cited by 22 · cited 2×
“Miller, [42] HIV Prevention Author description”
BackgroundWilliams, R. B., & Williams, V. P. (2011) · Translational Behavioral Medicine · doi:10.1007/s13142-011-0030-6 · Cited by 12 · cited 2×
“A total of 32 articles were identified [9,11,13,15-18,20,32-51], which described 258 unique modifications (see Table 1).”
BackgroundCultural adaptation in translational research: A review of field experiences
Jacobs, R. J., Dévieux, J. G., Malow, R. M., Rosenberg, R., Jean-Gilles, M. M., Samuels, D. M., & Ergon-Pérez, E. (2005) · NSUWorks (Nova Southeastern University) · Cited by 12
Malow, R. M., Jean-Gilles, M. M., Dévieux, J. G., Rosenberg, R., & Russell, A. (2004) · PubMed · Cited by 8
Research works printed but not linked: 1
- Schoenwald SK GA, Chapman JE, Frazier SL, Sheidow AJ, Southam-Gerow MA: Toward the effective and efficient measurement of implementation fidelity. Administration Policy Mental Health and Mental Health Serv Res. 2011, 38 (1): 32-43. 10.1007/s10488-010-0321-0. · DOI · Paper page
Cited by
Process evaluation of complex interventions: Medical Research Council guidance
Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., Moore, L., O'Cathain, A., Tinati, T., Wight, D., et al. (2015) · BMJ · doi:10.1136/bmj.h1258 · Cited by 6,204
Skivington, K., Matthews, L., Simpson, S. A., Craig, P., Baird, J., Blazeby, J. M., Boyd, K. A., Craig, N., French, D. P., McIntosh, E., et al. (2021) · BMJ · doi:10.1136/bmj.n2061 · Cited by 5,808
RE-AIM Planning and Evaluation Framework: Adapting to New Science and Practice With a 20-Year Review
Glasgow, R. E., Harden, S. M., Gaglio, B., Rabin, B., Smith, M. L., Porter, G. C., Ory, M. G., & Estabrooks, P. A. (2019) · Frontiers in Public Health · doi:10.3389/fpubh.2019.00064 · Cited by 2,528
“Rapid, iterative use and analysis of brief practical measures of RE-AIM factors can inform adaptations (55, 66, 71).”
MethodWiltsey Stirman, S., Baumann, A. A., & Miller, C. J. (2019) · Implementation Science · doi:10.1186/s13012-019-0898-y · Cited by 1,329
Self-citationSystematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework
Moullin, J. C., Dickson, K. S., Stadnick, N. A., Rabin, B., & Aarons, G. A. (2019) · Implementation Science · doi:10.1186/s13012-018-0842-6 · Cited by 1,223
Metadata
- Authors
- Stirman, S. W., Miller, C. J., Toder, K., & Calloway, A.
- Affiliations
- National Center for PTSD (US)
- VA Boston Healthcare System (US)
- University of Pennsylvania (US)
- University of Massachusetts Boston (US)
- Corresponding author
- Stirman, Shannon Wiltsey
- Venue
- Implementation Science
- Published
- 2013-06-10
- Volume
- 8
- Issue
- 1
- Pages
- 65-65
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC3686699
- OpenAlex
- W2109880945
- Cited by
- 691
- References
- 59
- Citations by year
- 2026: 49 · 2025: 50 · 2024: 61 · 2023: 79 · 2022: 76 · 2021: 97 · 2020: 72 · 2019: 69 · 2018: 40 · 2017: 40 · 2016: 19 · 2015: 22 · 2014: 12 · 2013: 3
- FWCI
- 33.81
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Implementation Science (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- U.S. Department of Veterans Affairs (R25 MH080916-01A2)
- Washington University in St. Louis (R25 MH080916-01A2)
- National Institutes of Health
- Quality Enhancement Research Initiative (R25 MH080916-01A2)
- National Institute of Mental Health (MH080916, R00 MH 01800, R00MH080100, R25MH080916)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Keywords
- evidence-based interventions, intervention adaptation, coding scheme, implementation fidelity, community settings, rater agreement, intervention effectiveness
- MeSH terms
- Mental Disorders, Behavioral Medicine, Community Mental Health Services, Decision Making, Delivery of Health Care, Humans, Evidence-Based Medicine, Clinical Coding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

