Expert recommendations for implementing change (ERIC): protocol for a mixed methods study
Open accessThe DOI matches its Crossref record; 80 of the 81 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
BACKGROUND: Identifying feasible and effective implementation strategies that are contextually appropriate is a challenge for researchers and implementers, exacerbated by the lack of conceptual clarity surrounding terms and definitions for implementation strategies, as well as a literature that provides imperfect guidance regarding how one might select strategies for a given healthcare quality improvement effort. In this study, we will engage an Expert Panel comprising implementation scientists and mental health clinical managers to: establish consensus on a common nomenclature for implementation strategy terms, definitions and categories; and develop recommendations to enhance the match between implementation strategies selected to facilitate the use of evidence-based programs and the context of certain service settings, in this case the U.S. Department of Veterans Affairs (VA) mental health services. METHODS/DESIGN: This study will use purposive sampling to recruit an Expert Panel comprising implementation science experts and VA mental health clinical managers. A novel, four-stage sequential mixed methods design will be employed. During Stage 1, the Expert Panel will participate in a modified Delphi process in which a published taxonomy of implementation strategies will be used to establish consensus on terms and definitions for implementation strategies. In Stage 2, the panelists will complete a concept mapping task, which will yield conceptually distinct categories of implementation strategies as well as ratings of the feasibility and effectiveness of each strategy. Utilizing the common nomenclature developed in Stages 1 and 2, panelists will complete an innovative menu-based choice task in Stage 3 that involves matching implementation strategies to hypothetical implementation scenarios with varying contexts. This allows for quantitative characterizations of the relative necessity of each implementation strategy for a given scenario. In Stage 4, a live web-based facilitated expert recommendation process will be employed to establish expert recommendations about which implementations strategies are essential for each phase of implementation in each scenario. DISCUSSION: Using a novel method of selecting implementation strategies for use within specific contexts, this study contributes to our understanding of implementation science and practice by sharpening conceptual distinctions among a comprehensive collection of implementation strategies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In 2015, the Expert Recommendations for Implementing Change (ERIC) study organized a panel of implementation scientists to compile a standardized set of implementation strategy terms and definitions [2–4].”
From A systematic review of experimentally tested implementation strategies across health and human service settings: evidence from 2010-2022 (2024) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Implementation strategies were classified using standard definitions (MJC, SSR, DEG), based on minor modifications to the ERIC taxonomy [2–4]. Modifications resulted in 70 named strategies and were made to decrease redundancy and improve clarity.”
From A systematic review of experimentally tested implementation strategies across health and human service settings: evidence from 2010-2022 (2024) · cites this paper as [3]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“However, efforts to identify, develop, and test implementation strategies have been complicated by a lack of conceptual clarity [1,6-9].”
From A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project (2015) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rogers, E. M., Singhal, A., & Quinlan, M. M. (2019) · An Integrated Approach to Communication Theory and Research · doi:10.4324/9780203710753-35 · Cited by 37,322
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496 · cited 2×
“For instance, settings are likely to vary substantially with regard to patient characteristics [43,44]; provider-level factors such as attitudes toward EBPPs [45]; organizationallevel characteristics such as culture and climate [46], implementation climate [47], organizational readiness for change [48], leadership [49…”
BackgroundSelf-citationValente, T. W. (2003) · Genetics in Medicine · doi:10.1097/01.gim.0000061743.67794.c4 · Cited by 13,723
Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-013-0528-y · Cited by 12,038 · cited 1×
“purposive sample is consistent with the qualitative methods employed in the study design [60].”
ResultMichie, S., Richardson, M., Johnston, M., Abraham, C., Francis, J., Hardeman, W., Eccles, M. P., Cane, J., & Wood, C. E. (2013) · Annals of Behavioral Medicine · doi:10.1007/s12160-013-9486-6 · Cited by 7,629 · cited 1×
“, [9,14,15] and behavior change techniques [16] have been developed to encourage more consistent use of terms and definitions in the published literature.”
MethodImplementation strategies: recommendations for specifying and reporting
Proctor, E. K., Powell, B. J., & McMillen, J. C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-139 · Cited by 2,585 · cited 3×
“Frequently, terms and definitions for implementation strategies are inconsistently applied [5,9], and they are rarely defined or described in sufficient detail to be useful to implementation stakeholders [6,10].”
BackgroundSelf-citationKnowledge translation of research findings
Grimshaw, J. M., Eccles, M. P., Lavis, J. N., Hill, S. J., & Squires, J. E. (2012) · Implementation Science · doi:10.1186/1748-5908-7-50 · Cited by 2,407 · cited 1×
“The evidence base for specific implementation strategies has advanced considerably [19,20]; however, it rarely provides adequate guidance regarding which strategies are likely to be effective in specific circumstances.”
BackgroundWelcome to Implementation Science
Eccles, M. P., & Mittman, B. S. (2006) · Implementation Science · doi:10.1186/1748-5908-1-1 · Cited by 2,254 · cited 1×
“, barriers and facilitators) that can influence organizational, provider and patient behavior, and by building an evidence base for specific implementation strategies that can move evidence-based programs and practices (EBPPs) into routine care [1,2].”
BackgroundConsensus development methods, and their use in clinical guideline development.
Murphy, Black, Lamping, McKee, Sanderson, Askham, & Marteau (1998) · Health Technology Assessment · doi:10.3310/hta2030 · Cited by 2,051 · cited 1×
“There are only marginal increases in the reliability of expert consensus methods after sampling crosses the threshold of 12 participants [61], and a minimum enrollment of 20 should ensure adequate saturation in qualitative analyses for the expert consensus and recommendation meetings in Stages 1 and 4 [62].”
BackgroundDiffusion of Innovations, 5th Edition
Rogers, E. M., Simon, 。., & Schuster (2003) · Cited by 2,049
Bridging Research and Practice
Tabak, R. G., Khoong, E. C., Chambers, D. A., & Brownson, R. C. (2012) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2012.05.024 · Cited by 1,386 · cited 1×
“In the absence of empirical evidence to guide the selection of strategies, one might turn to the considerable number of theories and conceptual models pertaining to implementation in order to guide the selection of strategies [32,33].”
BackgroundDissemination and Implementation Research in HealthTranslating Science to Practice
Brownson, R. C., Colditz, G. A., & Proctor, E. K. (2012) · doi:10.1093/acprof:oso/9780199751877.001.0001 · Cited by 1,355
Self-citationKitson, A. L., Rycroft-Malone, J., Harvey, G., McCormack, B., Seers, K., & Titchen, A. (2008) · Implementation Science · doi:10.1186/1748-5908-3-1 · Cited by 1,314
Mixed method designs in implementation research
Palinkas, L. A., Aarons, G. A., Horwitz, S., Chamberlain, P., Hurlburt, M., & Landsverk, J. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0314-z · Cited by 1,187 · cited 1×
“The ERIC process involves a four-stage sequential mixed methods design (qualitative→QUANTITATIVE) [57].”
MethodFlottorp, S. A., Oxman, A. D., Krause, J., Musila, N. R., Wensing, M., Godycki-Cwirko, M., Baker, R., & Eccles, M. P. (2013) · Implementation Science · doi:10.1186/1748-5908-8-35 · Cited by 1,138
Getting Started with Conjoint Analysis: Strategies for Product Design and Pricing Research
Orme, B. K. (2005) · Cited by 1,082
A compilation of strategies for implementing clinical innovations in health and mental health
Powell, B. J., McMillen, J. C., Proctor, E. K., Carpenter, C. R., Griffey, R. T., Bunger, A. C., Glass, J. E., & York, J. L. (2012) · Medical Care Research and Review · doi:10.1177/1077558711430690 · Cited by 1,069 · cited 9×
“[9] will be separated from its accompanying ancillary material, allowing for the feedback from the first round to be summarized in terms of concerns with the core definition, alternative definitions, and concerns or addendum to the ancillary materials for the strategy.”
BackgroundMethodInfluentialSelf-citationBaker, R., Camosso-Stefinovic, J., Gillies, C., Shaw, E. J., Cheater, F., Flottorp, S., & Robertson, N. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005470.pub2 · Cited by 1,003
Specifying and reporting complex behaviour change interventions: the need for a scientific method
Michie, S., Fixsen, D., Grimshaw, J. M., & Eccles, M. P. (2009) · Implementation Science · doi:10.1186/1748-5908-4-40 · Cited by 976
Fitzpatrick, Shortall, Sculpher, Murray, Morris, Lodge, Dawson, Carr, Britton, & Briggs (1998) · Health Technology Assessment · doi:10.3310/hta2200 · Cited by 969
Consensus development methods, and their use in clinical guideline development.
Murphy, M. P., Black, N. A., Lamping, D. L., et al. (1998) · PubMed · Cited by 965
Planning and Studying Improvement in Patient Care: The Use of Theoretical Perspectives
GROL, R. P. T. M., BOSCH, M. C., HULSCHER, M. E. J. L., ECCLES, M. P., & WENSING, M. (2007) · The Milbank Quarterly · doi:10.1111/j.1468-0009.2007.00478.x · Cited by 943 · cited 2×
“The characteristics of the EBPPs themselves present another challenge to the selection of implementation strategies [32,38,39].”
BackgroundIndividual determinants of research utilization by nurses: a systematic review update
Squires, J. E., Estabrooks, C. A., Gustavsson, P., & Wallin, L. (2011) · Implementation Science · doi:10.1186/1748-5908-6-1 · Cited by 798
Weiner, B. J., Amick, H., & Lee, S.-Y. D. (2008) · Medical Care Research and Review · doi:10.1177/1077558708317802 · Cited by 786
Enhancing rigour in the Delphi technique research
Hasson, F., & Keeney, S. (2011) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2011.04.005 · Cited by 760 · cited 1×
“Stage 1 involves a three-round modified Delphi process [63].”
MethodKontos, P. C., & Poland, B. D. (2009) · Implementation Science · doi:10.1186/1748-5908-4-1 · Cited by 760
Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010) · Clinical Psychology Review · doi:10.1016/j.cpr.2010.02.005 · Cited by 713 · cited 1×
“This is particularly true in mental health and social service settings where the number of randomized controlled trials and head-to-head comparisons of implementation strategies pales in comparison to those conducted in other medical and health service settings [21-25].”
ResultThe mass customization decade: An updated review of the literature
Fogliatto, F. S., da Silveira, G. J. C., & Borenstein, D. (2012) · International Journal of Production Economics · doi:10.1016/j.ijpe.2012.03.002 · Cited by 710 · cited 1×
“As a result, mass customization of consumer products has expanded greatly over the last decade [29].”
BackgroundVollmar, H. C., Mayer, H., Ostermann, T., Butzlaff, M. E., Sandars, J. E., Wilm, S., & Rieger, M. A. (2010) · Implementation Science · doi:10.1186/1748-5908-5-1 · Cited by 693
A Glossary for Dissemination and Implementation Research in Health
Rabin, B. A., Brownson, R. C., Haire-Joshu, D., Kreuter, M. W., & Weaver, N. L. (2008) · Journal of Public Health Management and Practice · doi:10.1097/01.phh.0000311888.06252.bb · Cited by 676
Davies, P., Walker, A. E., & Grimshaw, J. M. (2010) · Implementation Science · doi:10.1186/1748-5908-5-14 · Cited by 619 · cited 1×
“However, reviews of the published literature have found that theories and models have been drastically underutilized [23,34,35].”
Glisson, C., Landsverk, J., Schoenwald, S., Kelleher, K., Hoagwood, K. E., Mayberg, S., Green, P., & The Research Network on Youth Mental Health (2008) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-007-0148-5 · Cited by 608 · cited 1×
“For instance, settings are likely to vary substantially with regard to patient characteristics [43,44]; provider-level factors such as attitudes toward EBPPs [45]; organizationallevel characteristics such as culture and climate [46], implementation climate [47], organizational readiness for change [48], leadership [49…”
BackgroundPublic health program capacity for sustainability: a new framework
Schell, S. F., Luke, D. A., Schooley, M. W., Elliott, M. B., Herbers, S. H., Mueller, N. B., & Bunger, A. C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-15 · Cited by 563 · cited 1×
“For instance, settings are likely to vary substantially with regard to patient characteristics [43,44]; provider-level factors such as attitudes toward EBPPs [45]; organizationallevel characteristics such as culture and climate [46], implementation climate [47], organizational readiness for change [48], leadership [49…”
BackgroundConcept mapping: an introduction to structured conceptualization in health care
Trochim, W., & Kane, M. (2005) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzi038 · Cited by 477 · cited 2×
“Final data analyses will include visual summaries of data including weighted and unweighted cluster maps, ladder graphs, and go-zone graphs, all specific tools from the web platform used for this analysis [66,68].”
BackgroundMethodOrganizational Mechanisms for Enhancing User Innovation in Information Technology1
Nambisan, S., Agarwal, R., & Tanniru, M. (1999) · MIS Quarterly · doi:10.2307/249468 · Cited by 456 · cited 1×
“Grounding the initial Delphi round in concepts derived from the literature is more efficient for panels composed of experts who are familiar with the key concepts versus using multiple Delphi rounds for the panelists to generate the key concepts on their own [64].”
BackgroundAn introduction to concept mapping as a participatory public health research method
Burke, J. G., O’Campo, P., Peak, G. L., Gielen, A. C., McDonnell, K. A., & Trochim, W. M. K. (2005) · Qualitative Health Research · doi:10.1177/1049732305278876 · Cited by 437
Marketing Mass-Customized Products: Striking a Balance between Utility and Complexity
Dellaert, B. G. C., & Stremersch, S. (2005) · Journal of Marketing Research · doi:10.1509/jmkr.42.2.219.62293 · Cited by 417 · cited 1×
“However, decision-makers with high levels of product expertise consider large choice menus less complex than do consumers with low levels of product expertise [31].”
BackgroundSocial Science Methodology: A Criterial Framework
Gerring, J. (2001) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815492 · Cited by 384
Versteeg, M. H., Laurant, M. G., Franx, G. C., Jacobs, A. J., & Wensing, M. J. (2012) · Implementation Science · doi:10.1186/1748-5908-7-1 · Cited by 382
Pai, M., Lloyd, N. S., Cheng, J., Thabane, L., Spencer, F. A., Cook, D. J., Haynes, R. B., Schünemann, H. J., & Douketis, J. D. (2013) · Implementation Science · doi:10.1186/1748-5908-8-1 · Cited by 376
Implementation of the CALM intervention for anxiety disorders: a qualitative study
Curran, G. M., Sullivan, G., Mendel, P., Craske, M. G., Sherbourne, C. D., Stein, M. B., McDaniel, A., & Roy-Byrne, P. (2012) · Implementation Science · doi:10.1186/1748-5908-7-14 · Cited by 348
McKibbon, K. A., Lokker, C., Wilczynski, N. L., Ciliska, D., Dobbins, M., Davis, D. A., Haynes, R. B., & Straus, S. E. (2010) · Implementation Science · doi:10.1186/1748-5908-5-16 · Cited by 345
Social science methodology: a criterial framework
(2002) · Choice Reviews Online · doi:10.5860/choice.39-6491 · Cited by 336
Aarons, G. A., & Sommerfeld, D. H. (2012) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2012.01.018 · Cited by 318 · cited 1×
“For instance, settings are likely to vary substantially with regard to patient characteristics [43,44]; provider-level factors such as attitudes toward EBPPs [45]; organizationallevel characteristics such as culture and climate [46], implementation climate [47], organizational readiness for change [48], leadership [49…”
BackgroundNovins, D. K., Green, A. E., Legha, R. K., & Aarons, G. A. (2013) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2013.07.012 · Cited by 270 · cited 1×
“This is particularly true in mental health and social service settings where the number of randomized controlled trials and head-to-head comparisons of implementation strategies pales in comparison to those conducted in other medical and health service settings [21-25].”
ResultA Systematic Review of Strategies for Implementing Empirically Supported Mental Health Interventions
Powell, B. J., Proctor, E. K., & Glass, J. E. (2014) · Research on Social Work Practice · doi:10.1177/1049731513505778 · Cited by 268 · cited 2×
“However, reviews of the published literature have found that theories and models have been drastically underutilized [23,34,35].”
BackgroundResultSelf-citationThe meaning and measurement of implementation climate
Weiner, B. J., Belden, C. M., Bergmire, D. M., & Johnston, M. (2011) · Implementation Science · doi:10.1186/1748-5908-6-78 · Cited by 260 · cited 1×
“For instance, settings are likely to vary substantially with regard to patient characteristics [43,44]; provider-level factors such as attitudes toward EBPPs [45]; organizationallevel characteristics such as culture and climate [46], implementation climate [47], organizational readiness for change [48], leadership [49…”
BackgroundAlbrecht, L., Archibald, M., Arseneau, D., & Scott, S. D. (2013) · Implementation Science · doi:10.1186/1748-5908-8-52 · Cited by 252 · cited 1×
“Additionally, several groups have advanced reporting guidelines and advocated for the improved reporting of implementation strategies [6,10,17,18].”
BackgroundA systematic review of the use of theory in randomized controlled trials of audit and feedback
Colquhoun, H. L., Brehaut, J. C., Sales, A., Ivers, N., Grimshaw, J., Michie, S., Carroll, K., Chalifoux, M., & Eva, K. W. (2013) · Implementation Science · doi:10.1186/1748-5908-8-66 · Cited by 231 · cited 1×
“However, reviews of the published literature have found that theories and models have been drastically underutilized [23,34,35].”
BackgroundAarons, G. A., Cafri, G., Lugo, L., & Sawitzky, A. (2012) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0302-3 · Cited by 225
Dissemination and Implementation of Evidence-Based Psychological Interventions
McHugh, R. K., & Barlow, D. H. (2015) · Oxford Clinical Psychology · doi:10.1093/med:psych/9780195389050.003.0013 · Cited by 211
Raghavan, R., Bright, C. L., & Shadoin, A. L. (2008) · Implementation Science · doi:10.1186/1748-5908-3-26 · Cited by 205 · cited 12×
“While this particular application of the ERIC process focuses on the implementation of EBPPs in mental health service settings within the VA, these methods are suitable for other practice areas.”
BackgroundInfluentialTailored Implementation for Chronic Diseases (TICD): a project protocol
Wensing, M., Oxman, A., Baker, R., Godycki-Cwirko, M., Flottorp, S., Szecsenyi, J., Grimshaw, J., & Eccles, M. (2011) · Implementation Science · doi:10.1186/1748-5908-6-103 · Cited by 184
Stetler, C. B., Mittman, B. S., & Francis, J. (2008) · Implementation Science · doi:10.1186/1748-5908-3-8 · Cited by 176
Refining a taxonomy for guideline implementation: results of an exercise in abstract classification
Mazza, D., Bairstow, P., Buchan, H., Chakraborty, S. P., Van Hecke, O., Grech, C., & Kunnamo, I. (2013) · Implementation Science · doi:10.1186/1748-5908-8-32 · Cited by 173 · cited 1×
“, [9,14,15] and behavior change techniques [16] have been developed to encourage more consistent use of terms and definitions in the published literature.”
MethodLinking Sustainability Research to Intervention Types
Scheirer, M. A. (2013) · American Journal of Public Health · doi:10.2105/ajph.2012.300976 · Cited by 161 · cited 1×
“The characteristics of the EBPPs themselves present another challenge to the selection of implementation strategies [32,38,39].”
BackgroundWhat implementation interventions increase cancer screening rates? a systematic review
Brouwers, M. C., De Vito, C., Bahirathan, L., Carol, A., Carroll, J. C., Cotterchio, M., Dobbins, M., Lent, B., Levitt, C., Lewis, N., et al. (2011) · Implementation Science · doi:10.1186/1748-5908-6-111 · Cited by 152
The State Policy Context of Implementation Issues for Evidence-Based Practices in Mental Health
Isett, K. R., Burnam, M. A., Coleman-Beattie, B., Hyde, P. S., Morrissey, J. P., Magnabosco, J., Rapp, C. A., Ganju, V., & Goldman, H. H. (2007) · Psychiatric Services · doi:10.1176/appi.ps.58.7.914 · Cited by 146 · cited 1×
“Different types of EBPPs often require unique implementation strategies to ensure their implementation and sustainment [40,41].”
BackgroundBhattacharyya, O., Reeves, S., Garfinkel, S., & Zwarenstein, M. (2006) · Implementation Science · doi:10.1186/1748-5908-1-5 · Cited by 144 · cited 1×
“The underutilization of theory may also be indicative of limitations of the theories and models themselves [36,37], and signal the need to develop more pragmatic tools that can guide the selection of implementation strategies in practice settings.”
BackgroundDeveloping the Terminology for Dissemination and Implementation Research
Rabin, B. A., & Brownson, R. C. (2012) · Dissemination and Implementation Research in HealthTranslating Science to Practice · doi:10.1093/acprof:oso/9780199751877.003.0002 · Cited by 128 · cited 1×
“The lack of clarity in terminology and definitions in the implementation literature has been well-documented [5-8].”
BackgroundProctor, E. K., Landsverk, J., Baumann, A. A., Mittman, B. S., Aarons, G. A., Brownson, R. C., Glisson, C., & Chambers, D. (2013) · Implementation Science · doi:10.1186/1748-5908-8-105 · Cited by 126 · cited 1×
“These groups include: the editorial board for the journal ‘Implementation Science,’ implementation research coordinators (IRCs) for VA QUERIs [4], and faculty and fellows from the Implementation Research Institute [58].”
Self-citationGreen, A. E., & Aarons, G. A. (2011) · Implementation Science · doi:10.1186/1748-5908-6-104 · Cited by 126 · cited 1×
“Then, panelists will rate each discrete implementation strategy in terms of its importance and feasibility [68-70].”
BackgroundCollins, K. M. T. (2010) · Sage eBooks · doi:10.4135/9781506335193.n15 · Cited by 126
The OFF theory of research utilization
Oxman, A. D., Fretheim, A., & Flottorp, S. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.10.002 · Cited by 107 · cited 1×
“The underutilization of theory may also be indicative of limitations of the theories and models themselves [36,37], and signal the need to develop more pragmatic tools that can guide the selection of implementation strategies in practice settings.”
BackgroundSystematic review of economic evaluations and cost analyses of guideline implementation strategies
Vale, L., Thomas, R., MacLennan, G., & Grimshaw, J. (2007) · The European Journal of Health Economics · doi:10.1007/s10198-007-0043-8 · Cited by 107 · cited 1×
“The dearth of economic evaluations in implementation research also makes it difficult to ascertain the costs and benefits of specific implementation strategies [27,28].”
BackgroundBrownson, R. C., Kelly, C. M., Eyler, A. A., Carnoske, C., Grost, L., Handy, S. L., Maddock, J. E., Pluto, D., Ritacco, B. A., Sallis, J. F., et al. (2008) · Journal of Physical Activity and Health · doi:10.1123/jpah.5.4.488 · Cited by 101
Mental Health Team Leadership and Consumers' Satisfaction and Quality of Life
Corrigan, P. W., Lickey, S. E., Campion, J., & Rashid, F. (2000) · Psychiatric Services · doi:10.1176/appi.ps.51.6.781 · Cited by 100 · cited 1×
“For instance, settings are likely to vary substantially with regard to patient characteristics [43,44]; provider-level factors such as attitudes toward EBPPs [45]; organizationallevel characteristics such as culture and climate [46], implementation climate [47], organizational readiness for change [48], leadership [49…”
BackgroundMagnabosco, J. L. (2006) · Implementation Science · doi:10.1186/1748-5908-1-13 · Cited by 91 · cited 1×
“Different types of EBPPs often require unique implementation strategies to ensure their implementation and sustainment [40,41].”
BackgroundLandsverk, J., Brown, C. H., Rolls Reutz, J., Palinkas, L., & Horwitz, S. M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0315-y · Cited by 89 · cited 1×
“This is particularly true in mental health and social service settings where the number of randomized controlled trials and head-to-head comparisons of implementation strategies pales in comparison to those conducted in other medical and health service settings [21-25].”
ResultOrganizational Measurement and the Implementation of Innovations in Mental Health Services
Kimberly, J. R., & Cook, J. M. (2007) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-007-0143-x · Cited by 84 · cited 1×
“For instance, settings are likely to vary substantially with regard to patient characteristics [43,44]; provider-level factors such as attitudes toward EBPPs [45]; organizationallevel characteristics such as culture and climate [46], implementation climate [47], organizational readiness for change [48], leadership [49…”
BackgroundUniform Mental Health Services in VA Medical Centers and Clinics
(2008) · PsycEXTRA Dataset · doi:10.1037/e553462011-001 · Cited by 72
Health Decision Making: Lynchpin of Evidence-Based Practice
Spring, B. (2008) · Medical Decision Making · doi:10.1177/0272989x08326146 · Cited by 61 · cited 1×
“For instance, settings are likely to vary substantially with regard to patient characteristics [43,44]; provider-level factors such as attitudes toward EBPPs [45]; organizationallevel characteristics such as culture and climate [46], implementation climate [47], organizational readiness for change [48], leadership [49…”
BackgroundDisposable nappies for preventing napkin dermatitis in infants
Baer, E. L., Davies, M. W., & Easterbrook, K. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004262.pub2 · Cited by 54
The Community: An Introduction to a Social System.
Mercer, B. E., & Sanders, I. T. (1958) · American Sociological Review · doi:10.2307/2088844 · Cited by 52
The Role of Economic Evaluation in Dissemination and Implementation Research
Raghavan, R. (2012) · Dissemination and Implementation Research in HealthTranslating Science to Practice · doi:10.1093/acprof:oso/9780199751877.003.0005 · Cited by 51 · cited 1×
“The dearth of economic evaluations in implementation research also makes it difficult to ascertain the costs and benefits of specific implementation strategies [27,28].”
BackgroundKnowledge Translation in Mental Health: A Scoping Review
Goldner, E., Jeffries, V., Bilsker, D., Jenkins, E., Menear, M., & Petermann, L. (2011) · Healthcare Policy | Politiques de Santé · doi:10.12927/hcpol.2011.22620 · Cited by 37 · cited 1×
“This is particularly true in mental health and social service settings where the number of randomized controlled trials and head-to-head comparisons of implementation strategies pales in comparison to those conducted in other medical and health service settings [21-25].”
ResultKauth, M. R., Sullivan, G., Cully, J., & Blevins, D. (2011) · Psychological Services · doi:10.1037/a0022250 · Cited by 32 · cited 1×
“[12] note that ‘terms such as educator, academic detailer, coach, mentor, opinion leader, and champion are often confused with facilitator ’, (italics in original) and are not differentiated from each other despite important conceptual distinctions.”
BackgroundOn Preference for Flexibility and Complexity Aversion: Experimental Evidence1
Sonsino, D., & Mandelbaum, M. (2001) · Theory and Decision · doi:10.1023/a:1015555026870 · Cited by 29 · cited 1×
“, cognitive burden) of making decisions [30].”
The Community, an Introduction to a Social System
Dynes, R. R., & Sanders, I. T. (1958) · Marriage and Family Living · doi:10.2307/348275 · Cited by 26 · cited 2×
“Aim 2 To develop a set of recommendations that specifies implementation strategies likely to be effective in integrating EBBPs into VA mental health service settings.”
BackgroundExpected utility and approval voting
Fishburn, P. C., & Brams, S. J. (1981) · Behavioral Science · doi:10.1002/bs.3830260206 · Cited by 20 · cited 1×
“Approval voting is useful for efficiently identifying the most acceptable choice [65], and it also allows for the characterization of approval for the original definitions from Powell et al.”
BackgroundCartwright, J. (2015) · The New Scientist · doi:10.1016/s0262-4079(15)60407-9 · Cited by 6
Research works printed but not linked: 3
- Cochrane Effective Practice and Organisation of Care Group: EPOC Taxonomy of professional and organisational interventions. 2002, in [ http://epoc.cochrane.org/epoc-author-resources ]
- Trochim WMK: The reliability of concept mapping. 1993, Dallas, Texas
- Orme B: Menu-Based Choice (MBC) for Multi-Check Choice Experiments. 2012, Oren, UT: Sawtooth Software, Inc.
Cited by
Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015) · Implementation Science · doi:10.1186/s13012-015-0209-1 · Cited by 4,963
“Taken together, these two deficiencies complicate the acquisition and interpretation of knowledge, preclude research syntheses such as systematic reviews and meta-analyses, limit replication in both research and practice, and ultimately stymie the translation and application of empirical studies that could inform impl…”
BackgroundMethodInfluentialSelf-citationWaltz, T. J., Powell, B. J., Matthieu, M. M., Damschroder, L. J., Chinman, M. J., Smith, J. L., Proctor, E. K., & Kirchner, J. E. (2015) · Implementation Science · doi:10.1186/s13012-015-0295-0 · Cited by 1,108
“Recruitment had been restricted to the time zones within the continental United States to minimize scheduling conflicts for elements of the ERIC project that required real time interactions among participants.”
BackgroundMethodInfluentialSelf-citationMethods to Improve the Selection and Tailoring of Implementation Strategies
Powell, B. J., Beidas, R. S., Lewis, C. C., Aarons, G. A., McMillen, J. C., Proctor, E. K., & Mandell, D. S. (2017) · The Journal of Behavioral Health Services & Research · doi:10.1007/s11414-015-9475-6 · Cited by 967
“Menu-based choice methods are particularly relevant for obtaining stakeholders’ insights regarding the types of strategies that should be considered among the many that could potentially be applied.(5,8) Menu-based choice methods are useful for generating data that can inform the mass customization of products and ser…”
InfluentialSelf-citationLewis, C. C., Klasnja, P., Powell, B. J., Lyon, A. R., Tuzzio, L., Jones, S., Walsh-Bailey, C., & Weiner, B. (2018) · Frontiers in Public Health · doi:10.3389/fpubh.2018.00136 · Cited by 588
“org May 2018 | Volume 6 | Article 136 developed by a multidisciplinary team through a structured literature review (20), Delphi process, and concept mapping exercise (19, 21, 22).”
BackgroundSelf-citationGetting a clinical innovation into practice: An introduction to implementation strategies
Kirchner, J. E., Smith, J. L., Powell, B. J., Waltz, T. J., & Proctor, E. K. (2020) · Psychiatry Research · doi:10.1016/j.psychres.2019.06.042 · Cited by 258
“The ERIC consensus development process resulted in a compilation of 73 discrete implementation strategies and their definitions (Waltz et al., 2014; Powell et al., 2015).”
Self-citation
Metadata
- Authors
- Waltz, T. J., Powell, B. J., Chinman, M. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., Damschroder, L. J., & Kirchner, J. E.
- Affiliations
- Health Services Research & Development (US)
- Washington University in St. Louis (US)
- RAND Corporation (US)
- Saint Louis University (US)
- VA Ann Arbor Healthcare System (US)
- Corresponding author
- Waltz, Thomas J
- ORCID
- Venue
- Implementation Science
- Published
- 2014-03-26
- Volume
- 9
- Issue
- 1
- Pages
- 39-39
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC3987065
- OpenAlex
- W2169090669
- Cited by
- 202
- References
- 79
- Citations by year
- 2026: 20 · 2025: 25 · 2024: 34 · 2023: 25 · 2022: 26 · 2021: 18 · 2020: 16 · 2019: 12 · 2018: 8 · 2017: 7 · 2016: 4 · 2015: 6 · 2014: 1
- FWCI
- 10.86
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Implementation Science (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Digital Commons@Becker (Washington University School of Medicine) (submittedVersion)
- Funders
- U.S. Department of Veterans Affairs
- Doris Duke Charitable Foundation
- Health Services Research and Development (QLP 55-025)
- Fahs-Beck Fund for Research and Experimentation
- National Institute of Mental Health (F31MH098478, T32MH019960)
- National Institute of Diabetes and Digestive and Kidney Diseases (P30DK092950)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- implementation strategies, expert consensus, context matching, mental health services, veterans' mental health, Department of Veterans Affairs, evidence-based programs, Delphi method, concept mapping, expert recommendations
- MeSH terms
- Clinical Protocols, Delphi Technique, Diffusion of Innovation, Humans, Mental Health Services, Organizational Innovation, Research Design, United States, United States Department of Veterans Affairs, Translational Research, Biomedical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.