The DOI matches its Crossref record; 80 of the 81 references it cites were found in a registry or an index, and 1 are not checked yet.
Expert recommendations for implementing change (ERIC): protocol for a mixed methods study
Open accessAbstract
BACKGROUND: Identifying feasible and effective implementation strategies that are contextually appropriate is a challenge for researchers and implementers, exacerbated by the lack of conceptual clarity surrounding terms and definitions for implementation strategies, as well as a literature that provides imperfect guidance regarding how one might select strategies for a given healthcare quality improvement effort. In this study, we will engage an Expert Panel comprising implementation scientists and mental health clinical managers to: establish consensus on a common nomenclature for implementation strategy terms, definitions and categories; and develop recommendations to enhance the match between implementation strategies selected to facilitate the use of evidence-based programs and the context of certain service settings, in this case the U.S. Department of Veterans Affairs (VA) mental health services. METHODS/DESIGN: This study will use purposive sampling to recruit an Expert Panel comprising implementation science experts and VA mental health clinical managers. A novel, four-stage sequential mixed methods design will be employed. During Stage 1, the Expert Panel will participate in a modified Delphi process in which a published taxonomy of implementation strategies will be used to establish consensus on terms and definitions for implementation strategies. In Stage 2, the panelists will complete a concept mapping task, which will yield conceptually distinct categories of implementation strategies as well as ratings of the feasibility and effectiveness of each strategy. Utilizing the common nomenclature developed in Stages 1 and 2, panelists will complete an innovative menu-based choice task in Stage 3 that involves matching implementation strategies to hypothetical implementation scenarios with varying contexts. This allows for quantitative characterizations of the relative necessity of each implementation strategy for a given scenario. In Stage 4, a live web-based facilitated expert recommendation process will be employed to establish expert recommendations about which implementations strategies are essential for each phase of implementation in each scenario. DISCUSSION: Using a novel method of selecting implementation strategies for use within specific contexts, this study contributes to our understanding of implementation science and practice by sharpening conceptual distinctions among a comprehensive collection of implementation strategies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In 2015, the Expert Recommendations for Implementing Change (ERIC) study organized a panel of implementation scientists to compile a standardized set of implementation strategy terms and definitions [2–4].”
From A systematic review of experimentally tested implementation strategies across health and human service settings: evidence from 2010-2022 (2024) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Implementation strategies were classified using standard definitions (MJC, SSR, DEG), based on minor modifications to the ERIC taxonomy [2–4]. Modifications resulted in 70 named strategies and were made to decrease redundancy and improve clarity.”
From A systematic review of experimentally tested implementation strategies across health and human service settings: evidence from 2010-2022 (2024) · cites this paper as [3]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“However, efforts to identify, develop, and test implementation strategies have been complicated by a lack of conceptual clarity [1,6-9].”
From A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project (2015) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rogers, E. M., Singhal, A., & Quinlan, M. M. (2019) · An Integrated Approach to Communication Theory and Research · doi:10.4324/9780203710753-35 · Cited by 37,322
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496 · cited 2×
“For instance, settings are likely to vary substantially with regard to patient characteristics [43,44]; provider-level factors such as attitudes toward EBPPs [45]; organizationallevel characteristics such as culture and climate [46], implementation climate [47], organizational readiness for change [48], leadership [49…”
BackgroundSelf-citationValente, T. W. (2003) · Genetics in Medicine · doi:10.1097/01.gim.0000061743.67794.c4 · Cited by 13,723
Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-013-0528-y · Cited by 12,038 · cited 1×
“purposive sample is consistent with the qualitative methods employed in the study design [60].”
ResultMichie, S., Richardson, M., Johnston, M., Abraham, C., Francis, J., Hardeman, W., Eccles, M. P., Cane, J., & Wood, C. E. (2013) · Annals of Behavioral Medicine · doi:10.1007/s12160-013-9486-6 · Cited by 7,629 · cited 1×
“, [9,14,15] and behavior change techniques [16] have been developed to encourage more consistent use of terms and definitions in the published literature.”
MethodImplementation strategies: recommendations for specifying and reporting
Proctor, E. K., Powell, B. J., & McMillen, J. C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-139 · Cited by 2,585 · cited 3×
“Frequently, terms and definitions for implementation strategies are inconsistently applied [5,9], and they are rarely defined or described in sufficient detail to be useful to implementation stakeholders [6,10].”
BackgroundSelf-citationKnowledge translation of research findings
Grimshaw, J. M., Eccles, M. P., Lavis, J. N., Hill, S. J., & Squires, J. E. (2012) · Implementation Science · doi:10.1186/1748-5908-7-50 · Cited by 2,407 · cited 1×
“The evidence base for specific implementation strategies has advanced considerably [19,20]; however, it rarely provides adequate guidance regarding which strategies are likely to be effective in specific circumstances.”
BackgroundWelcome to Implementation Science
Eccles, M. P., & Mittman, B. S. (2006) · Implementation Science · doi:10.1186/1748-5908-1-1 · Cited by 2,254 · cited 1×
“, barriers and facilitators) that can influence organizational, provider and patient behavior, and by building an evidence base for specific implementation strategies that can move evidence-based programs and practices (EBPPs) into routine care [1,2].”
BackgroundConsensus development methods, and their use in clinical guideline development.
Murphy, Black, Lamping, McKee, Sanderson, Askham, & Marteau (1998) · Health Technology Assessment · doi:10.3310/hta2030 · Cited by 2,051 · cited 1×
“There are only marginal increases in the reliability of expert consensus methods after sampling crosses the threshold of 12 participants [61], and a minimum enrollment of 20 should ensure adequate saturation in qualitative analyses for the expert consensus and recommendation meetings in Stages 1 and 4 [62].”
BackgroundDiffusion of Innovations, 5th Edition
Rogers, E. M., Simon, 。., & Schuster (2003) · Cited by 2,049
Research works printed but not linked: 3
- Cochrane Effective Practice and Organisation of Care Group: EPOC Taxonomy of professional and organisational interventions. 2002, in [ http://epoc.cochrane.org/epoc-author-resources ]
- Trochim WMK: The reliability of concept mapping. 1993, Dallas, Texas
- Orme B: Menu-Based Choice (MBC) for Multi-Check Choice Experiments. 2012, Oren, UT: Sawtooth Software, Inc.
Cited by
Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015) · Implementation Science · doi:10.1186/s13012-015-0209-1 · Cited by 4,963
“Taken together, these two deficiencies complicate the acquisition and interpretation of knowledge, preclude research syntheses such as systematic reviews and meta-analyses, limit replication in both research and practice, and ultimately stymie the translation and application of empirical studies that could inform impl…”
BackgroundMethodInfluentialSelf-citationWaltz, T. J., Powell, B. J., Matthieu, M. M., Damschroder, L. J., Chinman, M. J., Smith, J. L., Proctor, E. K., & Kirchner, J. E. (2015) · Implementation Science · doi:10.1186/s13012-015-0295-0 · Cited by 1,108
“Recruitment had been restricted to the time zones within the continental United States to minimize scheduling conflicts for elements of the ERIC project that required real time interactions among participants.”
BackgroundMethodInfluentialSelf-citationMethods to Improve the Selection and Tailoring of Implementation Strategies
Powell, B. J., Beidas, R. S., Lewis, C. C., Aarons, G. A., McMillen, J. C., Proctor, E. K., & Mandell, D. S. (2017) · The Journal of Behavioral Health Services & Research · doi:10.1007/s11414-015-9475-6 · Cited by 967
“Menu-based choice methods are particularly relevant for obtaining stakeholders’ insights regarding the types of strategies that should be considered among the many that could potentially be applied.(5,8) Menu-based choice methods are useful for generating data that can inform the mass customization of products and ser…”
InfluentialSelf-citationLewis, C. C., Klasnja, P., Powell, B. J., Lyon, A. R., Tuzzio, L., Jones, S., Walsh-Bailey, C., & Weiner, B. (2018) · Frontiers in Public Health · doi:10.3389/fpubh.2018.00136 · Cited by 588
“org May 2018 | Volume 6 | Article 136 developed by a multidisciplinary team through a structured literature review (20), Delphi process, and concept mapping exercise (19, 21, 22).”
BackgroundSelf-citationGetting a clinical innovation into practice: An introduction to implementation strategies
Kirchner, J. E., Smith, J. L., Powell, B. J., Waltz, T. J., & Proctor, E. K. (2020) · Psychiatry Research · doi:10.1016/j.psychres.2019.06.042 · Cited by 258
“The ERIC consensus development process resulted in a compilation of 73 discrete implementation strategies and their definitions (Waltz et al., 2014; Powell et al., 2015).”
Self-citation
Metadata
- Authors
- Waltz, T. J., Powell, B. J., Chinman, M. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., Damschroder, L. J., & Kirchner, J. E.
- Affiliations
- Health Services Research & Development (US)
- Washington University in St. Louis (US)
- RAND Corporation (US)
- Saint Louis University (US)
- VA Ann Arbor Healthcare System (US)
- Corresponding author
- Waltz, Thomas J
- ORCID
- Venue
- Implementation Science
- Published
- 2014-03-26
- Volume
- 9
- Issue
- 1
- Pages
- 39-39
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC3987065
- OpenAlex
- W2169090669
- Cited by
- 202
- References
- 79
- Citations by year
- 2026: 20 · 2025: 25 · 2024: 34 · 2023: 25 · 2022: 26 · 2021: 18 · 2020: 16 · 2019: 12 · 2018: 8 · 2017: 7 · 2016: 4 · 2015: 6 · 2014: 1
- FWCI
- 10.86
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Implementation Science (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Digital Commons@Becker (Washington University School of Medicine) (submittedVersion)
- Funders
- U.S. Department of Veterans Affairs
- Doris Duke Charitable Foundation
- Health Services Research and Development (QLP 55-025)
- Fahs-Beck Fund for Research and Experimentation
- National Institute of Mental Health (F31MH098478, T32MH019960)
- National Institute of Diabetes and Digestive and Kidney Diseases (P30DK092950)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- implementation strategies, expert consensus, context matching, mental health services, veterans' mental health, Department of Veterans Affairs, evidence-based programs, Delphi method, concept mapping, expert recommendations
- MeSH terms
- Clinical Protocols, Delphi Technique, Diffusion of Innovation, Humans, Mental Health Services, Organizational Innovation, Research Design, United States, United States Department of Veterans Affairs, Translational Research, Biomedical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Expert recommendations for implementing change (ERIC): protocol for a mixed methods study. https://app.cochat.ai/papers/10.1186/1748-5908-9-39
- The paper
- Waltz, T. J., Powell, B. J., Chinman, M. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., Damschroder, L. J., & Kirchner, J. E. (2014). Expert recommendations for implementing change (ERIC): protocol for a mixed methods study. Implementation Science. https://doi.org/10.1186/1748-5908-9-39
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.