CoChat

A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project

Open access

The DOI matches its Crossref record; 61 of the 62 references it cites were found in a registry or an index; 15 of the 30 sources its claims cite were checked for support.

Abstract

BACKGROUND: Identifying, developing, and testing implementation strategies are important goals of implementation science. However, these efforts have been complicated by the use of inconsistent language and inadequate descriptions of implementation strategies in the literature. The Expert Recommendations for Implementing Change (ERIC) study aimed to refine a published compilation of implementation strategy terms and definitions by systematically gathering input from a wide range of stakeholders with expertise in implementation science and clinical practice. METHODS: Purposive sampling was used to recruit a panel of experts in implementation and clinical practice who engaged in three rounds of a modified Delphi process to generate consensus on implementation strategies and definitions. The first and second rounds involved Web-based surveys soliciting comments on implementation strategy terms and definitions. After each round, iterative refinements were made based upon participant feedback. The third round involved a live polling and consensus process via a Web-based platform and conference call. RESULTS: Participants identified substantial concerns with 31% of the terms and/or definitions and suggested five additional strategies. Seventy-five percent of definitions from the originally published compilation of strategies were retained after voting. Ultimately, the expert panel reached consensus on a final compilation of 73 implementation strategies. CONCLUSIONS: This research advances the field by improving the conceptual clarity, relevance, and comprehensiveness of implementation strategies that can be used in isolation or combination in implementation research and practice. Future phases of ERIC will focus on developing conceptually distinct categories of strategies as well as ratings for each strategy's importance and feasibility. Next, the expert panel will recommend multifaceted strategies for hypothetical yet real-world scenarios that vary by sites' endorsement of evidence-based programs and practices and the strength of contextual supports that surround the effort.

Overview

CoChat Summary

TL;DR

This paper reports the ERIC project’s modified Delphi process to refine a published compilation of implementation strategy terms and definitions through expert consensus. The result is a final compilation of 73 discrete implementation strategies, with 75% of original definitions retained after voting.

Key findings

  • A panel of 71 experts participated across three modified Delphi rounds; round participation was 57, 43, and 40 experts, respectively.
  • Participants identified substantial concerns with 31% of the original terms and/or definitions and suggested five additional strategies.
  • Seventy-five percent of definitions from the original Powell et al. compilation were retained after voting, and the final compilation included 73 discrete implementation strategies.
  • Of 21 alternative definitions suggested, an alternative was selected 81% of the time and the original definition was maintained 19% of the time; all five new strategies were retained in some form, with an average 84% vote to retain (range 100–71%).

Methods

  • Purposive sampling recruited an expert panel of 71 implementation and clinical practice experts, mostly from the USA, with some Canadian affiliation.
  • A three-round modified Delphi process was used: two Web-based surveys for feedback and iterative refinement, followed by a live Web-based polling and consensus meeting.
  • Consensus voting used approval voting with a ≥60% supermajority threshold, followed by structured discussion and runoff voting when no clear winner emerged.

Limitations

  • The paper states that using a different starting taxonomy may have yielded different results; the panel was limited to North America and mostly USA experts; online rather than in-person meetings may have produced less nuanced discussion; and some participants abstained from Round 3 voting for unknown reasons.
  • The paper also states content limitations: the evidence base for each strategy was not considered, strategies were not explicitly tied to theories or conceptual models, and some strategies may be more applicable to US or North American settings.
  • Evident from the text: the compilation is not intended as a checklist, and the authors note that no implementation effort could feasibly use every strategy; future research is needed to identify contexts and circumstances under which each discrete strategy is effective.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s13012-015-0209-1

Cites

61 of 61 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Overview of the voting process in the final round of the modified Delphi task. In the third and final round of the modified Delphi task, expert panelists will vote on all strategies where concerns were raised regarding the core definition in the first two online survey rounds. For each strategy, the original and proposed alternate definitions will be presented for an approval poll where participants can vote to approve all definition alternatives they find acceptable. In the first round of voting, if one definition receives a supermajority of votes (≥60%) and receives more votes than all others, that definition will be declared the winner and the poll will move to the next term. If there is no consensus, a 5-min discussion period is opened. When the discussion concludes, a runoff poll is conducted to determine the most acceptable definition alternative [13].
Figure 1 Overview of the voting process in the final round of the modified Delphi task. In the third and final round of the modified Delphi task, expert panelists will vote on all strategies where concerns were raised regarding the core definition in the first two online survey rounds. For each strategy, the original and proposed alternate definitions will be presented for an approval poll where participants can vote to approve all definition alternatives they find acceptable. In the first round of voting, if one definition receives a supermajority of votes (≥60%) and receives more votes than all others, that definition will be declared the winner and the poll will move to the next term. If there is no consensus, a 5-min discussion period is opened. When the discussion concludes, a runoff poll is conducted to determine the most acceptable definition alternative [13].CC BY · PubMed Central

Verification

Claims the paper makes

9 Valid · 0 Not valid · 8 Not checked · 53 Own results and interpretations

17 more claims · 3 with their cited sources checked

Create a free account to see all 70 claims and to verify the 15 references still open.

See all claims — free account
  • Probably not a substantial claim. 40
  • It may state two separate claims as one. 23
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

11 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 61 references

Research works printed but not linked: 4

  1. Cochrane Effective Practice and Organisation of Care Group. Data collection checklist. 2002. p. 1–30.
  2. Walter I, Nutley S, Davies H. Developing a taxonomy of interventions used to increase the impact of research. Andrews: University of St. Andrews of St. Andrews; 2003.
  3. Ryan R, Lowe D, Santesso N, Hill S. Development of a taxonomy of interventions directed at consumers to promote evidence-based prescribing and medicines use: a tool for evidence-based decision-making. Melbourne, Australia: In National Medicines Symposium; 2010.
  4. Shojania KG, McDonald KM, Wachter RM, Owens DK. Closing the quality gap: a critical analysis of quality improvement strategies, volume 1—series overview and methodology. Technical review 9. Agency for Healthcare Research and Quality: Rockville, MD; 2004.

Cited by

See all 4,963

Metadata

Authors
Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E.
Affiliations
  • University of Pennsylvania (US)
  • Eastern Michigan University (US)
  • RAND Corporation (US)
  • VA Ann Arbor Healthcare System (US)
  • Central Arkansas Veterans Healthcare System (US)
  • Washington University in St. Louis (US)
Corresponding author
Powell, Byron J
Venue
Implementation Science
Published
2015-02-12
Volume
10
Issue
1
Pages
21-21
ISSN
1748-5908 (ISSN-L)
OpenAlex
W2010049072
Cited by
4,963
References
62
Citations by year
2026: 687 · 2025: 734 · 2024: 800 · 2023: 659 · 2022: 645 · 2021: 576 · 2020: 357 · 2019: 217 · 2018: 142 · 2017: 77 · 2016: 43 · 2015: 20
FWCI
171.87
Type
journal article
Language
en
Open access
Locations
  • Implementation Science (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
  • U.S. Department of Veterans Affairs (VISN4 MIRECC)
  • Doris Duke Charitable Foundation
  • Purdue University
  • University of Pennsylvania
  • Health Services Research and Development (QLP 55-025)
  • University of Minnesota (55-025)
  • Washington University in St. Louis
  • University of Washington
  • Johns Hopkins University
  • Indiana University-Purdue University Indianapolis
  • University of Arkansas for Medical Sciences
  • Fahs-Beck Fund for Research and Experimentation
  • Saint Louis University
  • University of Massachusetts Boston
  • Eastern Michigan University
  • University of Toronto
  • Quality Enhancement Research Initiative
  • University of California, Los Angeles
  • Perelman School of Medicine, University of Pennsylvania
  • VISN 4 Mental Illness Research, Education, and Clinical Center
  • National Institute of Mental Health (F31MH098478)
  • National Institute of Diabetes and Digestive and Kidney Diseases (P30DK092950)
  • NIH Clinical Center
Topics
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
  • Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
Keywords
implementation strategies, Expert Recommendations for Implementing Change, terminology and definitions, expert consensus, Delphi method, stakeholder feedback, contextual support
MeSH terms
Delphi Technique, Humans, Organizational Innovation, Program Development, Quality Improvement

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. https://app.cochat.ai/papers/10.1186/s13012-015-0209-1
The paper
Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015). A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science. https://doi.org/10.1186/s13012-015-0209-1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.