Cited by 38Health Policy Implementation ScienceDelphi Technique in ResearchHealth Education and Validation

Refining Expert Recommendations for Implementing Change (ERIC) strategy surveys using cognitive interviews with frontline providers

Open access
Yakovchenko, Vera, Chinman, Matthew J., Lamorte, Carolyn, Powell, Byron J., Waltz, Thomas J., Merante, Monica, Gibson, Sandra, Neely, Brittney +2 more·2023·Implementation Science Communications

Abstract

Background: The Expert Recommendations for Implementing Change (ERIC) compilation includes 73 defined implementation strategies clustered into nine content areas. This taxonomy has been used to track implementation strategies over time using surveys. This study aimed to improve the ERIC survey using cognitive interviews with non-implementation scientist clinicians. Methods: Starting in 2015, we developed and fielded annual ERIC surveys to evaluate liver care in the Veterans Health Administration (VA). We invited providers who had completed at least three surveys to participate in cognitive interviews (October 2020 to October 2021). Before the interviews, participants reviewed the complete 73-item ERIC survey and marked which strategies were unclear due to wording, conceptual confusion, or overlap with other strategies. They then engaged in semi-structured cognitive interviews to describe the experience of completing the survey and elaborate on which strategies required further clarification. Results: Twelve VA providers completed surveys followed by cognitive interviews. The “Engage Consumer” and “Support Clinicians” clusters were rated most highly in terms of conceptual and wording clarity. In contrast, the “Financial” cluster had the most wording and conceptual confusion. The “Adapt and Tailor to Context” cluster strategies were considered to have the most redundancy. Providers outlined ways in which the strategies could be clearer in terms of wording (32%), conceptual clarity (51%), and clarifying the distinction between strategies (51%). Conclusions: Cognitive interviews with ERIC survey participants allowed us to identify and address issues with strategy wording, combine conceptually indistinct strategies, and disaggregate multi-barreled strategies. Improvements made to the ERIC survey based on these findings will ultimately assist VA and other institutions in designing, evaluating, and replicating quality improvement efforts.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s43058-023-00409-3

Cites

30 of 30 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Yakovchenko, V., Chinman, M. J., Lamorte, C., Powell, B. J., Waltz, T. J., Merante, M., Gibson, S., Neely, B., Morgan, T. R., & Rogal, S. S.
Affiliations
  • VA Pittsburgh Healthcare System (US)
  • RAND Corporation (US)
  • Washington University in St. Louis (US)
  • Eastern Michigan University (US)
  • University of Pittsburgh (US)
  • University of California, Irvine (US)
Corresponding author
Rogal, Shari S.
Venue
Implementation Science Communications
Published
2023-04-21
Volume
4
Issue
1
Pages
42-42
ISSN
2662-2211 (ISSN-L)
OpenAlex
W4366774448
Cited by
38
References
30
Citations by year
2026: 11 · 2025: 9 · 2024: 17 · 2023: 1
FWCI
13.46
Type
journal article
Language
en
Open access
Locations
  • Implementation Science Communications (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • eScholarship (California Digital Library) (submittedVersion, cc-by)
  • Digital Commons@Becker (Washington University School of Medicine) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Quality Enhancement Research Initiative (PEC 19-307)
  • National Institute on Drug Abuse (K23DA048182)
Topics
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
  • Social Sciences › Education › Health Education and Validation
Keywords
Expert Recommendations for Implementing Change, implementation strategies, questionnaire development, cognitive interviewing, frontline health workers, Veterans Health Administration, quality improvement

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review