Refining Expert Recommendations for Implementing Change (ERIC) strategy surveys using cognitive interviews with frontline providers
Open accessAbstract
Background: The Expert Recommendations for Implementing Change (ERIC) compilation includes 73 defined implementation strategies clustered into nine content areas. This taxonomy has been used to track implementation strategies over time using surveys. This study aimed to improve the ERIC survey using cognitive interviews with non-implementation scientist clinicians. Methods: Starting in 2015, we developed and fielded annual ERIC surveys to evaluate liver care in the Veterans Health Administration (VA). We invited providers who had completed at least three surveys to participate in cognitive interviews (October 2020 to October 2021). Before the interviews, participants reviewed the complete 73-item ERIC survey and marked which strategies were unclear due to wording, conceptual confusion, or overlap with other strategies. They then engaged in semi-structured cognitive interviews to describe the experience of completing the survey and elaborate on which strategies required further clarification. Results: Twelve VA providers completed surveys followed by cognitive interviews. The “Engage Consumer” and “Support Clinicians” clusters were rated most highly in terms of conceptual and wording clarity. In contrast, the “Financial” cluster had the most wording and conceptual confusion. The “Adapt and Tailor to Context” cluster strategies were considered to have the most redundancy. Providers outlined ways in which the strategies could be clearer in terms of wording (32%), conceptual clarity (51%), and clarifying the distinction between strategies (51%). Conclusions: Cognitive interviews with ERIC survey participants allowed us to identify and address issues with strategy wording, combine conceptually indistinct strategies, and disaggregate multi-barreled strategies. Improvements made to the ERIC survey based on these findings will ultimately assist VA and other institutions in designing, evaluating, and replicating quality improvement efforts.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Implementation strategies were classified using standard definitions based on minor modifications to the ERIC taxonomy, resulting in 70 named strategies.
“Implementation strategies were classified using standard definitions (MJC, SSR, DEG), based on minor modifications to the ERIC taxonomy [2–4]. Modifications resulted in 70 named strategies and were made to decrease redundancy and improve clarity.”
From A systematic review of experimentally tested implementation strategies across health and human service settings: evidence from 2010-2022 (2024) · cites this paper as [37]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Tong, A., Sainsbury, P., & Craig, J. (2007) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzm042 · Cited by 42,509 · see citing works
Fereday, J., & Muir-Cochrane, E. (2006) · International Journal of Qualitative Methods · doi:10.1177/160940690600500107 · Cited by 11,829 · cited 1× · see citing works
“Coders used a priori codes based on the interview guides and generated new codes through a general inductive approach [12, 13].”
MethodA General Inductive Approach for Analyzing Qualitative Evaluation Data
Thomas, D. R. (2006) · American Journal of Evaluation · doi:10.1177/1098214005283748 · Cited by 10,858 · see citing works
Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015) · Implementation Science · doi:10.1186/s13012-015-0209-1 · Cited by 4,963 · see citing works
Self-citationThe answer is 17 years, what is the question: understanding time lags in translational research
Morris, Z. S., Wooding, S., & Grant, J. (2011) · Journal of the Royal Society of Medicine · doi:10.1258/jrsm.2011.110180 · Cited by 3,340 · see citing works
Implementation strategies: recommendations for specifying and reporting
Proctor, E. K., Powell, B. J., & McMillen, J. C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-139 · Cited by 2,585 · see citing works
Self-citationThe think aloud method: A practical guide to modelling cognitive processes
Solomon, P. J. (1995) · Information Processing & Management · doi:10.1016/0306-4573(95)90031-4 · Cited by 748 · see citing works
Willis, G., & Artino, A. R. (2013) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-13-00154.1 · Cited by 691 · see citing works
Lewis, C. C., Klasnja, P., Powell, B. J., Lyon, A. R., Tuzzio, L., Jones, S., Walsh-Bailey, C., & Weiner, B. (2018) · Frontiers in Public Health · doi:10.3389/fpubh.2018.00136 · Cited by 588 · see citing works
Self-citationRapid and Rigorous Qualitative Data Analysis
Watkins, D. C. (2017) · International Journal of Qualitative Methods · doi:10.1177/1609406917712131 · Cited by 580 · see citing works
Knowledge translation in health: how implementation science could contribute more
Wensing, M., & Grol, R. P. T. M. (2019) · BMC Medicine · doi:10.1186/s12916-019-1322-9 · Cited by 415 · see citing works
Cook, C. R., Lyon, A. R., Locke, J. J., Waltz, T. J., & Powell, B. J. (2019) · Prevention Science · doi:10.1007/s11121-019-01017-1 · Cited by 346 · see citing works
Self-citationPerry, C. K., Damschroder, L. J., Hemler, J. R., Woodson, T. T., Ono, S. S., & Cohen, D. J. (2019) · Implementation Science · doi:10.1186/s13012-019-0876-4 · Cited by 288 · see citing works
Implementation science made too simple: a teaching tool
Curran, G. M. (2020) · Implementation Science Communications · doi:10.1186/s43058-020-00001-z · Cited by 285 · see citing works
Beidas, R. S., Dorsey, S., Lewis, C. C., Lyon, A. R., Powell, B. J., Purtle, J., Saldana, L., Shelton, R. C., Stirman, S. W., & Lane-Fall, M. B. (2022) · Implementation Science · doi:10.1186/s13012-022-01226-3 · Cited by 230 · see citing works
Self-citationExpert recommendations for implementing change (ERIC): protocol for a mixed methods study
Waltz, T. J., Powell, B. J., Chinman, M. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., Damschroder, L. J., & Kirchner, J. E. (2014) · Implementation Science · doi:10.1186/1748-5908-9-39 · Cited by 202 · see citing works
Self-citationRudd, B. N., Davis, M., & Beidas, R. S. (2020) · Implementation Science · doi:10.1186/s13012-020-01060-5 · Cited by 157 · cited 1× · see citing works
“A pragmatic implementation strategy reporting tool is in process now by Rudd and colleagues [29] and may aid in strategy use and specification among those with no specialized implementation science training.”
MethodLane-Fall, M. B., Curran, G. M., & Beidas, R. S. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0783-z · Cited by 144 · see citing works
Strategy Configurations Directly Linked to Higher Hepatitis C Virus Treatment Starts
Yakovchenko, V., Miech, E. J., Chinman, M., Chartier, M., Gonzalez, R. I., Kirchner, J. E., Morgan, T. R. M., Park, A. Y. S., Powell, B. J., Proctor, E. K., et al. (2020) · Medical Care · doi:10.1097/mlr.0000000000001319 · Cited by 113 · see citing works
Self-citationNathan, N., Powell, B. J., Shelton, R. C., Laur, C. V., Wolfenden, L., Hailemariam, M., Yoong, S. L., Sutherland, R., Kingsland, M., Waltz, T. J., et al. (2022) · Frontiers in Health Services · doi:10.3389/frhs.2022.905909 · Cited by 105 · see citing works
Self-citationRogal, S. S., Yakovchenko, V., Waltz, T. J., Powell, B. J., Kirchner, J. E., Proctor, E. K., Gonzalez, R., Park, A., Ross, D., Morgan, T. R., et al. (2017) · Implementation Science · doi:10.1186/s13012-017-0588-6 · Cited by 98 · see citing works
Self-citationPatey, A. M., Grimshaw, J., & Francis, J. (2021) · Implementation Science · doi:10.1186/s13012-021-01089-0 · Cited by 97 · see citing works
Lewis, C. C., Powell, B. J., Brewer, S. K., Nguyen, A. M., Schriger, S. H., Vejnoska, S., Walsh‐Bailey, C., Aarons, G. A., Beidas, R. S., Lyon, A. R., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-053474 · Cited by 82 · see citing works
Self-citationMechanism mapping to advance research on implementation strategies
Geng, E. H., Baumann, A. A., & Powell, B. J. (2022) · PLoS Medicine · doi:10.1371/journal.pmed.1003918 · Cited by 79 · see citing works
Self-citationMcHugh, S., Presseau, J., Luecking, C. T., & Powell, B. J. (2022) · Implementation Science · doi:10.1186/s13012-022-01227-2 · Cited by 64 · see citing works
Self-citationRogal, S. S., Yakovchenko, V., Waltz, T. J., Powell, B. J., Gonzalez, R., Park, A., Chartier, M., Ross, D., Morgan, T. R., Kirchner, J. E., et al. (2019) · Implementation Science · doi:10.1186/s13012-019-0881-7 · Cited by 52 · see citing works
Self-citationA pilot study comparing tools for tracking implementation strategies and treatment adaptations
Walsh‐Bailey, C., Palazzo, L. G., Jones, S. M. W., Mettert, K. D., Powell, B. J., Stirman, S. W., Lyon, A. R., Rohde, P., & Lewis, C. C. (2021) · Implementation Research and Practice · doi:10.1177/26334895211016028 · Cited by 42 · see citing works
Self-citationYakovchenko, V., Morgan, T. R., Chinman, M. J., Powell, B. J., Gonzalez, R., Park, A., Malone, P. S., Chartier, M., Ross, D., & Rogal, S. S. (2021) · BMC Health Services Research · doi:10.1186/s12913-021-07312-4 · Cited by 27 · see citing works
Self-citationCore implementation strategies for improving cirrhosis care in the Veterans Health Administration
Yakovchenko, V., Morgan, T. R. M., Miech, E. J., Neely, B., Lamorte, C., Gibson, S. P., Beste, L. A., McCurdy, H. M., Scott, D., Gonzalez, R. I., et al. (2022) · Hepatology · doi:10.1002/hep.32395 · Cited by 24 · see citing works
Self-citationRogal, S. S., Powell, B. J., Chinman, M., Agbalajobi, O. M., Kanwa, F., Gonzalez, R. I., Lamorte, C., Mellinger, J. L., Morgan, T. R. M., Neely, B., et al. (2020) · Gastroenterology · doi:10.1053/j.gastro.2020.07.063 · Cited by 10 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Ashcraft, L. E., Goodrich, D. E., Hero, J., Phares, A., Bachrach, R. L., Quinn, D. A., Qureshi, N., Ernecoff, N. C., Lederer, L. G., Scheunemann, L. P., et al. (2024) · Implementation Science · doi:10.1186/s13012-024-01369-5 · Cited by 89 · see citing works
Self-citationBalis, L. E., & Houghtaling, B. (2023) · Implementation Science Communications · doi:10.1186/s43058-023-00532-1 · Cited by 58 · see citing works
Lafferty, L., Altice, F. L., Leone, F., Stoove, M. A., Lloyd, A. R., Hajarizadeh, B., & Kronfli, N. (2024) · International Journal of Drug Policy · doi:10.1016/j.drugpo.2024.104549 · Cited by 25 · see citing works
Nyantakyi, E., Baenziger, J., Caci, L., Blum, K., Wolfensberger, A., Dramowski, A., Albers, B., Castro, M., Schultes, M.-T., & Clack, L. (2025) · Antimicrobial Resistance and Infection Control · doi:10.1186/s13756-025-01516-7 · Cited by 11 · see citing works
Donovan, T., Carter, H. E., McPhail, S., & Abell, B. (2024) · Implementation Science Communications · doi:10.1186/s43058-024-00648-y · Cited by 8 · see citing works
Mamani-Mategula, E., Dinklage, N. V., Sabanovic, H., Verbunt, E., Prang, K.-H., Chipeta, E. K., & Manda‐Taylor, L. (2024) · Implementation Science Communications · doi:10.1186/s43058-024-00661-1 · Cited by 9 · see citing works
Afaq, S., Zala, Z., Aleem, S., Qazi, F. K., Jamal, S. F., Khan, Z., Sarfraz, M., Niazi, A. K., Safdar, N., Basit, A., et al. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-093747 · Cited by 9 · see citing works
Okamura, K. H., Palafu, T., An, K. J., Marshall, S. M., Chin, S. K., Stern, K. A., Powell, B. J., Becker, S. J., Mandell, D. S., & Okamoto, S. K. (2024) · School Mental Health · doi:10.1007/s12310-024-09660-y · Cited by 5 · see citing works
Self-citationMachavariani, E., Dumchev, K. V., Pykalo, I., Filippovych, M., Ivasiy, R., Esserman, D., Madden, L. M., Bromberg, D. J., Haddad, M. S., Morozova, O., et al. (2024) · Contemporary Clinical Trials · doi:10.1016/j.cct.2024.107690 · Cited by 5 · see citing works
Implementation strategies for WHO guidelines to prevent, detect, and treat postpartum hemorrhage
Semrau, K. E. A., Litman, E. A., Molina, R. L., Delaney, M. M., Choi, L., Robertson, L., Noel-Storr, A. H., & Guise, J. (2025) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd016223 · Cited by 5 · see citing works
Hui, C. Y., Condon, K., Kolekar, S., Roberts, N., Sreter, K. B., Simons, S. O., Figueiredo, C. E. D. S., McKeough, Z. J., Salim, H. S., Gawlik‐Lipinski, A., et al. (2024) · PLoS ONE · doi:10.1371/journal.pone.0314914 · Cited by 3 · see citing works
Hinrichs‐Kinney, L., Derlein, D. L., Pontiff, M. E., Malone, D. J., Holtrop, J. S., & Stevens‐Lapsley, J. E. (2025) · Implementation Science Communications · doi:10.1186/s43058-025-00747-4 · Cited by 2 · see citing works
Maki, K. A., Xu, S., Wallen, G. R., Gerrard, C., Sung, C., Papneja, S., Tuason, R. T. S., Ramchandani, V. A., Diazgranados, N., & Barb, J. J. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-101197 · Cited by 2 · see citing works
Ackerley, S., Hung, S. H., Sheehy, L., Donkers, S. J., Timofeeva, P., Best, K. L., Peters, S. P., Park, S. S., Ouellet, B., Ezeugwu, V. E., et al. (2026) · Implementation Science Communications · doi:10.1186/s43058-025-00812-y · Cited by 2 · see citing works
Ramani, V. K., Sadashivaiah, A. B., & Naik, R. (2025) · Frontiers in Oncology · doi:10.3389/fonc.2025.1561692 · Cited by 1 · see citing works
Moore, S. A., Calder, J., & Potthoff, S. (2025) · Clinical Rehabilitation · doi:10.1177/02692155251382502 · Cited by 1 · see citing works
Chandrasekaran, R., Eng, E. M., Knecht, M. K., & Sacca, L. (2025) · Health Promotion International · doi:10.1093/heapro/daaf187 · Cited by 1 · see citing works
Bogale, B., Tiruneh, G. T., Argaw, M. D., Semahegn, A., Fesseha, N., Tesfaye, C., Teferi, M., Tadesse, H., Belete, M., Girma, A., et al. (2026) · PLoS ONE · doi:10.1371/journal.pone.0353519 · Cited by 1 · see citing works
Chen, P., Altice, F. L., Ghoshal, D., Ahmad, A., Long, J., Saifi, R. A., Koh, H. M., Nagadevi, N., Baharudin, M. N., Fui, E. W. M., et al. (2024) · medRxiv · doi:10.1101/2024.11.05.24316771
Opening the Black Box: A Causal Pathway Methodology for Precise Implementation Strategy DesignPreprint
Pouzols, S., Raisaro, J. L., & Mabire, C. (2025) · Research Square · doi:10.21203/rs.3.rs-7396236/v1
Gallardo-Cartagena, J. A., Oliveros, D., Cabello, R. L., Benites, C. M., Buchbinder, S. P., Konda, K. A., Altice, F. L., & Sánchez, J. (2026) · AIDS and Behavior · doi:10.1007/s10461-026-05147-w · Cited by 1 · see citing works
Ibrahim, A. M., Mohamed, L. M., Khanna, A., Fuje, M. M. A., Jimale, L. H., Dewelbait, R. M., & Omer, M. A. (2025) · medRxiv · doi:10.1101/2025.10.27.25338921
Ashcraft, L. E., Goodrich, D. E., Hero, J. O., Phares, A. D., Bachrach, R. L., Quinn, D. A., Qureshi, N., Ernecoff, N. C., Lederer, L. G., Scheunemann, L. P., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26738325
Self-citationAshcraft, L. E., Goodrich, D. E., Hero, J. O., Phares, A. D., Bachrach, R. L., Quinn, D. A., Qureshi, N., Ernecoff, N. C., Lederer, L. G., Scheunemann, L. P., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26738319
Self-citationAshcraft, L. E., Goodrich, D. E., Hero, J. O., Phares, A. D., Bachrach, R. L., Quinn, D. A., Qureshi, N., Ernecoff, N. C., Lederer, L. G., Scheunemann, L. P., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26738313
Self-citationAshcraft, L. E., Goodrich, D. E., Hero, J. O., Phares, A. D., Bachrach, R. L., Quinn, D. A., Qureshi, N., Ernecoff, N. C., Lederer, L. G., Scheunemann, L. P., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26738304
Self-citationAshcraft, L. E., Goodrich, D. E., Hero, J. O., Phares, A. D., Bachrach, R. L., Quinn, D. A., Qureshi, N., Ernecoff, N. C., Lederer, L. G., Scheunemann, L. P., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26738322
Self-citationAshcraft, L. E., Goodrich, D. E., Hero, J. O., Phares, A. D., Bachrach, R. L., Quinn, D. A., Qureshi, N., Ernecoff, N. C., Lederer, L. G., Scheunemann, L. P., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26738310
Self-citationAshcraft, L. E., Goodrich, D. E., Hero, J. O., Phares, A. D., Bachrach, R. L., Quinn, D. A., Qureshi, N., Ernecoff, N. C., Lederer, L. G., Scheunemann, L. P., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26738316
Self-citationAshcraft, L. E., Goodrich, D. E., Hero, J. O., Phares, A. D., Bachrach, R. L., Quinn, D. A., Qureshi, N., Ernecoff, N. C., Lederer, L. G., Scheunemann, L. P., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26738307
Self-citationEdwards, D., Mitchell, C., Rolls-Reutz, J., & Kerns, S. E. U. (2026) · Global Implementation Research and Applications · doi:10.1007/s43477-026-00207-8
Ilyasova, A. A., Altice, F. L., Madybaeva, D., Kurmanalieva, A., Shumskaya, N., Bromberg, D. J., Kim, J. H., Oliveros, D., & Madden, L. M. (2026) · Journal of Substance Use and Addiction Treatment · doi:10.1016/j.josat.2026.209908
B̀ouchard, D. R., Leadbetter, B., Diaz, M. F. F., Shannon, C. S., & Sibley, K. M. (2026) · Health & Social Care in the Community · doi:10.1155/hsc/5554842
Influence of Hospital Policy and Nurse Competency on Rates of Skin-to-Skin Contact in Neonates
Solly, H. H., Lisanti, A. J., Kamen, A. S., & Spatz, D. L. (2026) · Nursing for Women s Health · doi:10.1016/j.nwh.2025.11.005
Sanuade, O. A., Baatiema, L., Kretchy, I. A., Velazquez, A. I., Kpolar, W. Y., Ogundare, O., Samed, F. A., Yalley, R. P., Carroll, A. J., & Smith, J. D. (2026) · Implementation Science Communications · doi:10.1186/s43058-026-00908-z
Simon, A. M., Pasha, I., Sulfath, T. S., Hafis, A., Jabeen, N., Thomas, C., Begum, R., Chhabra, M., Hisham, M., & Rashid, M. (2026) · Current Infectious Disease Reports · doi:10.1007/s11908-026-00882-6
Kernaghan, D., Alshammari, H. A., Akram, G., Pratt, J., Maxwell, M., Watson, M. C., & Weir, N. (2026) · Depression and Anxiety · doi:10.1155/da/9977874
Bernués-Caudillo, L., Drury, A., González‐María, E., & Moreno-Casbas, M. T. (2026) · Implementation Science Communications · doi:10.1186/s43058-026-01076-w
- Authors
- Yakovchenko, V., Chinman, M. J., Lamorte, C., Powell, B. J., Waltz, T. J., Merante, M., Gibson, S., Neely, B., Morgan, T. R., & Rogal, S. S.
- Affiliations
- VA Pittsburgh Healthcare System (US)
- RAND Corporation (US)
- Washington University in St. Louis (US)
- Eastern Michigan University (US)
- University of Pittsburgh (US)
- University of California, Irvine (US)
- Corresponding author
- Rogal, Shari S.
- ORCID
- Venue
- Implementation Science Communications
- Published
- 2023-04-21
- Volume
- 4
- Issue
- 1
- Pages
- 42-42
- ISSN
- 2662-2211 (ISSN-L)
- PMC
- PMC10122282
- OpenAlex
- W4366774448
- Cited by
- 38
- References
- 30
- Citations by year
- 2026: 11 · 2025: 9 · 2024: 17 · 2023: 1
- FWCI
- 13.46
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Implementation Science Communications (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- Digital Commons@Becker (Washington University School of Medicine) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Quality Enhancement Research Initiative (PEC 19-307)
- National Institute on Drug Abuse (K23DA048182)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Education › Health Education and Validation
- Keywords
- Expert Recommendations for Implementing Change, implementation strategies, questionnaire development, cognitive interviewing, frontline health workers, Veterans Health Administration, quality improvement
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review