CoChat
Sign Up
Cited by 2,585Health Policy Implementation ScienceClinical practice guidelines implementationHealth Sciences Research and Education

Implementation strategies: recommendations for specifying and reporting

Open access
Proctor, Enola K, Powell, Byron J, McMillen, J Curtis·2013·Implementation Science
DOI 10.1186/1748-5908-8-139PMC PMC3882890
Open this paper in CoChat — free accountOpen PDF

Abstract

Implementation strategies have unparalleled importance in implementation science, as they constitute the 'how to' component of changing healthcare practice. Yet, implementation researchers and other stakeholders are not able to fully utilize the findings of studies focusing on implementation strategies because they are often inconsistently labelled and poorly described, are rarely justified theoretically, lack operational definitions or manuals to guide their use, and are part of 'packaged' approaches whose specific elements are poorly understood. We address the challenges of specifying and reporting implementation strategies encountered by researchers who design, conduct, and report research on implementation strategies. Specifically, we propose guidelines for naming, defining, and operationalizing implementation strategies in terms of seven dimensions: actor, the action, action targets, temporality, dose, implementation outcomes addressed, and theoretical justification. Ultimately, implementation strategies cannot be used in practice or tested in research without a full description of their components and how they should be used. As with all intervention research, their descriptions must be precise enough to enable measurement and 'reproducibility.' We propose these recommendations to improve the reporting of implementation strategies in research studies and to stimulate further identification of elements pertinent to implementation strategies that should be included in reporting guidelines for implementation strategies.

AI snapshot

Written from the paper’s full text.

TL;DR

This paper argues that implementation strategies are inconsistently labelled and poorly described, limiting replication, meta-analysis, and practice use. It proposes reporting standards for naming, defining, and operationalizing strategies across seven dimensions: actor, action, action targets, temporality, dose, implementation outcomes addressed, and theoretical justification.

Key findings

  • Implementation strategies often lack consistent labels, theoretical justification, operational definitions or manuals, and are part of packaged approaches whose specific elements are poorly understood; this impedes searching, meta-analysis, and replication.
  • The authors propose seven dimensions for specification: actor, action, action targets, temporality, dose, implementation outcomes affected, and justification (theoretical, empirical, or pragmatic).
  • They recommend two standards: all implementation studies should name and define strategies using existing literature/taxonomies, and specify/operationalize them across the seven dimensions, ideally with manuals or protocols.
  • Examples cited include Powell et al.’s menu of 68 implementation strategies grouped by six processes, Michie et al.’s taxonomy of 93 behavior change techniques, Eccles et al.’s five modifiable audit-and-feedback elements producing 288 potential forms, and a review of 61 conceptual models finding 98% addressed more than one of five socioecological levels.

Methods

  • Design: conceptual/recommendation paper, not an empirical study; the authors synthesize prior literature and existing reporting guidelines such as WIDER, SQUIRE, and the Equator Network.
  • Data or sample: no study sample or dataset; illustrative examples and tables are used, including Table 1 prerequisites and Table 2 specification of “clinical supervision” and “clinician implementation team.”
  • Main analysis: narrative argument and proposed extension of reporting guidelines; no statistical analysis is reported.

Limitations

  • The paper does not report a formal limitations section; the authors acknowledge room for national or international consensus processes to formalize and extend the proposed guidelines.
  • Evident from the text: the recommendations are not empirically tested here, and no data are presented on whether adopting them improves reproducibility, meta-analysis, or implementation outcomes.
  • Evident from the text: the scope is limited to specification and reporting of implementation strategies, not a comprehensive reporting guideline for all implementation research designs.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/1748-5908-8-139

Cites

102 of 102 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 49 cited sources checked · 0 supported · 0 flagged

  • Premisecitation

    Implementation strategies are rarely justified theoretically.

    “are rarely justified theoretically [10,11]”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [10] Davies P, Walker AE, Grimshaw JM: A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci. 2010, 5: 1-6. 10.1186/1748-5908-5-1. source

      Not checked in the free run.

    • cites [11] Colquhoun HL, Brehaut JC, Sales A, Ivers N, Grimshaw J, Michie S, Carroll K, Cahlifoux M, Eva KW: A systematic review of the use of theory in randomized controlled trials of audit and feedback. Implement Sci. 2013, 8: 1-8. 10.1186/1748-5908-8-1. source

      Not checked in the free run.

  • Premisecitation

    Implementation strategies are part of 'packaged' approaches whose specific elements are poorly understood.

    “are part of ‘packaged’ approaches whose specific elements are poorly understood [12]”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [12] Alexander JA, Hearld LR: Methods and metrics challenges of delivery-systems research. Implement Sci. 2012, 7: 1-11. 10.1186/1748-5908-7-1. source

      Not checked in the free run.

  • Premisecitation

    Hysong has produced a meta-analysis that documents how different features of audit and feedback impact its effectiveness.

    “Hysong [68] has produced a meta-analysis that documents how different features of audit and feedback impact its effectiveness.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [68] Hysong SJ: Audit and feedback features impact effectiveness on care quality. Med Care. 2009, 47: 1-8. 10.1097/MLR.0b013e3181808bb5. source

      Not checked in the free run.

  • Interpretationreasoning

    Assessing the fidelity of implementation strategy delivery enables a clear test of effectiveness by showing whether or not the strategy was delivered as intended.

    “As with clinical interventions, assessing the fidelity of implementation strategy delivery enables a clear test of effectiveness by showing whether or not the strategy was delivered as intended.”
    Quote in the paperSubstantial claim
  • Premisecitation

    The order or sequence of strategy use may be critical in some cases.

    “The order or sequence of strategy use may be critical in some cases.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [77] Lyon AR, Wiltsey Stirman S, Kerns SEU, Burns EJ: Developing the mental health workforce: review and application of training approaches from multiple disciplines. Adm Policy Ment Health. 2011, 38: 238-253. 10.1007/s10488-010-0331-y. source

      Not checked in the free run.

  • May combine two claims. 7
  • Classified as probably not a substantial claim. 22

  • May combine two claims.

    Premisecitation

    Implementation strategies are often inconsistently labelled and poorly described.

    “Implementation strategies are often inconsistently labelled and poorly described [9]”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [9] Michie S, Fixsen DL, Grimshaw JM, Eccles MP: Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implement Sci. 2009, 4: 1-6. 10.1186/1748-5908-4-1. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    All these factors significantly contribute to the challenge of measuring, testing, and effectively employing implementation strategies in actual healthcare practice.

    “All these factors significantly contribute to the challenge of measuring, testing, and effectively employing implementation strategies in actual healthcare practice.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Some implementation strategies could be employed only by certain actors.

    “Some strategies could, arguably, be employed only by certain actors.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    The selection of implementation strategies may be justified by prospective assessments that identify potential needs, barriers, or facilitators, sometimes termed determinants of practice.

    “The selection of implementation strategies may be justified by prospective assessments that identify potential needs, barriers, or facilitators—sometimes termed ‘determinants of practice’ [43,55,85,86].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [43] Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, Baker R, Eccles MP: A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013, 8: 1-11. 10.1186/1748-5908-8-1. source

      Not checked in the free run.

    • cites [55] Wensing M, Oxman A, Baker R, Godycki-Cwirko M, Flottorp S, Szecsenyi J, Grimshaw J, Eccles M: Tailored implementation for chronic diseases (TICD): a project protocol. Implement Sci. 2011, 6: 1-8. 10.1186/1748-5908-6-1. source

      Not checked in the free run.

    • cites [85] Bosch M, van der Weijden T, Wensing M, Grol R: Tailoring quality improvement interventions to identified barriers: a multiple case analysis. J Eval Clin Pract. 2007, 13: 161-168. 10.1111/j.1365-2753.2006.00660.x. source

      Not checked in the free run.

    • cites [86] Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, Robertson N:Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews. 2010, Art. No (Issue 3): CD005470-doi: 10.1002/14651858.CD005470.pub2,

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Adopting the proposed reporting guidelines would facilitate meta-analysis and replication (in research and practice) and increase the comparability of implementation strategies by allowing them to be described in similar ways.

    “Specifically, it would facilitate meta-analysis and replication (in both research and practice), and would increase the comparability of implementation strategies by allowing them to be described in similar ways.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Both naming and defining implementation strategies conceptually makes it possible to distinguish one strategy from another.

    “Indeed, both naming and defining implementation strategies conceptually makes it possible to distinguish one strategy from another.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    A wide range of stakeholders can fill the actor function of delivering implementation strategies.

    “A wide range of stakeholders can fill this function, as implementation strategies may be employed or enacted by payers, administrators, intervention developers, outside consultants, personnel within an organization charged with being ‘implementers,’ providers/clinicians/support staff, clients/patients/consumers, or community stakeholders.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Proctor et al. proposed a taxonomy of implementation outcomes consisting of acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration and sustainability.

    “Proctor et al.[47] proposed a taxonomy of implementation outcomes (acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration and sustainability).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [47] Proctor EK, Silmere H, Raghavan R, Hovmand P, Aarons GA, Bunger A, Griffey R, Hensley M: Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health Ment Health Serv Res. 2011, 38: 65-76. 10.1007/s10488-010-0319-7. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    Certain implementation strategies may target one or more implementation outcomes, including outcomes not identified in the Proctor et al. taxonomy.

    “Certain strategies may target one or more these implementation outcomes (or other outcomes not identified in the Proctor et al.[47] taxonomy).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    The 'Tower of Babel' characterization of the implementation literature makes it difficult to search for empirical studies of implementation strategies and to compare the effects of different implementation strategies through meta-analyses.

    “which makes it difficult to search for empirical studies of implementation strategies, and to compare the effects of different implementation strategies through meta-analyses [9]”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [9] Michie S, Fixsen DL, Grimshaw JM, Eccles MP: Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implement Sci. 2009, 4: 1-6. 10.1186/1748-5908-4-1. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationreasoning

    Applying reporting guidelines to both implementation trials and articles focusing on the intervention being tested would push detail about implementation processes in treatment effectiveness trials and accelerate understanding of strategies.

    “Applying such guidelines not only to implementation trials but also to articles that focus on the intervention being tested would pushing detail about implementation processes in treatment effectiveness trials and thus accelerate our understanding of strategies.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Implementation strategies are inherently complex social interventions, as they address multifaceted and complicated processes within interpersonal, organizational, and community contexts.

    “Implementation strategies are inherently complex social interventions, as they address multifaceted and complicated processes within interpersonal, organizational, and community contexts [12,56-58].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [12] Alexander JA, Hearld LR: Methods and metrics challenges of delivery-systems research. Implement Sci. 2012, 7: 1-11. 10.1186/1748-5908-7-1. source

      Not checked in the free run.

    • cites [56] Craig P, Dieppe P, Macintyre S, Mitchie S, Nazareth I, Petticrew M: Developing and evaluating complex interventions: the new medical research council guidance. BMJ. 2008, 337: 979-983. 10.1136/bmj.a979. source

      Not checked in the free run.

    • cites [57] Mittman BS: Implementation science in health care. Dissemination and implementation research in health: translating science to practice. Edited by: Brownson RC, Colditz GA, Proctor EK. 2012, New York: Oxford University Press, 400-418. source

      Not checked in the free run.

    • cites [58] May C: Towards a general theory of implementation. Implement Sci. 2013, 8: 1-14. 10.1186/1748-5908-8-1. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The literature on implementation has been characterized as a 'Tower of Babel'.

    “The literature on implementation has been characterized as a ‘Tower of Babel’ [13]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [13] McKibbon KA, Lokker C, Wilczynski NL, Ciliska D, Dobbins M, Davis DA, Haynes RB, Straus S: A cross-sectional study of the number and frequency of terms used to refer to knowledge translation in a body of health literature in 2006: a tower of Babel?. Implement Sci. 2010, 5: 1-11. 10.1186/1748-5908-5-1. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Eccles et al. describe five modifiable elements of audit and feedback that alone produce 288 potential forms of audit and feedback.

    “Eccles et al.[67] describe five modifiable elements of audit and feedback that alone produce 288 potential forms of audit and feedback”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [67] The Improved Clinical Effectiveness through Behavioural Research Group (ICEBeRG): Designing theoretically-informed implementation interventions. Implement Sci. 2006, 1: 1-8. 10.1186/1748-5908-1-1. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    The complexity of implementation strategies is a function of where they are directed or the conceptual 'targets' they attempt to impact.

    “The complexity of implementation strategies is also a function of where they are directed or the conceptual ‘targets’ they attempt to impact.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The SQUIRE guidelines suggest that authors provide a description of the intervention and its component parts in sufficient detail to reproduce it, an indication of the main factors that contributed to the choice of the intervention, and initial plans for how the intervention was to be implemented, including specific steps and roles.

    “The Standards for Quality Improvement Reporting Excellence (SQUIRE) suggest that authors provide, among other things, a description of the intervention (in this case implementation strategy) and its component parts in sufficient detail so that others could reproduce it, an indication of the main factors that contributed to the choice of the intervention, and initial plans for how the intervention”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [93] Davidoff F, Batalden P, Stevens D, Ogrinc G, Mooney S: Publication guidelines for quality improvement in health care: evolution of the SQUIRE project. Qual Saf Health Care. 2008, 17 (Supplement 1): i3-i9. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationcitation

    Adopting the proposed reporting guidelines would address many current problems that make it difficult to interpret and use findings from implementation research, such as inconsistent labelling, poor descriptions, and unclear justification for specific implementation strategies.

    “Adopting these guidelines would address many of the current problems that make it difficult to interpret and use findings from implementation research, such as inconsistent labelling, poor descriptions, and unclear justification for specific implementation strategies [9-11,13].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [9] Michie S, Fixsen DL, Grimshaw JM, Eccles MP: Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implement Sci. 2009, 4: 1-6. 10.1186/1748-5908-4-1. source

      Not checked in the free run.

    • cites [10] Davies P, Walker AE, Grimshaw JM: A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci. 2010, 5: 1-6. 10.1186/1748-5908-5-1. source

      Not checked in the free run.

    • cites [11] Colquhoun HL, Brehaut JC, Sales A, Ivers N, Grimshaw J, Michie S, Carroll K, Cahlifoux M, Eva KW: A systematic review of the use of theory in randomized controlled trials of audit and feedback. Implement Sci. 2013, 8: 1-8. 10.1186/1748-5908-8-1. source

      Not checked in the free run.

    • cites [13] McKibbon KA, Lokker C, Wilczynski NL, Ciliska D, Dobbins M, Davis DA, Haynes RB, Straus S: A cross-sectional study of the number and frequency of terms used to refer to knowledge translation in a body of health literature in 2006: a tower of Babel?. Implement Sci. 2010, 5: 1-11. 10.1186/1748-5908-5-1. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Homonymy, synonymy, and instability are evident within the dissemination and implementation science literature.

    “Certainly, these problems are evident within the dissemination and implementation science literature [13,62-64]”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [13] McKibbon KA, Lokker C, Wilczynski NL, Ciliska D, Dobbins M, Davis DA, Haynes RB, Straus S: A cross-sectional study of the number and frequency of terms used to refer to knowledge translation in a body of health literature in 2006: a tower of Babel?. Implement Sci. 2010, 5: 1-11. 10.1186/1748-5908-5-1. source

      Not checked in the free run.

    • cites [62] Rabin BA, Purcell P, Naveed S, Moser RP, Henton MD, Proctor EK, Brownson RC, Glasgow RE: Advancing the application, quality and harmonization of implementation science measures. Implement Sci. 2012, 7: 1-11. 10.1186/1748-5908-7-1. source

      Not checked in the free run.

    • cites [63] Rabin BA, Brownson RC, Joshu-Haire D, Kreuter MW, Weaver NL: A glossary of dissemination and implementation research in health. J Public Health Manag. 2008, 14: 117-123. 10.1097/01.PHH.0000311888.06252.bb. source

      Not checked in the free run.

    • cites [64] Rabin BA, Brownson RC: Developing terminology for dissemination and implementation research. Dissemination and implementation research in health: translating science to practice. Edited by: Brownson RC, Colditz GA, Proctor EK. 2012, New York: Oxford University Press, 23-51. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    There is a call for hybrid trials that advance knowledge about both the treatment and the implementation.

    “This point is underscored by the call for ‘hybrid trials’ that advance knowledge about both the treatment and the implementation [15].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [15] Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C: Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012, 50: 217-226. 10.1097/MLR.0b013e3182408812. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    There is a need for the development of a suite of reporting guidelines for different types of implementation research.

    “However, there is a need for the development of a suite of reporting guidelines for different types of implementation research [3].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [3] Eccles MP, Armstrong D, Baker R, Cleary K, Davies H, Davies S, Gasziou P, Ilott I, Kinmonth ALL, Leng G, Logan S, Marteau T, Michie S, Rogers H, Rycroft-Malone J, Sibbald B: An implementation research agenda. Implement Sci. 2009, 4: 1-7. 10.1186/1748-5908-4-1. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    Linguistic harmony in implementation science will be advanced if authors label or describe implementation strategies using terms that already appear in a published review article, a strategy compilation or taxonomy, or another primary research article.

    “Linguistic harmony in implementation science will be advanced if authors label or describe implementation strategies using terms that already appear in a published review article, a strategy compilation or taxonomy, or another primary research article.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    Definitions and descriptions of implementation strategies in the literature often lack the clarity required to interpret study results and build upon the knowledge gained through replication and extension of the research.

    “However, definitions and descriptions of implementation strategies in the literature often lack the clarity required to interpret study results and build upon the knowledge gained through the replication and extension of the research.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Implementation strategies often lack operational definitions or manuals to guide their use.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The Equator Network is a repository of reporting guidelines (e.g., CONSORT and STROBE) that can provide guidance to specific research designs and methodologies used in implementation research.

    “The Equator Network [94] is a repository of reporting guidelines (e.g., CONSORT and STROBE) that can provide guidance to specific research designs and methodologies utilized in implementation research.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [94] Equator network. http://www.equator-network.org/ ,

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Determinants of practice can be identified through formal assessment processes, theory or conceptual models, research literature, or informal approaches such as brainstorming.

    “While these determinants of practice could be identified through formal assessment processes, they could also be identified using theory or conceptual models e.g., [87], research literature e.g., [59,60,88-90], or more informal approaches such as brainstorming e.g., [55].”
    Quote in the paperNot a substantial claim0 of 7 sources checked
    • cites [87] McSherry LA, Dombrowski SU, Francis JJ, Murphy J, Martin CM, O’Leary JJ, Sharp L, ATHENS Group: ‘It’s A can of worms’: understanding primary care practitioners’ behaviors in relation to HPV using the theoretical domains framework. Implement Sci. 2012, 7: 1-16. 10.1186/1748-5908-7-1. source

      Not checked in the free run.

    • cites [59] Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PACAC, Rubin HR: Why don’t physicians follow clinical practice guidelines?. JAMA. 1999, 282: 1458-10.1001/jama.282.15.1458. source

      Not checked in the free run.

    • cites [60] Cook JM, Biyanova T, Coyne JC: Barriers to adoption of new treatments: an internet study of practicing community psychotherapists. Adm Policy Ment Health Ment Health Serv Res. 2009, 36: 83-90. 10.1007/s10488-008-0198-3. source

      Not checked in the free run.

    • cites [88] Addis ME, Wade WA, Hatgis C: Barriers to dissemination of evidence-based practices: addressing practitioners’ concerns about manual-based psychotherapies. Clin Psychol Sci Pract. 1999, 6: 430-441. 10.1093/clipsy.6.4.430. source

      Not checked in the free run.

    • cites [89] Bartholomew NG, Joe GW, Rowan-Szai GA, Simpson DD: Counselor assessments of training and adoption barriers. J Subst Abuse Treat. 2007, 33: 193-199. 10.1016/j.jsat.2007.01.005. source

      Not checked in the free run.

    • cites [90] Shapiro CJ, Prinz RJ, Sanders MR: Facilitators and barriers to implementation of an evidence-based parenting intervention to prevent child maltreatment: the triple p-positive parenting program. Child Maltreat. 2012, 17: 86-95. 10.1177/1077559511424774. source

      Not checked in the free run.

    • cites [55] Wensing M, Oxman A, Baker R, Godycki-Cwirko M, Flottorp S, Szecsenyi J, Grimshaw J, Eccles M: Tailored implementation for chronic diseases (TICD): a project protocol. Implement Sci. 2011, 6: 1-8. 10.1186/1748-5908-6-1. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Adopting the proposed reporting guidelines would help accelerate understanding of how, why, when, and where implementation strategies work, and translation of those findings to real-world improvements in healthcare.

    “It would also help to accelerate our understanding of how, why, when, and where they work, and our translation of those findings to real-world improvements in healthcare.”
    Quote in the paperNot a substantial claim
25 of 25

  • The paper relies on this without support.

    Implementation strategies often lack operational definitions or manuals to guide their use.

    “lack operational definitions or manuals to guide their use”

  • The paper relies on this without support.

    Too rarely are the specific targets of implementation strategies clearly stated.

    “Yet too rarely are the specific targets of implementation strategies clearly stated.”

  • The paper relies on this without support.

    Most often a number of implementation strategies are combined to form a multifaceted strategy.

    “Most often a number of strategies are combined to form a multifaceted strategy such as training, consultation, and audit and feedback.”

  • The paper relies on this without support.

    Implementation strategies can vary tremendously in dosage or intensity.

    “implementation strategies also can vary tremendously in dosage or intensity.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The current study provides foundational support for developing a multifaceted implementation strategy specifically for ASD-CBOs.

    “The current study provides foundational support for developing a multifaceted implementation strategy (e.g., combining a number of discrete implementation strategies) specifically for ASD-CBOs (Proctor et al., 2013; Squires et al., 2014).”

    From Implementation-as-Usual in Community-Based Organizations Providing Specialized Services to Individuals with Autism Spectrum Disorder: A Mixed Methods Study (2021) · cites this paper as Proctor et al., 2013

  • Recent reviews and analyses of implementation science studies show limited or suboptimal use of implementation frameworks.

    “Recent reviews of the use and application of implementation frameworks in implementation efforts highlight the limited use of frameworks, despite the value in doing so.”

    From Ten recommendations for using implementation frameworks in research and practice (2020) · cites this paper as [8]

  • Proctor et al. outlined that implementation strategies should be named, defined, and specified based on seven components: actor, action, action targets, temporality, dose, outcomes, and theory/justification.

    “Furthermore, as outlined by Proctor et al. [8], implementation strategies should be named, defined, and specified, based on seven components enabling their measurement and replication: actor, action, action targets, temporality (when), dose (duration and how often), outcomes, and theory/justification.”

    From Ten recommendations for using implementation frameworks in research and practice (2020) · cites this paper as [8]

  • Even when strategies are reported in study protocols or empirical articles, there are numerous problems with their description, including inconsistent labeling; lack of operational definitions; poor description and absence of manuals to guide their use; and lack of a clear theoretical, empirical, or pragmatic justification for how the strategies were developed and applied.

    “Even when strategies are reported in study protocols or empirical articles, there are numerous problems with their description, including inconsistent labeling; lack of operational definitions; poor description and absence of manuals to guide their use; and lack of a clear theoretical, empirical, or pragmatic justification for how the strategies were developed and applied (4).”

    From Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda (2019) · cites this paper as [4]

  • A number of reporting guidelines could be leveraged to improve descriptions of strategies.

    “A number of reporting guidelines could be leveraged to improve descriptions of strategies (4, 96–100).”

    From Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda (2019) · cites this paper as [4]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • The "define it" step is described as both operational and conceptual

    Minor

    Table 1 says the second step is to define strategies operationally, but the corresponding section says that step is conceptual and explicitly distinct from operationalization.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 4

  1. Cochrane Effective Practice and Organisation of Care Group: Data Collection Checklist. 2002, 1-30. http://epoc.cochrane.org/sites/epoc.cochrane.org/files/uploads/datacollectionchecklist.pdf ,
  2. Institute for Healthcare Improvement: The breakthrough series: IHI’s colaborative model for achieving breakthrough improvement. 2003, Cambridge, Massachusetts: Institute for Healthcare Improvement
  3. Massoud MR, Nielsen GA, Nolan K, Nolan T, Schall MW, Sevin CA: A framework for spread: from local improvements to system-wide change. 2006, Institute for Healthcare Improvement: Cambridge, Massachusetts
  4. Proctor EK, Powell BJ, Feely M: Measurement issues in dissemination and implementation science. Child and adolescent therapy: dissemination and implementation of empirically supported treatments. Edited by: Beidas RS, Kendall PC. New York: Oxford University Press, In Press

Cited by

Sorted by citations

Related

Authors
Proctor, E. K., Powell, B. J., & McMillen, J. C.
Affiliations
  • Washington University in St. Louis (US)
  • University of Chicago (US)
Corresponding author
Proctor, Enola K
ORCID
Venue
Implementation Science
Publisher
Springer Science and Business Media LLC
Published
2013-12
Volume
8
Issue
1
Pages
139-139
ISSN
1748-5908 (ISSN-L)
DOI
10.1186/1748-5908-8-139
PMC
PMC3882890
OpenAlex
W2126340581
Semantic Scholar
68bffe0075c14ccf625a666aea2891c63262f1b3
Cited by
2,585
References
102
Citations by year
2026: 258 · 2025: 328 · 2024: 464 · 2023: 330 · 2022: 318 · 2021: 277 · 2020: 219 · 2019: 140 · 2018: 91 · 2017: 70 · 2016: 25 · 2015: 48 · 2014: 15
FWCI
53.13
Type
journal article
Language
en
Open access
Licence
Locations
  • Implementation Science (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
  • Washington University in St. Louis
  • Institute of Clinical and Translational Sciences
  • National Institutes of Health (UL1TR000448, TL1TR000449, 5p30mh068579-05)
  • National Institute of Mental Health (P30-MH-068579, F31MH098478, R25MH080916, T32MH019960)
  • National Cancer Institute (U54CA155496)
  • National Institute of Diabetes and Digestive and Kidney Diseases (P30DK092950)
  • National Center for Research Resources (RR-024992, UL1RR024992)
  • National Center for Advancing Translational Sciences
Topics
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
  • Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
Keywords
implementation strategies, reporting guidelines, operationalization, theoretical justification, implementation outcomes, reproducibility, intervention research
MeSH terms
Delivery of Health Care, Diffusion of Innovation, Research, Program Development

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review