Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures
Open accessThe DOI matches its Crossref record; 165 of the 165 references it cites were found in a registry or an index; 12 of the 74 sources its claims cite were checked for support.
Abstract
BACKGROUND: Two of the current methodological barriers to implementation science efforts are the lack of agreement regarding constructs hypothesized to affect implementation success and identifiable measures of these constructs. In order to address these gaps, the main goals of this paper were to identify a multi-level framework that captures the predominant factors that impact implementation outcomes, conduct a systematic review of available measures assessing constructs subsumed within these primary factors, and determine the criterion validity of these measures in the search articles. METHOD: We conducted a systematic literature review to identify articles reporting the use or development of measures designed to assess constructs that predict the implementation of evidence-based health innovations. Articles published through 12 August 2012 were identified through MEDLINE, CINAHL, PsycINFO and the journal Implementation Science. We then utilized a modified five-factor framework in order to code whether each measure contained items that assess constructs representing structural, organizational, provider, patient, and innovation level factors. Further, we coded the criterion validity of each measure within the search articles obtained. RESULTS: Our review identified 62 measures. Results indicate that organization, provider, and innovation-level constructs have the greatest number of measures available for use, whereas structural and patient-level constructs have the least. Additionally, relatively few measures demonstrated criterion validity, or reliable association with an implementation outcome (e.g., fidelity). DISCUSSION: In light of these findings, our discussion centers on strategies that researchers can utilize in order to identify, adapt, and improve extant measures for use in their own implementation research. In total, our literature review and resulting measures compendium increases the capacity of researchers to conceptualize and measure implementation-related constructs in their ongoing and future research.
Overview
CoChat Summary
TL;DR
This systematic review proposes a five-factor multi-level framework (structural, organizational, provider, patient, innovation) for constructs hypothesized to affect implementation outcomes and identifies 62 measures assessing them. It finds that organizational, provider, and innovation measures are most available, while structural and patient measures are scarce, and that criterion validity is rarely demonstrated.
Key findings
- The review identified 62 measures; most (42; 67.7%) assessed only one factor, and only one measure—the Barriers and Facilitators Assessment Instrument—included items for all five factors.
- Provider-level constructs were assessed by 35 measures (56.5%), organizational-level by 37 (59.7%), and innovation-level by 16 (25.8%); structural-level and patient-level constructs were each assessed by only 5 measures (8.1%).
- Almost half of measures (30; 48.4%) showed no criterion validity in the original validation study or search articles; among the 32 measures with criterion validity examined, adoption was the most frequent outcome (29 of 32; 90.1%), fidelity was examined in 5 of 32 (15.6%), and no studies examined implementation cost, penetration, or sustainability.
- Most measures (47; 75.8%) were developed and/or implemented in healthcare-related settings.
Methods
- Systematic literature review searching MEDLINE, PsycINFO, CINAHL, and Implementation Science through 12 August 2012, using implementation science, health, and measure-related keywords and database restrictions.
- Search yielded 589 unique peer-reviewed records; 210 full-text articles were reviewed, yielding 125 full-text articles from which 112 measures were extracted and 62 measures retained.
- Measures were item-coded for whether they contained two or more items assessing structural, organizational, provider, patient, or innovation factors; criterion validity was coded by identifying implementation outcomes (adoption, fidelity, implementation cost, penetration, sustainability) and whether measures significantly predicted them. Inter-rater reliability across coding stages ranged from 87% to 100% agreement.
Limitations
- The authors state that search strategies affect located articles and conclusions: use of MEDLINE, CINAHL, PsycINFO, and Implementation Science may miss relevant literature in databases such as ERIC or Business Source Complete; keyword and database restrictions, lack of truncation/spelling variants, and limiting to two instrument-related fields may have omitted relevant studies and measures.
- The authors state that criterion validity was assessed only from articles identified in the current search, not from separate exhaustive searches for each of the 62 measures, so criterion validity findings are based on a sampling of the literature; they also note recently developed scales may show weak criterion validity because they have been evaluated less frequently.
- Evident from the text: the review excluded non-health-related innovations, unpublished/unobtainable measures, foreign-language measures, and qualitative measures, which may further narrow the compendium beyond the authors’ stated database and keyword limitations.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
10 Valid · 1 Not valid · 31 Not checked · 52 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“And, the measures used to assess these constructs are frequently developed without direct connection to substantive theory or guiding frameworks and with minimal analysis of psychometric properties, such as internal reliability and construct validity [12].”
§ Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures > Background
What the sources say
- Supportedcites [12] Squires JE, Estabrooks CA, O'Rourke HM, Gustavsson P, Newburn-Cook CV, Wallin L: A systematic review of the psychometric properties of self-report research utilization measures used in healthcare. Implem Sci. 2011, 6: 83- source
Split into 2 atoms — 2 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The right side of the figure depicts the implementation outcomes—such as adoption, fidelity, implementation cost, penetration, and sustainability [6] — that are affected by the causal factors.”
§ Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures > Background > A multi-level framework guiding implementation science research
What the sources say
- Supportedcites [6] Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M: Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Hlth. 2011, 38: 65-76. source
The citation follows the list of examples, and the paper identifies adoption, fidelity, implementation cost, penetration, and sustainability as implementation outcomes. The claim about causal factors appears to describe the citing document’s figure rather than something attributed to this source. The Taxonomy of Implementation Outcomes and Level of Analysis for Outcomes passages establish the identification; page numbers were not provided. (LLM, conf=0.92)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Patient-level variables such as behavioral risk factors (e.g., alcohol use [29]) and motivation [30,31] often moderate the retention in and efficacy of behavioral risk reduction interventions.”
§ Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures > Background > A multi-level framework guiding implementation science research
What the sources say
- Partly supportedcites [29] Kalichman SC, Cain D, Eaton L, Jooste S, Simbayi LC: Randomized Clinical Trial of Brief Risk Reduction Counseling for Sexually Transmitted Infection Clinic Patients in Cape Town, South Africa. Am J Public Health. 2011, 101: e9-e17. sourceExperimental — accuracy not yet measured
The Discussion (page not provided) says alcohol use significantly moderated intervention outcomes, with effects less robust and durable for heavier drinkers. The paper also reports missed baseline sessions, but does not show that alcohol use moderated retention. The motivation claim is attached to other citations and is not assessed here. (LLM, conf=0.86)
- Partly supportedcites [30] Noguchi K, Albarracin D, Durantini MR, Glasman LR: Who participates in which health promotion programs? A meta-analysis of motivations underlying enrollment and retention in HIV-prevention interventions. Psychol Bull. 2007, 133: 955-975. sourceExperimental — accuracy not yet measured
The Results report that prior motivation and condom use were associated with retention, and that these relationships varied with intervention content. But the paper measured enrollment and completion of HIV-prevention programs, not whether patient-level variables moderated the interventions’ efficacy in reducing risk behavior. Its findings also do not establish the claim across behavioral risk-reduction interventions generally. (LLM, conf=0.88)
- Partly supportedcites [31] Vernon SW, Bartholomew LK, McQueen A, Bettencourt JL, Greisinger A, Coan SP, Lairson D, Chan W, Hawley ST, Myers RE: A randomized controlled trial of a tailored interactive computer-delivered intervention to promote colorectal cancer screening: sometimes more is just the same. nn Behav Med. 2011, 41: 284-299. sourceExperimental — accuracy not yet measured
The Results (Moderator Analyses; page unavailable) report that baseline stage of change, a measure of readiness, moderated the intervention’s effect on colorectal cancer screening. The Results (Preliminary Analyses; page unavailable) also report that participants who dropped out were less likely to have been in the contemplation stage. These findings support a role for motivation in this trial, but do not establish that it often moderates retention and efficacy across behavioral risk-reduction interventions. (LLM, conf=0.86)
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Many of these measures assessed constructs outlined in Roger’s diffusion of innovations theory [18] such as relative advantage, compatibility, complexity, trialability, and observability [26].”
§ Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures > Results > Factors assessed
What the sources say
- Not checked yetcites [18] Rogers EM: Diffusion of innovations 5th edn. 2003, New York: Free Press
- Supportedcites [26] Scott S, Plotnikoff R, Karunamuni N, Bize R, Rodgers W: Factors influencing the adoption of an innovation: An examination of the uptake of the Canadian Heart Health Kit (HHK). Implem Sci. 2008, 3: 41-Article included in review source
The Background identifies relative advantage, compatibility, complexity, trialability, and observability as the five attributes in Rogers’ diffusion of innovations theory. Methods > Questionnaire states that the study assessed attributes of the innovation using measures for each of those five constructs. (LLM, conf=0.95)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“we did come across a number of articles that included these outcomes (e.g., number of bloodstream infections [83]) alone or in combination with other implementation outcomes.”
§ Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures > Discussion > Criterion validity
What the sources say
- Supportedcites [83] Chan K, Hsu Y-J, Lubomski L, Marsteller J: Validity and usefulness of members reports of implementation progress in a quality improvement initiative: findings from the Team Check-up Tool (TCT). Implem Sci. 2011, 6: 115-Article included in review source
The cited paper is exactly the kind of article the citing sentence offers it as an example of: its Methods > Validation measures describe collecting 'the number of CLABSIs occurring within an ICU and number of catheter-line days ... monthly by the hospital infection preventionist', and the Abstract's Measures list CLABSI rate alongside the Team Check-up Tool implementation measures (intervention activities, unit-level prevention behaviours, barriers to team progress). Results > Measure validity then analyse time to first quarter with no CLABSI in combination with infection prevention behaviour outcomes, so a patient health outcome appears in combination with implementation outcomes rather than alone. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
52 more claims · 2 with their cited sources checked
Create a free account to see all 94 claims and to verify the 62 references still open.
- It may state two separate claims as one. 12
- Probably not a substantial claim. 28
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“we situate our intervention domain at the nexus of all the domains because intervention characteristics can both influence and be influenced by the other factors—a position also supported by the CFIR [32, 35].”
From Barriers and facilitators to implementation of evidence-based task-sharing mental health interventions in low- and middle-income countries: a systematic review using implementation science frameworks (2022) · cites this paper as [35]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“An update of that review in 2010 focused explicitly on organizational-level adoption and mainstreaming of technological innovations [3]. It identified some new literature on organizational-level routinization [4], but little new evidence on scale-up, spread, or sustainability—a finding that has been confirmed by other reviews since [5-9].”
From Beyond Adoption: A New Framework for Theorizing and Evaluating Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies (2017) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing instrument reviews are limited in scope, methods, and findings.”
From The Society for Implementation Research Collaboration Instrument Review Project: A methodology to promote rigorous evaluation (2016) · cites this paper as [5]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Second, existing methodologies for instrument reviews are narrowly focused and only provide limited psychometric assessments of the instruments.”
From The Society for Implementation Research Collaboration Instrument Review Project: A methodology to promote rigorous evaluation (2016) · cites this paper as [5]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Systematic reviews of implementation context measures have been conducted, all of which have found significant gaps and signal the continued need for measure validation and development [89–95].”
From The updated consolidated framework for implementation research based on user feedback (2022) · cites this paper as [89]
9 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures > Method > Measure and criterion validity coding
“the Organizational Readiness for Change scale [45] assesses several features of structural context in which drug treatment centers exist, including facility structure (independent versus part of a parent organization) and characteristics of the service area (rural, suburban, or urban).”
§ Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures > Results > Literature search results
“| Organizational Readiness for Change [45] | | X | X | | | O: Institutional resources, Organizational climate, motivational readiness | [45] | None | Mental Health/ Substance Abuse |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures > Background > A multi-level framework guiding implementation science research
“with one identifying unifying constructs from empirical results [20] and the other identifying unifying constructs from existing conceptual frameworks [19]”
§ Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures > Background > A multi-level framework guiding implementation science research
“Durlak and DuPre [19] reviewed meta-analyses and additional quantitative reports examining the predictors of successful implementation from over 500 studies. In contrast, Damschroder et al.[20] reviewed 19 existing implementation theories and frameworks”
Citations
Cites
The theory of planned behavior
Ajzen, I. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90020-t · Cited by 90,587
Rogers, E. M., Singhal, A., & Quinlan, M. M. (2019) · An Integrated Approach to Communication Theory and Research · doi:10.4324/9780203710753-35 · Cited by 37,322
Social Cognitive Theory: An Agentic Perspective
Bandura, A. (2001) · Annual Review of Psychology · doi:10.1146/annurev.psych.52.1.1 · Cited by 15,510
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496 · cited 5×
“, community-level [19]; outer-setting [20]), organizational(i.”
BackgroundInfluentialHealth Behavior and Health Education: Theory, Research, and Practice
(1992) · Annals of Internal Medicine · doi:10.7326/0003-4819-116-4-350_1 · Cited by 13,207
Organizational Culture and Leadership
Tierney, W. G., & Schein, E. H. (1986) · The Academy of Management Review · doi:10.2307/258322 · Cited by 9,601
Stages and processes of self-change of smoking: Toward an integrative model of change.
Prochaska, J. O., & DiClemente, C. C. (1983) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.51.3.390 · Cited by 9,098
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0319-7 · Cited by 8,721 · cited 7×
“For example, fidelity, or ‘the degree to which an intervention was implemented as it was prescribed in the original protocol or as it was intended by the program developers’ [6], necessarily involves and is affected by patient-level factors.”
BackgroundMethodInfluentialDiffusion of Innovations in Service Organizations: Systematic Review and Recommendations
GREENHALGH, T., ROBERT, G., MACFARLANE, F., BATE, P., & KYRIAKIDOU, O. (2004) · The Milbank Quarterly · doi:10.1111/j.0887-378x.2004.00325.x · Cited by 7,448 · cited 1×
“It involves attention to a wide array of multi-level variables related to the innovation itself, the local implementation context, and the behavioral strategies used to implement the innovation [8,9].”
MethodEvaluating the public health impact of health promotion interventions: the RE-AIM framework.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999) · American Journal of Public Health · doi:10.2105/ajph.89.9.1322 · Cited by 7,217 · cited 1×
“More recently, however, many frameworks have converged to outline a set of multi-level factors that are hypothesized to impact implementation outcomes [9,16-18].”
BackgroundDurlak, J. A., & DuPre, E. P. (2008) · American Journal of Community Psychology · doi:10.1007/s10464-008-9165-0 · Cited by 5,304
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
Organizational culture and leadership
Tohidi, H., & Jabbari, M. M. (2012) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2011.12.156 · Cited by 4,303
Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012) · Medical Care · doi:10.1097/mlr.0b013e3182408812 · Cited by 4,058
Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0327-7 · Cited by 3,240
Health behavior and health education
(1990) · Choice Reviews Online · doi:10.5860/choice.28-2173 · Cited by 2,922
Fisher, J. D., & Fisher, W. A. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.3.455 · Cited by 2,459
The Measurement of End-User Computing Satisfaction
Doll, W. J., & Torkzadeh, G. (1988) · MIS Quarterly · doi:10.2307/248851 · Cited by 2,412
Using Community-Based Participatory Research to Address Health Disparities
Wallerstein, N. B., & Duran, B. (2006) · Health Promotion Practice · doi:10.1177/1524839906289376 · Cited by 2,243
The Challenge of Innovation Implementation
Klein, K. J., & Sorra, J. S. (1996) · The Academy of Management Review · doi:10.2307/259164 · Cited by 2,113 · cited 1×
“, organizational climate for implementation: Implementation Effectiveness model [15]) impact implementation outcomes.”
BackgroundProctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-008-0197-4 · Cited by 1,894 · cited 2×
“More recently, however, many frameworks have converged to outline a set of multi-level factors that are hypothesized to impact implementation outcomes [9,16-18].”
BackgroundManaging Clinical Knowledge for Health Care Improvement
Balas, E. A., & Boren, S. A. (2000) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1637943 · Cited by 1,479
Dissemination and Implementation Research in HealthTranslating Science to Practice
Brownson, R. C., Colditz, G. A., & Proctor, E. K. (2012) · doi:10.1093/acprof:oso/9780199751877.001.0001 · Cited by 1,355
Marsick, V. J., & Watkins, K. E. (2003) · Advances in Developing Human Resources · doi:10.1177/1523422303005002002 · Cited by 1,318
Aarons, G. A. (2004) · Mental Health Services Research · doi:10.1023/b:mhsr.0000024351.12294.65 · Cited by 1,272 · cited 5×
“These aspects could include attitudes towards evidencebased practice [24] or perceived behavioral control for implementing the innovation [25].”
BackgroundInfluentialPacini, R., & Epstein, S. J. (1999) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.76.6.972 · Cited by 1,249
Feldstein, A. C., & Glasgow, R. E. (2008) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(08)34030-6 · Cited by 987
Planning and Studying Improvement in Patient Care: The Use of Theoretical Perspectives
GROL, R. P. T. M., BOSCH, M. C., HULSCHER, M. E. J. L., ECCLES, M. P., & WENSING, M. (2007) · The Milbank Quarterly · doi:10.1111/j.1468-0009.2007.00478.x · Cited by 943 · cited 2×
“As a number of scholars have noted, researchers currently face significant challenges in measuring implementationrelated phenomena [5-7].”
BackgroundThe PARIHS Framework—A Framework for Guiding the Implementation of Evidence-based Practice
Rycroft-Malone, J. (2004) · Journal of Nursing Care Quality · doi:10.1097/00001786-200410000-00002 · Cited by 923
General practitioners' perceptions of the route to evidence based medicine: a questionnaire survey
McColl, A., Smith, H., White, P., & Field, J. (1998) · BMJ · doi:10.1136/bmj.316.7128.361 · Cited by 888
Measuring Organizational Traits of Hospitals: The Revised Nursing Work Index
Aiken, L. H., & Patrician, P. A. (2000) · Nursing Research · doi:10.1097/00006199-200005000-00006 · Cited by 865
Assessing organizational readiness for change
Lehman, W. E. K., Greener, J. M., & Simpson, D. D. (2002) · Journal of Substance Abuse Treatment · doi:10.1016/s0740-5472(02)00233-7 · Cited by 863 · cited 6×
“These aspects could include the relative advantage of utilizing an innovation above existing practices [26] and quality of evidence supporting the innovation’s efficacy (Organization Readiness to Change Assessment, or ORCA [27]).”
BackgroundInfluentialStructural approaches to HIV prevention
Gupta, G. R., Parkhurst, J. O., Ogden, J. A., Aggleton, P., & Mahal, A. (2008) · The Lancet · doi:10.1016/s0140-6736(08)60887-9 · Cited by 842
General Social Surveys: 1972 - 1994; cumulative codebook
Davis, J. A., & Smith, T. (1994) · Cited by 842
Individual determinants of research utilization by nurses: a systematic review update
Squires, J. E., Estabrooks, C. A., Gustavsson, P., & Wallin, L. (2011) · Implementation Science · doi:10.1186/1748-5908-6-1 · Cited by 798
The cross‐level effects of culture and climate in human service teams
Glisson, C., & James, L. R. (2002) · Journal of Organizational Behavior · doi:10.1002/job.162 · Cited by 758
Cooke, R. A., & Rousseau, D. M. (1988) · Group & Organization Studies · doi:10.1177/105960118801300302 · Cited by 718
Shortell, S. M., O'Brien, J. L., Carman, J. M., FOSTER, R. W. G., Hughes, E. F., Boerstler, H., & O’Connor, E. J. (1995) · Europe PMC (PubMed Central) · Cited by 678
A Glossary for Dissemination and Implementation Research in Health
Rabin, B. A., Brownson, R. C., Haire-Joshu, D., Kreuter, M. W., & Weaver, N. L. (2008) · Journal of Public Health Management and Practice · doi:10.1097/01.phh.0000311888.06252.bb · Cited by 676
Glisson, C., Landsverk, J., Schoenwald, S., Kelleher, K., Hoagwood, K. E., Mayberg, S., Green, P., & The Research Network on Youth Mental Health (2008) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-007-0148-5 · Cited by 608 · cited 2×
“Organizational Social Context† [152] X X O: Climate, culture [152] None Mental Health/ Substance Abuse PR: Work attitudes [121] None”
BackgroundAbsenteeism and Turnover of Hospital Employees
Price, J. L., & Mueller, C. W. (1986) · Medical Entomology and Zoology · Cited by 593
Barriers: The barriers to research utilization scale
Funk, S. G., Champagne, M. T., Wiese, R. A., & Tornquist, E. M. (1991) · Applied Nursing Research · doi:10.1016/s0897-1897(05)80052-7 · Cited by 583 · cited 5×
“Most measures were utilized to examine factors that facilitate or inhibit adoption of evidence-based clinical care guidelines [56,64,65].”
BackgroundMethodInfluentialHelfrich, C. D., Li, Y.-F., Sharp, N. D., & Sales, A. E. (2009) · Implementation Science · doi:10.1186/1748-5908-4-38 · Cited by 564
Bridging the implementation gap between knowledge and action for health.
(2004) · Europe PMC (PubMed Central) · doi:10.1590/s0042-96862004001000005 · Cited by 527 · cited 3×
“The structural-level factor encompasses a number of constructs that represent the outer setting or external structure of the broader sociocultural context or community in which a specific organization is nested [3].”
BackgroundInfluentialStructural Interventions: Concepts, Challenges and Opportunities for Research
Blankenship, K. M., Friedman, S. R., Dworkin, S., & Mantell, J. E. (2006) · Journal of Urban Health · doi:10.1007/s11524-005-9007-4 · Cited by 515
A national survey of practicing psychologists' attitudes toward psychotherapy treatment manuals
Addis, M. E., & Krasnow, A. D. (2000) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.68.2.331 · Cited by 509 · cited 2×
“%) of the measures located in our search neither assessed criterion validity in Table 1 Coded measures (N = 62) Scale name and original source Structural (S) Organizational (O) Individual: Provider (PR) Individual: Patient (PA) Innovation (I) Construct information Search article/s Criterion validity Implementation con…”
MethodQuantitative and qualitative studies of organizational culture
Zammuto, R. F., & Krakower, J. Y. (1991) · Cited by 507
An implementation research agenda
Eccles, M. P., Armstrong, D., Baker, R., Cleary, K., Davies, H., Davies, S., Glasziou, P., Ilott, I., Kinmonth, A.-L., Leng, G., et al. (2009) · Implementation Science · doi:10.1186/1748-5908-4-18 · Cited by 481 · cited 1×
“At present, constructs that have been hypothesized to affect implementation outcomes are often poorly defined within studies [11,12].”
BackgroundBrown, C. E., Wickline, M. A., Ecoff, L., & Glaser, D. (2009) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2008.04878.x · Cited by 470 · cited 3×
“PR: Research skills, values, and awareness of EBP [77] Adoption*”
BackgroundDovidio, J. F., Penner, L. A., Albrecht, T. L., Norton, W. E., Gaertner, S. L., & Shelton, J. N. (2008) · Social Science & Medicine · doi:10.1016/j.socscimed.2008.03.019 · Cited by 455
Development of an evidence‐based practice questionnaire for nurses
Upton, D., & Upton, P. (2006) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2006.03739.x · Cited by 447
Diffusion of a School-Based Substance Abuse Prevention Program: Predictors of Program Implementation
Rohrbach, L. A., Graham, J. W., & Hansen, W. B. (1993) · Preventive Medicine · doi:10.1006/pmed.1993.1020 · Cited by 431
Stetler, C. B., Legro, M. W., Rycroft-Malone, J., Bowman, C., Curran, G., Guihan, M., Hagedorn, H., Pineros, S., & Wallace, C. M. (2006) · Implementation Science · doi:10.1186/1748-5908-1-23 · Cited by 411
The Role of Perceived Team Effectiveness in Improving Chronic Illness Care
Shortell, S. M., Marsteller, J. A., Lin, M., Pearson, M. L., Wu, S.-Y., Mendel, P., Cretin, S., & Rosen, M. (2004) · Medical Care · doi:10.1097/00005650-200411000-00002 · Cited by 395
Shiffman, R. N., Dixon, J., Brandt, C., Essaihi, A., Hsiao, A., Michel, G., & O'Connell, R. (2005) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-5-23 · Cited by 389
Versteeg, M. H., Laurant, M. G., Franx, G. C., Jacobs, A. J., & Wensing, M. J. (2012) · Implementation Science · doi:10.1186/1748-5908-7-1 · Cited by 382 · cited 1×
“[123] None ‡”
Measuring the working alliance in counselor supervision.
Efstation, J. F., Patton, M. J., & Kardash, C. M. (1990) · Journal of Counseling Psychology · doi:10.1037//0022-0167.37.3.322 · Cited by 380
Melnyk, B. M., Fineout‐Overholt, E., & Mays, M. Z. (2008) · Worldviews on Evidence-Based Nursing · doi:10.1111/j.1741-6787.2008.00126.x · Cited by 378
Do self- reported intentions predict clinicians' behaviour: a systematic review
Eccles, M. P., Hrisos, S., Francis, J., Kaner, E. F., Dickinson, H. O., Beyer, F., & Johnston, M. (2006) · Implementation Science · doi:10.1186/1748-5908-1-28 · Cited by 350
Overcoming Barriers and Promoting the Use of Research in Practice
Fink, R., Thompson, C. J., & Bonnes, D. (2005) · JONA: The Journal of Nursing Administration · doi:10.1097/00005110-200503000-00005 · Cited by 326
Multidimensionafity of Leader-Member Exchange: An Empirical Assessment through Scale Development
Liden, R. C., & Maslyn, J. M. (1998) · Journal of Management · doi:10.1177/014920639802400105 · Cited by 323
Instruments for Exploring Organizational Culture: A Review of the Literature
Jung, T., Scott, T., Davies, H. T. O., Bower, P., Whalley, D., McNally, R., & Mannion, R. (2009) · Public Administration Review · doi:10.1111/j.1540-6210.2009.02066.x · Cited by 320
Aarons, G. A., & Sommerfeld, D. H. (2012) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2012.01.018 · Cited by 318
Development and assessment of the Alberta Context Tool
Estabrooks, C. A., Squires, J. E., Cummings, G. G., Birdsell, J. M., & Norton, P. G. (2009) · BMC Health Services Research · doi:10.1186/1472-6963-9-234 · Cited by 317
Revised NEO Personality Inventory
(2011) · Encyclopedia of Clinical Neuropsychology · doi:10.1007/978-0-387-79948-3_5310 · Cited by 317
Hutchinson, A. M., & Johnston, L. (2004) · Journal of Clinical Nursing · doi:10.1046/j.1365-2702.2003.00865.x · Cited by 307
The Effects of Nurses’ Job Satisfaction on Retention: An Australian Perspective
Cowin, L. (2002) · JONA: The Journal of Nursing Administration · doi:10.1097/00005110-200205000-00008 · Cited by 303
Shortell, S. M., Jones, R. H., Rademaker, A. W., Gillies, R. R., Dranove, D. S., Hughes, E. F. X., Budetti, P. P., Reynolds, K. S. E., & Huang, C.-F. (2000) · Medical Care · doi:10.1097/00005650-200002000-00010 · Cited by 285
Aarons, G. A., Sommerfeld, D. H., & Walrath-Greene, C. M. (2009) · Implementation Science · doi:10.1186/1748-5908-4-83 · Cited by 276
The Research Network on Youth Mental Health, Aarons, G. A., Glisson, C., Green, P. D., Hoagwood, K., Kelleher, K. J., & Landsverk, J. A. (2012) · Implementation Science · doi:10.1186/1748-5908-7-56 · Cited by 271
Newton, M. S., Estabrooks, C. A., Norton, P., Birdsell, J. M., Adewale, A. J., & Thornley, R. (2007) · Implementation Science · doi:10.1186/1748-5908-2-1 · Cited by 264
Professional turnover: the case of nurses.
Price, J. L., & Mueller, C. W. (1981) · PubMed · Cited by 227
Wallen, G. R., Mitchell, S. A., Melnyk, B., Fineout‐Overholt, E., Miller‐Davis, C., Yates, J., & Hastings, C. (2010) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2010.05442.x · Cited by 224
Scott, S. D., Plotnikoff, R. C., Karunamuni, N., Bize, R., & Rodgers, W. (2008) · Implementation Science · doi:10.1186/1748-5908-3-41 · Cited by 217 · cited 5×
“These aspects could include the relative advantage of utilizing an innovation above existing practices [26] and quality of evidence supporting the innovation’s efficacy (Organization Readiness to Change Assessment, or ORCA [27]).”
BackgroundInfluentialThe Cooksey review of UK health research funding
Black, N. (2006) · BMJ · doi:10.1136/bmj.39059.444120.80 · Cited by 179
Helfrich, C. D., Li, Y.-F., Mohr, D. C., Meterko, M., & Sales, A. E. (2007) · Implementation Science · doi:10.1186/1748-5908-2-13 · Cited by 176 · cited 4×
“Measures were excluded because items did not assess any of the five factors (e.g., they instead measured an Chaudoir et al. Implementation Science 2013, 8:22 Page 6 of 20 http://www.implementationscience.com/content/8/1/22 implementation outcome such as fidelity [48]), were utilized only in articles examining a non-he…”
BackgroundMethodInfluentialChanging Physician Behavior to Improve Disease Prevention
Cohen, S. J., Halvorson, H. W., & Gosselink, C. A. (1994) · Preventive Medicine · doi:10.1006/pmed.1994.1040 · Cited by 175
Dissemination and Implementation Research on Community-Based Cancer Prevention
Rabin, B. A., Glasgow, R. E., Kerner, J. F., Klump, M. P., & Brownson, R. C. (2010) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2009.12.035 · Cited by 172 · cited 1×
“Existing gaps in measurement present a formidable barrier to efforts to advance implementation science [68,69].”
BackgroundInformation transfer: what do decision makers want and need from researchers?
Dobbins, M., Rosenbaum, P., Plews, N., Law, M., & Fysh, A. (2007) · Implementation Science · doi:10.1186/1748-5908-2-20 · Cited by 161
Use of Nursing Practice Research Findings
LUCKENBILL BRETT, J. L. (1987) · Nursing Research · doi:10.1097/00006199-198711000-00007 · Cited by 159
Shortell, S. M. (2004) · Medical Care Research and Review · doi:10.1177/1077558704266768 · Cited by 145 · cited 3×
“More recently, however, many frameworks have converged to outline a set of multi-level factors that are hypothesized to impact implementation outcomes [9,16-18].”
BackgroundMethodLavis, J. N., Oxman, A. D., Moynihan, R., & Paulsen, E. J. (2008) · Implementation Science · doi:10.1186/1748-5908-3-53 · Cited by 141
Wakefield, B. J., Blegen, M. A., Uden-Holman, T., Vaughn, T., Chrischilles, E., & Wakefield, D. S. (2001) · American Journal of Medical Quality · doi:10.1177/106286060101600404 · Cited by 140
El-Jardali, F., Lavis, J. N., Ataya, N., & Jamal, D. (2012) · Implementation Science · doi:10.1186/1748-5908-7-2 · Cited by 128
Development and Testing of the Context Assessment Index (CAI)
McCormack, B., McCarthy, G., Wright, J., & Coffey, A. (2009) · Worldviews on Evidence-Based Nursing · doi:10.1111/j.1741-6787.2008.00130.x · Cited by 126
Barriers to nurses' use of research: an Australian hospital study
Retsas, A., & Nolan, M. (1999) · International Journal of Nursing Studies · doi:10.1016/s0020-7489(99)00027-9 · Cited by 124
Estabrooks, C. A., Squires, J. E., Hayduk, L. A., Cummings, G. G., & Norton, P. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-107 · Cited by 122
Александрович, А. А. (2011) · Вестник Казанского государственного университета культуры и искусств · Cited by 118
Wangensteen, S., Johansson, I. S., Björkström, M. E., & Nordström, G. M. (2011) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.2010.03629.x · Cited by 116
Evidence-Based Practice: What Are Dietitians’ Perceptions, Attitudes, and Knowledge?
Byham-Gray, L. D., Gilbride, J. A., Dixon, L. B., & Stage, F. K. (2005) · Journal of the American Dietetic Association · doi:10.1016/j.jada.2005.07.007 · Cited by 105
Attitudes and Factors Affecting Research Utilization
Olade, R. A. (2003) · Nursing Forum · doi:10.1111/j.0029-6473.2003.00005.x · Cited by 105
A Study of Critical Thinking and Research Utilization Among Nurses
Profetto-McGrath, J., Hesketh, K. L., Lang, S., & Estabrooks, C. A. (2003) · Western Journal of Nursing Research · doi:10.1177/0193945902250421 · Cited by 104
Squires, J. E., Estabrooks, C. A., O'Rourke, H. M., Gustavsson, P., Newburn-Cook, C. V., & Wallin, L. (2011) · Implementation Science · doi:10.1186/1748-5908-6-83 · Cited by 101
DAVIES, A., Wong, C. A., & Laschinger, H. K. S. (2011) · Journal of Nursing Management · doi:10.1111/j.1365-2834.2011.01269.x · Cited by 99
Kauth, M. R., Sullivan, G., Blevins, D., Cully, J. A., Landes, R. D., Said, Q., & Teasdale, T. A. (2010) · Implementation Science · doi:10.1186/1748-5908-5-75 · Cited by 98 · cited 19×
“Therapist Characteristics and Barriers to Implementation [174] X X O: clinic type, location, organizational support [174] None Mental Health/ Substance Abuse PR: Perceived skills and ability, counseling discipline, level of education, workload, motivation”
InfluentialAppraisal of Guidelines Research & Evaluation:AGREE Instrument
Wu, W. (2007) · Journal of Evidence-Based Medicine · Cited by 96
Faculty Knowledge, Attitudes, and Perceived Barriers to Teaching Evidence-Based Nursing
Stichler, J. F., Fields, W., Kim, S. C., & Brown, C. E. (2011) · Journal of Professional Nursing · doi:10.1016/j.profnurs.2010.09.012 · Cited by 94
FIRE (facilitating implementation of research evidence): a study protocol
Seers, K., Cox, K., Crichton, N. J., Edwards, R. T., Eldh, A. C., Estabrooks, C. A., Harvey, G., Hawkes, C., Kitson, A., Linck, P., et al. (2012) · Implementation Science · doi:10.1186/1748-5908-7-25 · Cited by 93
A survey of Hong Kong nurses' perceptions of barriers to and facilitators of research utilization
Chau, J. P. C., López, V., & Thompson, D. R. (2008) · Research in Nursing & Health · doi:10.1002/nur.20289 · Cited by 89
Vernon, S. W., Bartholomew, L. K., McQueen, A., Bettencourt, J. L., Greisinger, A., Coan, S. P., Lairson, D., Chan, W., Hawley, S. T., & Myers, R. E. (2011) · Annals of Behavioral Medicine · doi:10.1007/s12160-010-9258-5 · Cited by 87
French, B., Thomas, L. H., Baker, P., Burton, C. R., Pennington, L., & Roddam, H. (2009) · Implementation Science · doi:10.1186/1748-5908-4-28 · Cited by 87
Measurement Issues in Dissemination and Implementation Research
Proctor, E. K., & Brownson, R. C. (2012) · Dissemination and Implementation Research in HealthTranslating Science to Practice · doi:10.1093/acprof:oso/9780199751877.003.0013 · Cited by 87
Stetler, C. B., Ritchie, J., Rycroft-Malone, J., Schultz, A., & Charns, M. (2007) · Implementation Science · doi:10.1186/1748-5908-2-3 · Cited by 84
Yamada, J., Stevens, B., Sidani, S., Watt‐Watson, J., & de Silva, N. (2010) · Worldviews on Evidence-Based Nursing · doi:10.1111/j.1741-6787.2010.00182.x · Cited by 83
Amemori, M., Michie, S., Korhonen, T., Murtomaa, H., & Kinnunen, T. H. (2011) · Implementation Science · doi:10.1186/1748-5908-6-50 · Cited by 82
Participatory Approaches for Study Design and Analysis in Dissemination and Implementation Research
Minkler, M., & Salvatore, A. L. (2012) · Dissemination and Implementation Research in HealthTranslating Science to Practice · doi:10.1093/acprof:oso/9780199751877.003.0010 · Cited by 80
Lyons, C., Brown, T., Tseng, M. H., Casey, J., & McDonald, R. (2011) · Australian Occupational Therapy Journal · doi:10.1111/j.1440-1630.2010.00900.x · Cited by 73 · cited 2×
“, adoption [77-80]).”
Changing nurses' pain assessment practice: a collaborative research utilization approach*
Dufault, M. A., Bielecki, C., Collins, E., & Willey, C. (1995) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.1995.21040634.x · Cited by 73
Systems Antecedents for Dissemination and Implementation: A Review and Analysis of Measures
Emmons, K. M., Weiner, B., Fernandez, M. E., & Tu, S.-P. (2012) · Health Education & Behavior · doi:10.1177/1090198111409748 · Cited by 72 · cited 1×
“A number of researchers have also provided reviews of limited portions of this literature [38-41].”
BackgroundReed, J. G., & Baxter, P. M. (2009) · Cited by 71
Brehaut, J. C., Graham, I. D., Wood, T. J., Taljaard, M., Eagles, D., Lott, A., Clement, C., Kelly, A.-M., Mason, S., Kellerman, A., et al. (2010) · Medical Decision Making · doi:10.1177/0272989x09344747 · Cited by 70
Estrada, N. A. (2009) · Worldviews on Evidence-Based Nursing · doi:10.1111/j.1741-6787.2009.00161.x · Cited by 70
Hagedorn, H., & Heideman, P. W. (2010) · Implementation Science · doi:10.1186/1748-5908-5-46 · Cited by 70
The relationship between characteristics of context and research utilization in a pediatric setting
Cummings, G. G., Hutchinson, A. M., Scott, S. D., Norton, P. G., & Estabrooks, C. A. (2010) · BMC Health Services Research · doi:10.1186/1472-6963-10-168 · Cited by 69
Norman, C. D., & Huerta, T. (2006) · Implementation Science · doi:10.1186/1748-5908-1-20 · Cited by 68
Noguchi, K., Albarracín, D., Durantini, M. R., & Glasman, L. R. (2007) · Psychological Bulletin · doi:10.1037/0033-2909.133.6.955 · Cited by 67
Historical Roots of Dissemination and Implementation Science
Dearing, J. W., & Kee, K. F. (2012) · Dissemination and Implementation Research in HealthTranslating Science to Practice · doi:10.1093/acprof:oso/9780199751877.003.0003 · Cited by 67 · cited 1×
“Though it has made tremendous strides in its relatively short life span, the field also continues to face formidable challenges in refining the conceptualization and measurement of the factors that affect implementation success [84].”
BackgroundBarriers to research utilisation among nurses in Turkey
Uysal, A., Temel, A. B., Ardahan, M., & Özkahraman, Ş. (2010) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.2010.03318.x · Cited by 64
de Vos, M. L. G., van der Veer, S. N., Graafmans, W. C., de Keizer, N. F., Jager, K. J., Westert, G. P., & van der Voort, P. H. J. (2010) · Implementation Science · doi:10.1186/1748-5908-5-52 · Cited by 63
Staff Perception of Barriers to Behavior Therapy at a Psychiatric Hospital
Corrigan, P. W., Kwartarini, W. Y., & Pramana, W. (1992) · Behavior Modification · doi:10.1177/01454455920161007 · Cited by 61
Adoption of Evidence-Based Practices among Substance Abuse Treatment Providers
Haug, N. A., Shopshire, M., Tajima, B., Gruber, V., & Guydish, J. (2008) · Journal of Drug Education · doi:10.2190/de.38.2.f · Cited by 59
Barriers to, and facilitators of, research utilisation: a survey of Hong Kong registered nurses
Thompson, D. R., Chau, J. P. C., & López, V. (2006) · International Journal of Evidence-Based Healthcare · doi:10.1111/j.1479-6988.2006.00036.x · Cited by 59
Good, L. R., & Nelson, D. A. (1973) · Psychological Reports · doi:10.2466/pr0.1973.33.2.551 · Cited by 58
Gagnon, M.-P., Sánchez, E., & Pons, J. M. (2006) · Implementation Science · doi:10.1186/1748-5908-1-8 · Cited by 57 · cited 1×
“, physician intention measure [53]).”
Bonetti, D., Johnston, M., Clarkson, J. E., Grimshaw, J., Pitts, N. B., Eccles, M., Steen, N., Thomas, R., Maclennan, G., Glidewell, L., et al. (2010) · Implementation Science · doi:10.1186/1748-5908-5-25 · Cited by 57
Klimes-Dougan, B., August, G. J., Lee, C.-Y. S., Realmuto, G. M., Bloomquist, M. L., Horowitz, J. L., & Eisenberg, T. L. (2009) · Professional Psychology: Research and Practice · doi:10.1037/a0014623 · Cited by 52
Strategic Planning for Research Use in Nursing Practice
Mullem, C. V., Burke, L. J., Dohmeyer, K., Farrell, M., Harvey, S., John, L. R., Kraly, C., Rowley, F., Sebern, M., Twite, K. A., et al. (1999) · JONA The Journal of Nursing Administration · doi:10.1097/00005110-199912000-00008 · Cited by 50
Research utilization among children's mental health providers
Barwick, M. A., Boydell, K. M., Stasiulis, E., Ferguson, H. B., Blase, K., & Fixsen, D. (2008) · Implementation Science · doi:10.1186/1748-5908-3-19 · Cited by 50
Kalichman, S. C., Cain, D., Eaton, L., Jooste, S., & Simbayi, L. C. (2011) · American Journal of Public Health · doi:10.2105/ajph.2011.300236 · Cited by 47 · cited 1×
“, alcohol use [29]) and motivation [30,31] often moderate the retention in and efficacy of behavioral risk reduction interventions.”
BackgroundGreen, L. A., Gorenflo, D. W., & Wyszewianski, L. (2002) · PubMed · Cited by 46
Fagan, A. A., Arthur, M. W., Hanson, K., Briney, J. S., & Hawkins, J. D. (2011) · Prevention Science · doi:10.1007/s11121-011-0226-5 · Cited by 46
Personal and organizational knowledge transfer: Implications for worklife engagement
Leiter, M. P., Day, A. L., Harvie, P., & Shaughnessy, K. (2007) · Human Relations · doi:10.1177/0018726706076025 · Cited by 46
Dückers, M. L., Wagner, C., & Groenewegen, P. P. (2008) · BMC Health Services Research · doi:10.1186/1472-6963-8-172 · Cited by 46
Green, L. A., Wyszewianski, L., Lowery, J. C., Kowalski, C. P., & Krein, S. L. (2007) · Implementation Science · doi:10.1186/1748-5908-2-41 · Cited by 44
Wang, W., Saldana, L., Brown, C. H., & Chamberlain, P. (2010) · Implementation Science · doi:10.1186/1748-5908-5-72 · Cited by 43
Lewis, C. C., & Simons, A. D. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-011-0348-x · Cited by 43 · cited 2×
“Organizational Readiness for Change [45] X X O: Institutional resources, Organizational climate, motivational readiness [45] None Mental Health/ Substance Abuse [78] Adoption”
BackgroundMedical-surgical nurses' utilization of research methods and products.
Ca, B., Blumberg, M., Js, C., CURRAN, J. A., Ag, F., Jm, J., McGrath, P., Mt, P., & Zr, W. (1994) · PubMed · Cited by 42
Research use in the care of older people: a survey among healthcare staff
Boström, A., Wallin, L., & Nordström, G. M. (2006) · International Journal of Older People Nursing · doi:10.1111/j.1748-3743.2006.00014.x · Cited by 42
Staff training to improve implementation and impact of behavioral rehabilitation programs
Corrigan, P. W., McCracken, S. G., Edwards, M., Kommana, S., & Simpatico, T. A. (1997) · Psychiatric Services · doi:10.1176/ps.48.10.1336 · Cited by 42
Scott, S. D., Grimshaw, J., Klassen, T. P., Nettel‐Aguirre, A., & Johnson, D. W. (2011) · Implementation Science · doi:10.1186/1748-5908-6-133 · Cited by 41
Barriers to Research Utilization Scale: psychometric properties of the Turkish version
Temel, A. B., Uysal, A., Ardahan, M., & Özkahraman, Ş. (2009) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2009.05162.x · Cited by 41
Bahtsevani, C., Willman, A., Khalaf, A., & Östman, M. (2008) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2007.00916.x · Cited by 41
Melnyk, B. M., Bullock, T., McGrath, J., Jacobson, D., Kelly, S., & Baba, L. (2010) · Journal of Perinatal & Neonatal Nursing · doi:10.1097/jpn.0b013e3181ce314b · Cited by 39
Impact of PubMed search filters on the retrieval of evidence by physicians
Shariff, S. Z., Sontrop, J. M., Haynes, R. B., Iansavichus, A. V., McKibbon, K. A., Wilczynski, N. L., Weir, M. A., Speechley, M. R., Thind, A., & Garg, A. X. (2012) · Canadian Medical Association Journal · doi:10.1503/cmaj.101661 · Cited by 38
A New Instrument for Measuring Clinician Satisfaction With Electronic Health Records
SOCKOLOW, P. S., Weiner, J. P., Bowles, K. H., & Lehmann, H. P. (2011) · CIN Computers Informatics Nursing · doi:10.1097/ncn.0b013e31821a1568 · Cited by 38
Synergizing expectation and execution for stroke communities of practice innovations
Poissant, L., Ahmed, S., Riopelle, R. J., Rochette, A., Lefebvre, H., & Radcliffe-Branch, D. (2010) · Implementation Science · doi:10.1186/1748-5908-5-44 · Cited by 33
Sladek, R., Bond, M. J., Huynh, L. T., Chew, D. P., & Phillips, P. A. (2008) · Implementation Science · doi:10.1186/1748-5908-3-23 · Cited by 32
Ofi, B. A., Sowunmi, L., Edet, O. B., & Anarado, N. (2008) · International Journal of Nursing Practice · doi:10.1111/j.1440-172x.2008.00684.x · Cited by 31
Lokker, C., Haynes, R. B., Wilczynski, N. L., McKibbon, K. A., & Walter, S. D. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000233 · Cited by 31
Jung, J., Nitzsche, A., Neumann, M., Wirtz, M., Kowalski, C., Wasem, J., Stieler-Lorenz, B., & Pfaff, H. (2010) · BMC Public Health · doi:10.1186/1471-2458-10-550 · Cited by 29 · cited 2×
“, willingness to implement worksite health promotion programs [67]), or other settings.”
BackgroundChabot, G., Godin, G., & Gagnon, M. (2010) · Implementation Science · doi:10.1186/1748-5908-5-93 · Cited by 28
Do diabetes nurse specialists utilise research evidence?
Humphris, D., O'Halloran, P., Littlejohns, P., Hamilton, S., & Fisher, S. (1999) · Practical Diabetes International · doi:10.1002/pdi.1960160213 · Cited by 27
Eccles, M., Hrisos, S., Francis, J., Steen, N., Bosch, M., & Johnston, M. (2009) · Implementation Science · doi:10.1186/1748-5908-4-24 · Cited by 26
Chan, K. S., Hsu, Y.-J., Lubomski, L. H., & Marsteller, J. A. (2011) · Implementation Science · doi:10.1186/1748-5908-6-115 · Cited by 25 · cited 1×
“, number of bloodstream infections [83]) alone or in combination with other implementation outcomes.”
BackgroundResearch utilization among dental hygienists in Sweden – a national survey
Öhrn, K., Olsson, C., & Wallin, L. (2005) · International Journal of Dental Hygiene · doi:10.1111/j.1601-5037.2005.00135.x · Cited by 24
van Dijk, L. J., Nelen, W. L., D'Hooghe, T. M., Dunselman, G. A., Hermens, R. P., Bergh, C., Nygren, K. G., Simons, A. H., de Sutter, P., Marshall, C., et al. (2011) · Implementation Science · doi:10.1186/1748-5908-6-7 · Cited by 24
THE ORGANIZATIONAL LEARNING SURVEY: A RE-EVALUATION OF UNIDIMENSIONALITY
Goh, S. C., Quon, T. K., & Cousins, J. B. (2007) · Psychological Reports · doi:10.2466/pr0.101.3.707-721 · Cited by 23
Community Characteristics and Implementation Factors Associated with Effective Systems of Care
Lunn, L. M., Heflinger, C. A., Wang, W., Greenbaum, P. E., Kutash, K., Boothroyd, R. A., & Friedman, R. M. (2011) · The Journal of Behavioral Health Services & Research · doi:10.1007/s11414-011-9244-0 · Cited by 16
Predictors of Research Utilization among Pediatric Occupational Therapists
Brown, T., Tseng, M., Casey, J., McDonald, R. L., & Lyons, C. (2009) · OTJR Occupational Therapy Journal of Research · doi:10.3928/15394492-20091022-01 · Cited by 16
Nurse Practitioners and Evidence-Based Nursing Practice
Butler, K. D. (2011) · Journal of Doctoral Nursing Practice · doi:10.1891/1939-2095.4.1.53 · Cited by 15
Norton, W. E. (2012) · Implementation Science · doi:10.1186/1748-5908-7-27 · Cited by 14
The Team Checkup Tool: Evaluating QI Team Activities and Giving Feedback to Senior Leaders
Lubomski, L. H., Marsteller, J. A., Hsu, Y., Goeschel, C. A., Holzmueller, C. G., & Pronovost, P. J. (2008) · The Joint Commission Journal on Quality and Patient Safety · doi:10.1016/s1553-7250(08)34078-1 · Cited by 12
Donat, D. C. (2001) · Psychiatric Rehabilitation Journal · doi:10.1037/h0095026 · Cited by 8
Smith, J. B., Lacey, S. R., Williams, A. R., Teasley, S. L., Olney, A., Hunt, C., Cox, K. S., & Kemper, C. (2011) · JONA The Journal of Nursing Administration · doi:10.1097/nna.0b013e31821c4634 · Cited by 7
(1998) · doi:10.1002/(sici)1099-1379(199805)19:3<235::aid-job837>3.0.co;2-c
Research works printed but not linked: 13
- Grimshaw J: Improving the scientific basis of health care research dissemination and implementation. 2007, Paper presented at the NIH Conference on Building the Sciences of Dissemination and Implementation in the Service of: Public Health
- Rogers EM: Diffusion of innovations 5th edn. 2003, New York: Free Press
- Peters MA, Harmsen M, Laurant MGH, Wensing M: Ruimte voor verandering? Knelpunten en mogelijkheden voor verbeteringen in de patiëntenzorg [Room for improvement? Barriers to and facilitators for improvement of patient care]. 2002, Nijmegen, the Netherlands: Centre for Quality of Care Research (WOK), Radboud University Nijmegen Medical Centre, Article included in review
- American Psychological Association: National Council on Measurement in Education, American Educational Research Association: Standards for educational and psychological testing. 1999, Washington, DC: American Educational Research Association
- Pain K, Hagler P, Warren S: Development of an instrument to evaluate the research orientation of clinical professionals. Can J Rehabil. 1996, 9: 93-100. Article included in review
- Thompson CJ: Extent and factors influencing research utilization among critical care nurses. 1997, Texas Woman's University, Unpublished doctoral dissertation
- Meissner HI: Proceedings from the 3rd Annual NIH Conference on the Science of Dissemination and Implementation: Methods and Measurement. 2010, Bethesda, MD
- Rogers EM: Diffusion of innovations 4th edn. 1995, New York: Free Press, 4
- Facione PA, Facione CF: The California Critical Thinking Disposition Inventory: Test manual. 1992, Millbrae, CA: California Academic Press, Article included in review
- Estrada N, Verran J: Nursing practice environments: Strengthening the future of health systems: learning organizations and evidence-based practice by RNs. Commun Nurs Res. 2007, 40: 187-Article included in review
- Fineout-Overholt E, Melnyk BM: Organizational Culture and Readiness for System-Wide Implementation of EBP (OCR-SIEP) scale. 2006, Gilbert, AZ: ARCC llc Publishing, Article included in review
- Melnyk BM, Fineout-Overholt E, Giggleman M: EBP organizational culture, EBP implementation, and intent to leave in nurses and health professionals. Commun Nurs Res. 2010, 43: 310-Article included in review
- James LR, Sells SB: Psychological climate. The situation: An interactional perspective. Edited by: Magnusson D. 1981, Hillsdale, NJ: Lawrence Erlbaum, Article included in review
Cited by
The updated consolidated framework for implementation research based on user feedback
Damschroder, L. J., Reardon, C. M., Widerquist, M. A. O., & Lowery, J. (2022) · Implementation Science · doi:10.1186/s13012-022-01245-0 · Cited by 4,452
Making sense of implementation theories, models and frameworks
Nilsen, P. (2015) · Implementation Science · doi:10.1186/s13012-015-0242-0 · Cited by 4,447
Greenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A'Court, C., Hinder, S., Fahy, N., Procter, R., & Shaw, S. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8775 · Cited by 2,887
“It identified some new literature on organizational-level routinization [4], but little new evidence on scale-up, spread, or sustainability—a finding that has been confirmed by other reviews since [5-9].”
An introduction to implementation science for the non-specialist
Bauer, M. S., Damschroder, L., Hagedorn, H., Smith, J., & Kilbourne, A. M. (2015) · BMC Psychology · doi:10.1186/s40359-015-0089-9 · Cited by 1,611
Pfadenhauer, L. M., Gerhardus, A., Mozygemba, K., Lysdahl, K. B., Booth, A., Hofmann, B., Wahlster, P., Polus, S., Burns, J., Brereton, L., et al. (2017) · Implementation Science · doi:10.1186/s13012-017-0552-5 · Cited by 1,016
“The success of implementation is highly dependent on the buy-in of individuals who become key stakeholders in both the intervention and the implementation effort [89].”
Background
Metadata
- Authors
- Chaudoir, S. R., Dugan, A. G., & Barr, C. H.
- Affiliations
- College of the Holy Cross (US)
- University of Connecticut (US)
- Corresponding author
- Chaudoir, Stephenie R
- Venue
- Implementation Science
- Published
- 2013-02-17
- Volume
- 8
- Issue
- 1
- Pages
- 22-22
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC3598720
- OpenAlex
- W2143900697
- Semantic Scholar
- 9193bae8cbdc5da77d5f18b44dced4be2b9f818a
- Cited by
- 800
- References
- 165
- Citations by year
- 2026: 25 · 2025: 33 · 2024: 56 · 2023: 60 · 2022: 62 · 2021: 96 · 2020: 88 · 2019: 68 · 2018: 76 · 2017: 78 · 2016: 38 · 2015: 53 · 2014: 49 · 2013: 17
- FWCI
- 6.30
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Implementation Science (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- OpenCommons - UConn (University of Connecticut) (submittedVersion)
- Funders
- University of Connecticut
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- implementation research, implementation outcomes, implementation measures, criterion validity, structural factors, organizational factors, provider factors, patient factors, multi-level framework, fidelity
- MeSH terms
- Delivery of Health Care, Diffusion of Innovation, Health Services Research, Humans, Organizational Innovation, Evidence-Based Practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

