Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda
Open accessThe DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index.
Abstract
An unresolved issue in the field of implementation research is how to conceptualize and evaluate successful implementation. This paper advances the concept of "implementation outcomes" distinct from service system and clinical treatment outcomes. This paper proposes a heuristic, working "taxonomy" of eight conceptually distinct implementation outcomes-acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration, and sustainability-along with their nominal definitions. We propose a two-pronged agenda for research on implementation outcomes. Conceptualizing and measuring implementation outcomes will advance understanding of implementation processes, enhance efficiency in implementation research, and pave the way for studies of the comparative effectiveness of implementation strategies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the failure of most implementation processes to incorporate ongoing, iterative intervention development increases the risk that local users will find new technologies to be low on key implementation outcomes such as acceptability, feasibility, and appropriateness (Proctor et al., 2011) and may limit effectiveness (Chambers et al., 2013)”
From User‐centered design for psychosocial intervention development and implementation. (2016) · cites this paper as Proctor et al., 2011
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Increasing use of implementation frameworks may offer a more systematic evaluation of newborn care packages while clarifying equitable service delivery in vulnerable populations often present in LMIC settings (35, 36).”
From Born too soon in a resource-limited setting: A 10-year mixed methods review of a special care baby unit for refugees and migrants on the Myanmar-Thailand border (2023) · cites this paper as [36]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While reporting systems like those by Proctor [1] or Pinnock [75] would optimize learning across studies, few manuscripts specify strategies as recommended [76, 77].”
From A systematic review of experimentally tested implementation strategies across health and human service settings: evidence from 2010-2022 (2024) · cites this paper as [76]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“implementation outcomes are influenced by a complex interaction of contextual, organizational, workforce, policy, and system-level factors [12,13,14].”
From A Scoping Review of Implemented Innovations in Cancer Care: Implications for Pan-Canadian Scaling (2026) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These factors align closely with established implementation science frameworks that emphasize the importance of organizational readiness, implementation climate, leadership engagement, and continuous learning systems [11,13,14].”
From A Scoping Review of Implemented Innovations in Cancer Care: Implications for Pan-Canadian Scaling (2026) · cites this paper as [14]
Experimental — accuracy not yet measured
39 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rogers, E. M., Singhal, A., & Quinlan, M. M. (2019) · An Integrated Approach to Communication Theory and Research · doi:10.4324/9780203710753-35 · Cited by 37,322
Crossing the Quality Chasm: A New Health System for the 21st Century
Baker, A. (2001) · BMJ · doi:10.1136/bmj.323.7322.1192 · Cited by 10,736
Crossing the Quality Chasm: A New Health System for the 21st Century
Committee on Quality of Health Care in America (2002) · Journal For Healthcare Quality · doi:10.1111/j.1945-1474.2002.tb00463.x · Cited by 8,483
Evaluating the public health impact of health promotion interventions: the RE-AIM framework.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999) · American Journal of Public Health · doi:10.2105/ajph.89.9.1322 · Cited by 7,217 · cited 2×
“The literature reflects quite varied uses of the term ‘‘sustainability,’’ but our proposed definition incorporates aspects of those offered by Johnson et al. (2004), Turner and Sanders (2006), Glasgow et al. (1999) , Goodman et al. (1993), and Rabin et al. (2008).”
Background(2005) · Cited by 3,748
Crossing the Quality Chasm: A New Health System for the 21st Century
Bloom, B. S. (2002) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.287.5.646-a · Cited by 3,726
Making psychological theory useful for implementing evidence based practice: a consensus approach
Michie, S. (2005) · Quality and Safety in Health Care · doi:10.1136/qshc.2004.011155 · Cited by 3,440
Gluesing, J. C. (2012) · Leadership in Science and Technology: A Reference Handbook · doi:10.4135/9781412994231.n14 · Cited by 3,210
Davis, F. D. (1993) · International Journal of Man-Machine Studies · doi:10.1006/imms.1993.1022 · Cited by 2,964 · cited 1×
“Lack of acceptability has long been noted as a challenge in implementation (Davis 1993).”
BackgroundAldenderfer, M. S., & Blashfield, R. K. (1984) · Cited by 2,358
The Challenge of Innovation Implementation
Klein, K. J., & Sorra, J. S. (1996) · The Academy of Management Review · doi:10.2307/259164 · Cited by 2,113 · cited 1×
“These perceptions may be function of the organization’s culture or climate (Klein and Sorra 1996).”
BackgroundThe Meaning of Translational Research and Why It Matters
Woolf, S. H. (2008) · JAMA · doi:10.1001/jama.2007.26 · Cited by 1,910 · cited 2×
“…Outcomes Our team’s observations of implementation suggest that implementation outcomes are themselves interrelated in dynamic and complex ways (Woolf 2008; Repenning 2002; Hovmand and Gillespie 2010; Klein and Knight 2005) and are likely to change throughout an agency’s process to adopt and…”
BackgroundProctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-008-0197-4 · Cited by 1,894 · cited 4×
“Level of Analysis for Outcomes Implementation of new treatments is an inherently multilevel enterprise, involving provider behavior, care organization, and policy (Proctor et al. 2009; Raghavan et al. 2008).”
BackgroundMethodInfluentialSelf-citationDisseminating Innovations in Health Care
Berwick, D. M. (2003) · JAMA · doi:10.1001/jama.289.15.1969 · Cited by 1,878 · cited 2×
“The diffusion of innovation literature posits that the implementation outcome, adoption of an EST, is a function of such factors as perceived need to do things differently (Rogers 1995) perception of the new treatment’s comparative advantage (Frambach and Schillewaert 2002; Henggeler et al. 2002) and as easy to unders…”
BackgroundDusenbury, L. (2003) · Health Education Research · doi:10.1093/her/18.2.237 · Cited by 1,671 · cited 2×
“Fidelity is defined as the degree to which an intervention was implemented as it was prescribed in the original protocol or as it was intended by the program developers (Dusenbury et al. 2003; Rabin et al. 2008).”
BackgroundMethodDane, A. V., & Schneider, B. H. (1998) · Clinical Psychology Review · doi:10.1016/s0272-7358(97)00043-3 · Cited by 1,557 · cited 2×
“The literature identifies five implementation fidelity dimensions including adherence, quality of delivery, program component differentiation, exposure to the intervention, and participant responsiveness or involvement (Mihalic 2004; Dane and Schneider 1998).”
BackgroundCOMMITTEE ON QUALITY OF HEALTH CARE IN AMERICA
Kohn, L. T., Corrigan, J. M., & Donaldson, M. S. (2000) · Cited by 1,540
Journal of General Internal Medicine
(2013) · Journal of General Internal Medicine · doi:10.1007/11606.1525-1497 · Cited by 1,283
Aarons, G. A. (2004) · Mental Health Services Research · doi:10.1023/b:mhsr.0000024351.12294.65 · Cited by 1,272 · cited 6×
“Aarons’ Evidence-Based Practice Attitude Scale (EBPAS) captures the acceptability of evidence-based mental health treatments among mental health providers (Aarons 2004). Aarons and Palinkas (2007) used semi-structured interviews to assess case managers’ acceptance of evidencebased practices in a child welfare setting.…”
BackgroundResultInfluentialSelf-citationFrambach, R. T., & Schillewaert, N. (2002) · Journal of Business Research · doi:10.1016/s0148-2963(00)00152-1 · Cited by 1,270 · cited 2×
“The diffusion of innovation literature posits that the implementation outcome, adoption of an EST, is a function of such factors as perceived need to do things differently (Rogers 1995) perception of the new treatment’s comparative advantage ( Frambach and Schillewaert 2002; Henggeler et al. 2002) and as easy to under…”
BackgroundSpecifying and reporting complex behaviour change interventions: the need for a scientific method
Michie, S., Fixsen, D., Grimshaw, J. M., & Eccles, M. P. (2009) · Implementation Science · doi:10.1186/1748-5908-4-40 · Cited by 976 · cited 4×
“Understanding how to achieve implementation outcomes requires the kind of work now underway by Michie et al. (2009) advance a taxonomy of implementation strategies and reflect their demonstrated effects.”
BackgroundResultInfluentialLuke, D. (2004) · doi:10.4135/9781412985147 · Cited by 792
A Glossary for Dissemination and Implementation Research in Health
Rabin, B. A., Brownson, R. C., Haire-Joshu, D., Kreuter, M. W., & Weaver, N. L. (2008) · Journal of Public Health Management and Practice · doi:10.1097/01.phh.0000311888.06252.bb · Cited by 676 · cited 4×
“Our definition is consistent with those proposed by Rabin et al. (2008) and Rye and Kimberly (2007).”
BackgroundResultInfluentialGrimshaw, J., Eccles, M., Thomas, R., MacLennan, G., Ramsay, C., Fraser, C., & Vale, L. (2006) · Journal of General Internal Medicine · doi:10.1007/s11606-006-0269-7 · Cited by 631 · cited 2×
“Our objective of advancing a taxonomy of implementation outcomes is comparable to the work of Michie et al. (2005, 2009), Grimshaw et al. (2006), the Cochrane group, and others who are working to develop taxonomies and common nomenclature for implementation strategies.”
BackgroundResultMadon, T., Hofman, K. J., Kupfer, L., & Glass, R. I. (2007) · Science · doi:10.1126/science.1150009 · Cited by 459
Implementation of Evidence-based Practice in Child Welfare: Service Provider Perspectives
Aarons, G. A., & Palinkas, L. A. (2007) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-007-0121-3 · Cited by 450 · cited 2×
“Aarons and Palinkas (2007) used semi-structured interviews to assess case managers’ acceptance of evidencebased practices in a child welfare setting.”
BackgroundSelf-citationBeyond usability: Designing effective technology implementation systems to promote patient safety
Karsh, B.-T. (2004) · Quality and Safety in Health Care · doi:10.1136/qshc.2004.010322 · Cited by 414 · cited 2×
“Feasibility is defined as the extent to which a new treatment, or an innovation, can be successfully used or carried out within a given agency or setting (Karsh 2004).”
BackgroundBuilding capacity and sustainable prevention innovations: a sustainability planning model
Johnson, K., Hays, C., Center, H., & Daley, C. (2004) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2004.01.002 · Cited by 411 · cited 2×
“The literature reflects quite varied uses of the term ‘‘sustainability,’’ but our proposed definition incorporates aspects of those offered by Johnson et al. (2004) , Turner and Sanders (2006), Glasgow et al. (1999), Goodman et al. (1993), and Rabin et al. (2008).”
BackgroundInnovation Implementation: Overcoming the Challenge
Klein, K. J., & Knight, A. P. (2005) · Current Directions in Psychological Science · doi:10.1111/j.0963-7214.2005.00373.x · Cited by 409
A Simulation-Based Approach to Understanding the Dynamics of Innovation Implementation
Repenning, N. P. (2002) · Organization Science · doi:10.1287/orsc.13.2.109.535 · Cited by 338
Newton, M. S., Estabrooks, C. A., Norton, P., Birdsell, J. M., Adewale, A. J., & Thornley, R. (2007) · Implementation Science · doi:10.1186/1748-5908-2-1 · Cited by 264 · cited 1×
“Michie et al. (2007) studied perceived difficulties implementing a mental health guideline, coding respondent descriptions of implementation difficulties as 0, 0.5, or 1.”
InfluentialKirkevold, M. (1997) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.1997.1997025977.x · Cited by 243
Measuring the Diffusion of Innovative Health Promotion Programs
Steckler, A., Goodman, R. M., McLeroy, K. R., Davis, S., & Koch, G. (1992) · American Journal of Health Promotion · doi:10.4278/0890-1171-6.3.214 · Cited by 238 · cited 1×
“Indeed Steckler et al. (1992) emphasize sustainability in terms of attaining long-term viability, as the final stage of the diffusion process during which innovations settle into organizations.”
BackgroundeHealth Evaluation and Dissemination Research
Glasgow, R. E. (2007) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.01.023 · Cited by 223 · cited 1×
“This paper advances the concept of ‘‘implementation outcomes’’ distinct from service system outcomes and clinical treatment outcomes (Proctor et al. 2009; Fixsen et al. 2005; Glasgow 2007a ).”
BackgroundRaghavan, R., Bright, C. L., & Shadoin, A. L. (2008) · Implementation Science · doi:10.1186/1748-5908-3-26 · Cited by 205 · cited 2×
“Level of Analysis for Outcomes Implementation of new treatments is an inherently multilevel enterprise, involving provider behavior, care organization, and policy (Proctor et al. 2009; Raghavan et al. 2008).”
BackgroundSelf-citationDevelopment of Level of Institutionalization Scales for Health Promotion Programs
Goodman, R. M., McLeroy, K. R., Steckler, A. B., & Hoyle, R. H. (1993) · Health Education Quarterly · doi:10.1177/109019819302000208 · Cited by 181
Turner, K. M. T., & Sanders, M. R. (2006) · Aggression and Violent Behavior · doi:10.1016/j.avb.2005.07.005 · Cited by 180 · cited 1×
“The literature reflects quite varied uses of the term ‘‘sustainability,’’ but our proposed definition incorporates aspects of those offered by Johnson et al. (2004), Turner and Sanders (2006) , Glasgow et al. (1999), Goodman et al. (1993), and Rabin et al. (2008).”
BackgroundJones, K. (2015) · The Qualitative Report · doi:10.46743/2160-3715/2004.1939 · Cited by 167
Michie, S., Pilling, S., Garety, P., Whitty, P., Eccles, M. P., Johnston, M., & Simmons, J. (2007) · Implementation Science · doi:10.1186/1748-5908-2-8 · Cited by 161
The Adoption of Innovations by Provider Organizations in Health Care
Rye, C. B., & Kimberly, J. R. (2007) · Medical Care Research and Review · doi:10.1177/1077558707299865 · Cited by 160 · cited 2×
“Adoption also may be referred to as ‘‘uptake.’’ Our definition is consistent with those proposed by Rabin et al. (2008) and Rye and Kimberly (2007) .”
ResultTreatment adherence process research in family therapy: A rationale and some practical guidelines.
Hogue, A., Liddle, H. A., & Rowe, C. (1996) · Psychotherapy: Theory, Research, Practice, Training · doi:10.1037/0033-3204.33.2.332 · Cited by 143 · cited 1×
“Adherence, or the extent to which the therapy occurred as intended, is frequently examined in psychotherapy process and outcomes research and is distinguished from other potentially pertinent implementation factors such as provider skill or competence (Hogue et al. 1996).”
BackgroundUsability of a Computerized PAINReportIt in the General Public with Pain and People with Cancer Pain
Wilkie, D. J., Judge, M. K. M., Berry, D. L., Dell, J., Zong, S., & Gilespie, R. (2003) · Journal of Pain and Symptom Management · doi:10.1016/s0885-3924(02)00638-3 · Cited by 142 · cited 1×
“For example, Wilkie et al. (2003) begin by describing the measurement of ‘‘usability’’ (of a computerized innovation), including its ‘‘acceptability’’ to clients but later use the findings to conclude that the innovation was feasible.”
BackgroundHenggeler, S. W., Chapman, J. E., Rowland, M. D., Halliday-Boykins, C. A., Randall, J., Shackelford, J., & Schoenwald, S. K. (2008) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.76.4.556 · Cited by 133 · cited 2×
“Haug et al. (2008) used pre-post items to capture substance abuse providers’ adoption of evidence-based practices, while Henggeler et al. (2008) report interview techniques to measure therapists’ adoption of contingency management.”
BackgroundMethodChamberlain, P., Brown, C. H., Saldana, L., Reid, J., Wang, W., Marsenich, L., Sosna, T., Padgett, C., & Bouwman, G. (2008) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-008-0167-x · Cited by 100 · cited 1×
“…has identified ten steps for the implementation of an evidence-based treatment, Multidimensional Treatment Foster Care (MTFC), beginning with consideration of adopting MTFC and concluding when a service site meets certification criteria for delivering the treatment (Chamberlain et al. 2008).”
Distinctions between treatment outcomes and their implications for treatment evaluation.
Rosen, A., & Proctor, E. K. (1981) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.49.3.418 · Cited by 98 · cited 2×
“And third, they are key intermediate outcomes (Rosen and Proctor 1981) in relation to service system or clinical outcomes in treatment effectiveness and quality of care research.”
BackgroundSelf-citationApplied Epidemiology: Theory to Practice
Muntaner, C., Brownson, R. C., & Петити, Д. (2000) · Journal of Public Health Policy · doi:10.2307/3343476 · Cited by 88
Fidelity of supported employment: Lessons learned from the National Evidence-Based Practice Project.
Bond, G. R., McHugo, G. J., Becker, D. R., Rapp, C. A., & Whitley, R. (2008) · Psychiatric Rehabilitation Journal · doi:10.2975/31.4.2008.300.305 · Cited by 86 · cited 1×
“Other examples from the mental health literature include Bond et al. (2008) 15-item Supported Employment Fidelity Scale (SE Fidelity Scale) and Hogue et al. (2008) Therapist Behavior Rating Scale-Competence (TBRS-C), an observational measure of fidelity in evidence based practices for adolescent…”
MethodCounselor assessments of training and adoption barriers
Bartholomew, N. G., Joe, G. W., Rowan-Szal, G. A., & Simpson, D. D. (2007) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2007.01.005 · Cited by 86
Assessing fidelity in individual and family therapy for adolescent substance abuse
Hogue, A., Dauber, S., Chinchilla, P., Fried, A., Henderson, C., Inclan, J., Reiner, R. H., & Liddle, H. A. (2008) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2007.09.002 · Cited by 78
Adair, J. C. (1990) · Cited by 71
Implementation of Evidence-Based Practice and Organizational Performance
Hovmand, P. S., & Gillespie, D. F. (2008) · The Journal of Behavioral Health Services & Research · doi:10.1007/s11414-008-9154-y · Cited by 69 · cited 2×
“Our team’s observations of implementation suggest that implementation outcomes are themselves interrelated in dynamic and complex ways (Woolf 2008; Repenning 2002; Hovmand and Gillespie 2010; Klein and Knight 2005) and are likely to change throughout an agency’s process to”
BackgroundSelf-citationShumway, M., Saunders, T., Shern, D., Pines, E., Downs, A., Burbine, T., & Beller, J. (2003) · Psychiatric Services · doi:10.1176/appi.ps.54.8.1124 · Cited by 69 · cited 2×
“Achieving and measuring fidelity in usual care is beset by a number of challenges (Proctor et al. 2009; Mihalic 2004; Schoenwald et al. 2005 ).”
BackgroundCost-Efficacy of Individual and Combined Treatments for Panic Disorder
McHugh, R. K., Otto, M. W., Barlow, D. H., Gorman, J. M., Shear, K., & Woods, S. W. (2007) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v68n0710 · Cited by 68 · cited 2×
“These cost estimations are combined with patient outcomes and used in cost-effectiveness studies (McHugh et al. 2007).”
BackgroundDistinctions between treatment outcomes and their implications for treatment evaluation.
Rosen, A., & Proctor, E. K. (1981) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.49.3.418 · Cited by 68
Self-citationAdoption of Evidence-Based Practices among Substance Abuse Treatment Providers
Haug, N. A., Shopshire, M., Tajima, B., Gruber, V., & Guydish, J. (2008) · Journal of Drug Education · doi:10.2190/de.38.2.f · Cited by 59
Predicting Therapist Adherence to a Transported Family-Based Treatment for Youth
Schoenwald, S. K., Letourneau, E. J., & Halliday-Boykins, C. (2005) · Journal of Clinical Child & Adolescent Psychology · doi:10.1207/s15374424jccp3404_8 · Cited by 53 · cited 1×
“Achieving and measuring fidelity in usual care is beset by a number of challenges (Proctor et al. 2009; Mihalic 2004; Schoenwald et al. 2005).”
BackgroundUptake of Electronic Prescribing in Community-Based Practices
Fischer, M. A., Vogeli, C., Stedman, M. R., Ferris, T. G. G., & Weissman, J. S. (2008) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0383-1 · Cited by 47 · cited 2×
“The literature suggests that studies usually capture fidelity during initial implementation, while adoption is often assessed at 6 (Waldorff et al. 2008), 12 (Adily et al. 2004; Fischer et al. 2008 ), or 18 months (Cooke et al. 2001) after initial implementation.”
BackgroundKarlsson, A., & Bendtsen, P. (2004) · Addictive Behaviors · doi:10.1016/j.addbeh.2004.08.022 · Cited by 44 · cited 2×
“Karlsson and Bendtsen (2005) measured patients’ acceptance of alcohol screening in an emergency department setting using a 12-item questionnaire.”
MethodService penetration by persons with severe mental illness: How should it be measured?
Stiles, P. G., Boothroyd, R. A., Snyder, K., & Zong, X. (2002) · The Journal of Behavioral Health Services & Research · doi:10.1007/bf02287706 · Cited by 42 · cited 4×
“This definition is similar to (Stiles et al. 2002) notion of service penetration and to Rabin et al.s’ (2008) notion of niche saturation.”
BackgroundMethodInfluentialHides, L. M., Lubman, D. I., Elkins, K., Catania, L., & Rogers, N. (2007) · Drug and Alcohol Review · doi:10.1080/09595230701499126 · Cited by 37
Development of Level of Institutionalization Scales for Health Promotion Programs [and] Commentary.
Goodman, R. M. (1993) · Health Education Quarterly · Cited by 36
What happens after the innovation is identified?
Henggeler, S. W., & Lee, T. (2002) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.9.2.191 · Cited by 36 · cited 1×
“…that the implementation outcome, adoption of an EST, is a function of such factors as perceived need to do things differently (Rogers 1995) perception of the new treatment’s comparative advantage (Frambach and Schillewaert 2002; Henggeler et al. 2002) and as easy to understand (Berwick 2003).”
BackgroundOntario doctors’ attitudes toward and use of clinical practice guidelines in oncology
Graham, I. D., Brouwers, M. C., Davies, C., & Tetroe, J. (2006) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2006.00670.x · Cited by 27 · cited 2×
“For example, Graham et al. (2007) assessed doctors’ attitudes and perceptions toward clinical practice guidelines with a survey that tapped all three of these outcomes, although none of them were explicitly labeled as such: acceptability (e.g.”
BackgroundCosts of the 'Hartslag Limburg' community heart health intervention
Ronckers, E. T., Groot, W. N. J., Steenbakkers, M. W. H. L., Ruland, E., & Ament, A. J. H. A. (2006) · BMC Public Health · doi:10.1186/1471-2458-6-51 · Cited by 26
Cooke, M., Mattick, R. P., & Walsh, R. A. (2000) · Drug and Alcohol Review · doi:10.1080/09595230124432 · Cited by 25 · cited 2×
“The literature suggests that studies usually capture fidelity during initial implementation, while adoption is often assessed at 6 (Waldorff et al. 2008), 12 (Adily et al. 2004; Fischer et al. 2008), or 18 months ( Cooke et al. 2001 ) after initial implementation.”
BackgroundWaldorff, F. B., Steenstrup, A. P., Nielsen, B., Rubak, J. M., & Bro, F. (2008) · BMC Family Practice · doi:10.1186/1471-2296-9-24 · Cited by 19
Outcome and experience of implementing spinal muscular atrophy carrier screening on sperm donors
Callum, P., Iger, J., Ray, M., Sims, C. A., & Falk, R. E. (2010) · Fertility and Sterility · doi:10.1016/j.fertnstert.2009.12.071 · Cited by 17 · cited 2×
“A review of literature on guideline implementation in professions allied to medicine notes that few studies report anything about the costs of guideline implementation (Callum et al. 2010).”
BackgroundGunderman, R. B., & Meesa, I. R. (2008) · Radiology · doi:10.1148/radiol.2463061859 · Cited by 13
Use of on-line evidence databases by Australian public health practitioners
Adily, A., Westbrook, J. I., Coiera, E. W., & Ward, J. (2004) · Medical Informatics and the Internet in Medicine · doi:10.1080/14639230410001723437 · Cited by 11 · cited 1×
“The literature suggests that studies usually capture fidelity during initial implementation, while adoption is often assessed at 6 (Waldorff et al. 2008), 12 (Adily et al. 2004; Fischer et al. 2008), or 18 months (Cooke et al. 2001) after initial implementation.”
BackgroundCooke, M., Mattick, R. P., & Walsh, R. A. (2000) · Drug and Alcohol Review · doi:10.1080/09595230020029536 · Cited by 9
What Happens After the Innovation Is Identified?
Henggeler, S. W. (2002) · Clinical Psychology Science and Practice · doi:10.1093/clipsy/9.2.191 · Cited by 8
Systematic Reviews in Public Health
Mcpheeters, M. L., Briss, P., Teutsch, S. J., & Truman, B. (2006) · Applied Epidemiology · doi:10.1093/acprof:oso/9780195187410.003.0004 · Cited by 6 · cited 2×
“According to McPheeters et al. (2006), narrative reviews are best conducted by a team.”
BackgroundMethod
Research works printed but not linked: 6
- 5. Bates, D. M., & Sarkar, D. (2007). lme4: Linear mixed-effects models using S4 classes. R Package Version, 0, 99875-99876.
- 18. Glasgow, R. E. (2007b). The RE-AIM model for planning, evaluation and reporting on implementation and dissemination research. Paper presented at the NIH Conference on Building the Science of Dissemination and Implementation in the Service of Public Health: Bethesda, MD.
- 24. Hayes, S. C. (1998). President's message: Dissemination research now. The Behavior Therapist, 21(9), 166-169.
- Institute of Medicine Committee on Crossing the Quality Chasm. (2006). Adaption to mental health and addictive disorder: Improving the quality of health care for mental and substance-use conditions. Washington, DC: Institute of Medicine, National Academies Press.
- 33. Jones, K. (1997). Mission drift in qualitative research, or moving toward a systematic review of qualitative studies. The Qualita- tive Report, 9(1), 95-112.
- 45. Mihalic, S. (2004). The importance of implementation fidelity. Emotional and Behavioral Disorders in Youth, 4, 83-86.
Cited by
The updated consolidated framework for implementation research based on user feedback
Damschroder, L. J., Reardon, C. M., Widerquist, M. A. O., & Lowery, J. (2022) · Implementation Science · doi:10.1186/s13012-022-01245-0 · Cited by 4,452
“The CFIR Outcomes Addendum provides high-level guidance for identifying implementation outcomes by drawing on other frameworks [81, 82].”
BackgroundMaking sense of implementation theories, models and frameworks
Nilsen, P. (2015) · Implementation Science · doi:10.1186/s13012-015-0242-0 · Cited by 4,447
Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0327-7 · Cited by 3,240
Self-citationPsychometric assessment of three newly developed implementation outcome measures
Weiner, B. J., Lewis, C. C., Stanick, C., Powell, B. J., Dorsey, C. N., Clary, A. S., Boynton, M. H., & Halko, H. (2017) · Implementation Science · doi:10.1186/s13012-017-0635-3 · Cited by 2,819
“This study contributes to the literature by developing valid and reliable measures of important implementation outcomes [1].”
BackgroundMethodInfluentialImplementation strategies: recommendations for specifying and reporting
Proctor, E. K., Powell, B. J., & McMillen, J. C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-139 · Cited by 2,585
“[47,49,50], and we direct readers to the dissemination and implementation section of the Grid Enabled Measurement Initiative and the Seattle Implementation Research Collaborative’s measures project for repositories of implementation outcomes [82,83].”
BackgroundMethodSelf-citation
Metadata
- Authors
- Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M.
- Affiliations
- Washington University in St. Louis (US)
- Roberts Wesleyan College (US)
- Corresponding author
- Proctor, Enola
- Venue
- Administration and Policy in Mental Health and Mental Health Services Research
- Published
- 2010-10-19
- Volume
- 38
- Issue
- 2
- Pages
- 65-76
- ISSN
- 0894-587X (ISSN-L)
- 1573-3289
- PMC
- PMC3068522
- OpenAlex
- W2140092196
- Semantic Scholar
- 3c7cf782e4094a7d66d8ad5da80a8083a8e72307
- Cited by
- 8,721
- References
- 71
- Citations by year
- 2026: 1487 · 2025: 1144 · 2024: 1303 · 2023: 1041 · 2022: 950 · 2021: 893 · 2020: 669 · 2019: 391 · 2018: 253 · 2017: 162 · 2016: 122 · 2015: 116 · 2014: 83 · 2013: 57 · 2012: 28
- FWCI
- 56.95
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc
- Locations
- Administration and Policy in Mental Health and Mental Health Services Research (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health (5p30mh068579-05)
- National Institute of Mental Health (P30-MH-068579, T32MH019960)
- National Center for Research Resources (UL1RR024992)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Health Sciences › Health Professions › General Health Professions › Mental Health and Patient Involvement
- Keywords
- implementation outcomes, implementation research, acceptability, adoption, appropriateness, feasibility, fidelity, implementation costs, penetration, sustainability, implementation strategies, mental health services
- MeSH terms
- Diffusion of Innovation, Humans, Mental Health Services, Outcome and Process Assessment, Health Care, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
