Multilevel Mechanisms of Implementation Strategies in Mental Health: Integrating Theory, Research, and Practice
Open accessAbstract
A step toward the development of optimally effective, efficient, and feasible implementation strategies that increase evidence-based treatment integration in mental health services involves identification of the multilevel mechanisms through which these strategies influence implementation outcomes. This article (a) provides an orientation to, and rationale for, consideration of multilevel mediating mechanisms in implementation trials, and (b) systematically reviews randomized controlled trials that examined mediators of implementation strategies in mental health. Nine trials were located. Mediation-related methodological deficiencies were prevalent and no trials supported a hypothesized mediator. The most common reason was failure to engage the mediation target. Discussion focuses on directions to accelerate implementation strategy development in mental health.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Current literature reflects an increasing number of studies testing the effectiveness of implementation strategies and the mechanisms that explain how these strategies may influence implementation outcomes.
“Current literature also reflects an increasing number of studies testing the effectiveness of implementation strategies and the mechanisms that explain how these strategies may influence implementation outcomes [25–31].”
From Ten years of implementation outcomes research: a scoping review (2023) · cites this paper as [25]
The Introduction reports a proliferation of trials testing implementation strategies in mental health settings, and the Methods identify nine trials that tested an intervening variable. But the Introduction says strategies rarely test theory-based mechanisms; the paper does not establish that studies testing those mechanisms are increasing in number. The indexed passages provide section labels but no page numbers. (LLM, conf=0.84)
Experimental — accuracy not yet measuredIt is not common that mechanisms of implementation strategies are even mentioned, much less tested.
“Unfortunately, it is not common that mechanisms are even mentioned, much less tested (63, 65, 66).”
From Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda (2019) · cites this paper as [63]
Advancing these priorities will lead to a better understanding of when, where, why, and how implementation strategies exert their effects.
“Advancing these priorities will lead to a better understanding of when, where, why, and how implementation strategies exert their effects (29, 63).”
From Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda (2019) · cites this paper as [63]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Baron, R. M., & Kenny, D. A. (1986) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.51.6.1173 · Cited by 91,542 · see citing works
Baron, R. M., & Kenny, D. A. (1986) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.51.6.1173 · Cited by 91,532 · see citing works
Preacher, K. J., & Hayes, A. F. (2008) · Behavior Research Methods · doi:10.3758/brm.40.3.879 · Cited by 34,117 · see citing works
Hierarchical Linear Models: Applications and Data Analysis Methods.
Longford, N. T., Bryk, A. S., & Raudenbush, S. W. (1993) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2075823 · Cited by 18,768 · see citing works
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496 · see citing works
Michie, S., van Stralen, M. M., & West, R. (2011) · Implementation Science · doi:10.1186/1748-5908-6-42 · Cited by 13,697 · see citing works
Asymptotic Confidence Intervals for Indirect Effects in Structural Equation Models
Sobel, M. E. (1982) · Sociological Methodology · doi:10.2307/270723 · Cited by 12,810 · see citing works
Research in Organizational Behavior
Hammer, E. R., Cummings, L. L., & Staw, B. M. (1984) · Academy of Management Review · doi:10.2307/258300 · Cited by 11,176 · see citing works
Crossing the Quality Chasm: A New Health System for the 21st Century
Baker, A. (2001) · BMJ · doi:10.1136/bmj.323.7322.1192 · Cited by 10,754 · see citing works
Addressing Moderated Mediation Hypotheses: Theory, Methods, and Prescriptions
Preacher, K. J., Rucker, D. D., & Hayes, A. F. (2007) · Multivariate Behavioral Research · doi:10.1080/00273170701341316 · Cited by 9,973 · see citing works
A comparison of methods to test mediation and other intervening variable effects
MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.1.83 · Cited by 9,406 · see citing works
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0319-7 · Cited by 8,721 · see citing works
Crossing the Quality Chasm: A New Health System for the 21st Century
Committee on Quality of Health Care in America (2002) · Journal for Healthcare Quality · doi:10.1111/j.1945-1474.2002.tb00463.x · Cited by 8,508 · see citing works
Confidence Limits for the Indirect Effect: Distribution of the Product and Resampling Methods
MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3901_4 · Cited by 7,806 · see citing works
Diffusion of Innovations in Service Organizations: Systematic Review and Recommendations
GREENHALGH, T., ROBERT, G., MACFARLANE, F., BATE, P., & KYRIAKIDOU, O. (2004) · The Milbank Quarterly · doi:10.1111/j.0887-378x.2004.00325.x · Cited by 7,448 · see citing works
MacKinnon, D. P., Fairchild, A. J., & Fritz, M. S. (2007) · Annual Review of Psychology · doi:10.1146/annurev.psych.58.110405.085542 · Cited by 5,688 · see citing works
Introduction to Statistical Mediation Analysis
MacKinnon, D. P. (2012) · doi:10.4324/9780203809556 · Cited by 5,618 · see citing works
Klein, K. J., & Kozlowski, S. W. J. (2000) · Cited by 4,914 · see citing works
Edwards, J. R., & Lambert, L. S. (2007) · Psychological Methods · doi:10.1037/1082-989x.12.1.1 · Cited by 4,711 · see citing works
A general multilevel SEM framework for assessing multilevel mediation
Preacher, K. J., Zyphur, M. J., & Zhang, Z. (2010) · Psychological Methods · doi:10.1037/a0020141 · Cited by 3,796 · see citing works
Crossing the Quality Chasm: A New Health System for the 21st Century
Bloom, B. S. (2002) · JAMA · doi:10.1001/jama.287.5.646-a · Cited by 3,732 · see citing works
Answers to 20 questions about interrater reliability and interrater agreement
LeBreton, J. M., & Senter, J. L. (2008) · Organizational Research Methods · doi:10.1177/1094428106296642 · Cited by 3,608 · cited 3× · see citing works
“…variables have potentially distinct meanings and empirical relationships with outcome variables at different levels of analysis and therefore must be validated and tested in analytic models using appropriate procedures (Klein et al. 1994; LeBreton and Senter 2008; Raudenbush and Bryk 2002).”
Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0327-7 · Cited by 3,240 · cited 4× · see citing works
“Compositional variables are common in implementation research and include such variables as organizational culture, EBT implementation climate, and organizational readiness for change (Aarons et al. 2011; Williams and Glisson 2014).”
InfluentialChan, D. K. (1998) · Journal of Applied Psychology · doi:10.1037/0021-9010.83.2.234 · Cited by 2,981 · see citing works
The global prevalence of common mental disorders: a systematic review and meta-analysis 1980–2013
Steel, Z., Marnane, C., Iranpour, C., Chey, T., Jackson, J. W., Patel, V., & Silove, D. (2014) · International Journal of Epidemiology · doi:10.1093/ije/dyu038 · Cited by 2,846 · see citing works
Mediators and Mechanisms of Change in Psychotherapy Research
Kazdin, A. E. (2007) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.3.022806.091432 · Cited by 2,504 · cited 6× · see citing works
“Mediation ‘‘knockout’’ studies in which the mediator is manipulated but its theorized action is blocked in one condition can provide especially strong evidence to support a hypothesized mediator (Kazdin 2007).”
InfluentialProcess Analysis: Estimating Mediation in Treatment Evaluations
Judd, C. M., & Kenny, D. A. (1981) · Evaluation Review · doi:10.1177/0193841x8100500502 · Cited by 2,442 · see citing works
Hayes, A. F., & Scharkow, M. (2013) · Psychological Science · doi:10.1177/0956797613480187 · Cited by 2,438 · see citing works
Grand challenges in global mental health
Collins, P. Y., Patel, V., Joestl, S. S., March, D., Insel, T. R., Daar, A. S., Bordin, I. A., Costello, E. J., Durkin, M., Fairburn, C., et al. (2011) · Nature · doi:10.1038/475027a · Cited by 2,043 · see citing works
Advantages of Monte Carlo confidence intervals for indirect effects
Preacher, K. J., & Selig, J. P. (2012) · Communication Methods and Measures · doi:10.1080/19312458.2012.679848 · Cited by 1,908 · see citing works
The global burden of mental disorders: An update from the WHO World Mental Health (WMH) surveys
Kessler, R. C., Aguilar-Gaxiola, S., Alonso, J., Chatterji, S., Lee, S., Ormel, J., Üstün, T. B., & Wang, P. S. (2009) · Epidemiologia e Psichiatria Sociale · doi:10.1017/s1121189x00001421 · Cited by 1,895 · see citing works
Leavitt, M. (2001) · PubMed · Cited by 1,835 · see citing works
Bauer, D. J., Preacher, K. J., & Gil, K. M. (2006) · Psychological Methods · doi:10.1037/1082-989x.11.2.142 · Cited by 1,704 · see citing works
Levels issues in theory development, data collection and analysis
Klein, K. J., Dansereau, F., & Hall, R. J. (1994) · The Academy of Management Review · doi:10.2307/258703 · Cited by 1,607 · see citing works
Coming to terms with the terms of risk
Kraemer, H. C. (1997) · Archives of General Psychiatry · doi:10.1001/archpsyc.1997.01830160065009 · Cited by 1,432 · see citing works
Bridging Research and Practice
Tabak, R. G., Khoong, E. C., Chambers, D. A., & Brownson, R. C. (2012) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2012.05.024 · Cited by 1,386 · see citing works
Multilevel Modeling of Individual and Group Level Mediated Effects
Krull, J. L., & MacKinnon, D. P. (2001) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3602_06 · Cited by 1,325 · see citing works
Kitson, A. L., Rycroft-Malone, J., Harvey, G., McCormack, B., Seers, K., & Titchen, A. (2008) · Implementation Science · doi:10.1186/1748-5908-3-1 · Cited by 1,314 · see citing works
Chen, H.-T. (1990) · Medical Entomology and Zoology · Cited by 1,227 · see citing works
Tests of the three-path mediated effect
Taylor, A. B., MacKinnon, D. P., & Tein, J.-Y. (2008) · Organizational Research Methods · doi:10.1177/1094428107300344 · Cited by 1,209 · see citing works
Testing Multilevel Mediation Using Hierarchical Linear Models
Zhang, Z., Zyphur, M. J., & Preacher, K. J. (2008) · Organizational Research Methods · doi:10.1177/1094428108327450 · Cited by 1,096 · see citing works
Godin, G., Bélanger-Gravel, A., Eccles, M., & Grimshaw, J. (2008) · Implementation Science · doi:10.1186/1748-5908-3-36 · Cited by 1,096 · see citing works
Planning and Studying Improvement in Patient Care: The Use of Theoretical Perspectives
GROL, R. P. T. M., BOSCH, M. C., HULSCHER, M. E. J. L., ECCLES, M. P., & WENSING, M. (2007) · The Milbank Quarterly · doi:10.1111/j.1468-0009.2007.00478.x · Cited by 943 · see citing works
The Effect of Different Forms of Centering in Hierarchical Linear Models
Kreft, I. G. G., de Leeuw, J., & Aiken, L. S. (1995) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3001_1 · Cited by 855 · see citing works
Berwick, D. M. (2008) · JAMA · doi:10.1001/jama.299.10.1182 · Cited by 809 · see citing works
Chaudoir, S. R., Dugan, A. G., & Barr, C. H. (2013) · Implementation Science · doi:10.1186/1748-5908-8-22 · Cited by 800 · see citing works
Lower level mediation in multilevel models.
Kenny, D. A., Korchmaros, J. D., & Bolger, N. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.2.115 · Cited by 687 · see citing works
Davies, P., Walker, A. E., & Grimshaw, J. M. (2010) · Implementation Science · doi:10.1186/1748-5908-5-14 · Cited by 619 · see citing works
Lipsey, M. W. (1991) · Evaluation and Program Planning · doi:10.1016/0149-7189(91)90027-e · Cited by 598 · see citing works
Between- and Within-Cluster Covariate Effects in the Analysis of Clustered Data
Neuhaus, J. M., & Kalbfleisch, J. D. (1998) · Biometrics · doi:10.2307/3109770 · Cited by 528 · see citing works
Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-032013-182447 · Cited by 474 · see citing works
Madon, T., Hofman, K. J., Kupfer, L. E., & Glass, R. I. (2007) · Science · doi:10.1126/science.1150009 · Cited by 460 · see citing works
Insel, T. R. (2009) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2008.540 · Cited by 459 · see citing works
Diffusion of a School-Based Substance Abuse Prevention Program: Predictors of Program Implementation
Rohrbach, L. A., Graham, J. A., & Hansen, W. B. (1993) · Preventive Medicine · doi:10.1006/pmed.1993.1020 · Cited by 431 · see citing works
Weisz, J. R., Kuppens, S., Eckshtain, D., Ugueto, A. M., Hawley, K. M., & Jensen-Doss, A. (2013) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2013.1176 · Cited by 412 · see citing works
Assessing Mediational Models: Testing and Interval Estimation for Indirect Effects
Biesanz, J. C., Falk, C. F., & Savalei, V. (2010) · Multivariate Behavioral Research · doi:10.1080/00273171.2010.498292 · Cited by 397 · see citing works
A framework for testing meso‐mediational relationships in Organizational Behavior
Mathieu, J. E., & Taylor, S. R. (2007) · Journal of Organizational Behavior · doi:10.1002/job.436 · Cited by 374 · see citing works
Novins, D. K., Green, A. E., Legha, R. K., & Aarons, G. A. (2013) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2013.07.012 · Cited by 270 · see citing works
A Systematic Review of Strategies for Implementing Empirically Supported Mental Health Interventions
Powell, B. J., Proctor, E. K., & Glass, J. E. (2014) · Research on Social Work Practice · doi:10.1177/1049731513505778 · Cited by 268 · see citing works
Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy.
Glisson, C., Schoenwald, S. K., Hemmelgarn, A., Green, P., Dukes, D., Armstrong, K. S., & Chapman, J. E. (2010) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0019160 · Cited by 243 · see citing works
Dissemination and Implementation of Evidence-Based Psychological Interventions
McHugh, R. K., & Barlow, D. H. (2015) · Oxford University Press eBooks · doi:10.1093/med:psych/9780195389050.003.0013 · Cited by 211 · see citing works
Changing the Way We Study Change in Psychotherapy
Doss, B. (2004) · Clinical Psychology Science and Practice · doi:10.1093/clipsy.bph094 · Cited by 188 · see citing works
Teacher key opinion leaders and mental health consultation in low-income urban schools.
Atkins, M. S., Frazier, S. L., Leathers, S. J., Graczyk, P. A., Talbott, E., Jakobsons, L., Adil, J. A., Marinez-Lora, A., Demirtas, H., Gibbons, R. B., et al. (2008) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0013036 · Cited by 174 · see citing works
Mediational Inferences in Organizational Research
Mathieu, J. E., DeShon, R. P., & Bergh, D. D. (2008) · Organizational Research Methods · doi:10.1177/1094428107310089 · Cited by 154 · see citing works
Baer, J. S., Wells, E. A., Rosengren, D. B., Hartzler, B. J., Beadnell, B. A., & Dunn, C. W. (2009) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2009.01.003 · Cited by 149 · see citing works
Dissemination of the Coping Power program: Importance of intensity of counselor training.
Lochman, J. E., Boxmeyer, C. L., Powell, N. P., Lixin Qu, A., Wells, K., & Windle, M. T. (2009) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0014514 · Cited by 146 · see citing works
Improving Community-Based Mental Health Care for Children: Translating Knowledge into Action
Garland, A. F., Haine‐Schlagel, R., Brookman-Frazee, L. I., Baker-Ericzén, M. J., Trask, E. V., & Fawley-King, K. (2012) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-012-0450-8 · Cited by 144 · see citing works
Dissemination and Implementation of Evidence-Based Practices in Child and Adolescent Mental Health
Beidas, R. S., Kendall, P. C., & McClelland, H. (2017) · Clinical Psychology Forum · doi:10.53841/bpscpf.2017.1.300.46 · Cited by 109 · see citing works
Kauth, M. R., Sullivan, G., Blevins, D., Cully, J. A., Landes, R. D., Said, Q., & Teasdale, T. A. (2010) · Implementation Science · doi:10.1186/1748-5908-5-75 · Cited by 98 · see citing works
Landsverk, J. A., Brown, C. H., Rolls-Reutz, J., Palinkas, L. A., & Horwitz, S. M. (2010) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0315-y · Cited by 89 · see citing works
Assessing and Changing Organizational Social Contexts for Effective Mental Health Services
Glisson, C. A., & Williams, N. J. (2015) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031914-122435 · Cited by 69 · see citing works
Self-citationCentering Predictors and Contextual Effects
Enders, C. K. (2013) · SAGE Publications Ltd eBooks · doi:10.4135/9781446247600.n6 · Cited by 66 · see citing works
Saloner, B. K., Carson, N. J., & Cook, B. L. (2014) · Medical Care Research and Review · doi:10.1177/1077558713518347 · Cited by 60 · see citing works
Three-Level Models for Indirect Effects in School- and Class-Randomized Experiments in Education
Pituch, K. A., Murphy, D. L., & Tate, R. L. (2009) · The Journal of Experimental Education · doi:10.1080/00220970903224685 · Cited by 58 · see citing works
Knowledge Translation Efforts in Child and Youth Mental Health: A Systematic Review
Barwick, M. A., Schachter, H., Bennett, L. M., McGowan, J., Ly, M., Wilson, A., BENNETT, K. J., Buchanan, D. H., Fergusson, D. A., & Manion, I. G. (2012) · Journal of Evidence-Based Social Work · doi:10.1080/15433714.2012.663667 · Cited by 35 · see citing works
Williams, N. J., & Glisson, C. A. (2014) · Cited by 30 · see citing works
Self-citationThe impact of pay-for-performance on therapists' intentions to deliver high-quality treatment
Garner, B. R., Godley, S. H., & Bair, C. M. L. (2011) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2011.01.012 · Cited by 22 · see citing works
Holth, P., Torsheim, T., Sheidow, A. J., Ogden, T., & Henggeler, S. W. (2011) · Journal of Child & Adolescent Substance Abuse · doi:10.1080/1067828x.2011.581974 · Cited by 21 · see citing works
Williams, J. R., Williams, W. O., Dusablon, T., Blais, M. P., Tregear, S., Banks, D., & Hennessy, K. D. (2013) · The Journal of Behavioral Health Services & Research · doi:10.1007/s11414-013-9369-4 · Cited by 12 · see citing works
The Reach of Evidence-Based Psychological Interventions
McHugh, R. K., & Barlow, D. H. (2015) · Oxford University Press eBooks · doi:10.1093/med:psych/9780195389050.003.0001 · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Williams NJ Mechanisms of change in an organizational culture and climate intervention for increasing clinicians’ evidence-based practice adoption in mental health (Unpublished doctoral dissertation) 2015 Knoxville University of Tennessee http://trace.tennessee.edu/utk_graddiss/3482.
Cited by
Sorted by citations
Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda
Powell, B. J., Fernandez, M. E., Williams, N. J., Aarons, G. A., Beidas, R. S., Lewis, C. C., McHugh, S. M., & Weiner, B. J. (2019) · Frontiers in Public Health · doi:10.3389/fpubh.2019.00003 · Cited by 775 · see citing works
Self-citationLewis, C. C., Klasnja, P., Powell, B. J., Lyon, A. R., Tuzzio, L., Jones, S., Walsh-Bailey, C., & Weiner, B. (2018) · Frontiers in Public Health · doi:10.3389/fpubh.2018.00136 · Cited by 588 · see citing works
Ten years of implementation outcomes research: a scoping review
Proctor, E. K., Bunger, A. C., Lengnick-Hall, R., Gerke, D. R., Martin, J. K., Phillips, R. J., & Swanson, J. C. (2023) · Implementation Science · doi:10.1186/s13012-023-01286-z · Cited by 424 · see citing works
“Current literature also reflects an increasing number of studies testing the effectiveness of implementation strategies and the mechanisms that explain how these strategies may influence implementation outcomes [25–31].”
BackgroundA systematic review of empirical studies examining mechanisms of implementation in health
Lewis, C. C., Boyd, M. R., Walsh-Bailey, C., Lyon, A. R., Beidas, R., Mittman, B., Aarons, G. A., Weiner, B. J., & Chambers, D. A. (2020) · Implementation Science · doi:10.1186/s13012-020-00983-3 · Cited by 363 · see citing works
Williams, N. J., & Beidas, R. S. (2019) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12960 · Cited by 233 · see citing works
Self-citationBoyd, M. R., Powell, B. J., Endicott, D., & Lewis, C. C. (2018) · Behavior Therapy · doi:10.1016/j.beth.2017.11.012 · Cited by 137 · see citing works
Powell, B. J., Mettert, K. D., Dorsey, C. N., Weiner, B. J., Stanick, C. F., Lengnick-Hall, R., Ehrhart, M. G., Aarons, G. A., Barwick, M. A., Damschroder, L. J., et al. (2021) · Implementation Research and Practice · doi:10.1177/26334895211018862 · Cited by 79 · see citing works
Lyon, A. R., Liu, F. F., Connors, E. H., King, K. M., Coifman, J. I., Cook, H., McRee, E., Ludwig, K., Law, A., Dorsey, S., et al. (2022) · Implementation Science Communications · doi:10.1186/s43058-022-00325-y · Cited by 28 · see citing works
Beidas, R. S., Volpp, K. G., Buttenheim, A. N., Marcus, S. C., Olfson, M., Pellecchia, M., Stewart, R. E., Williams, N. J., Becker-Haimes, E. M., Candon, M., et al. (2019) · JMIR Research Protocols · doi:10.2196/12121 · Cited by 28 · see citing works
Self-citationLewis, C. C., Stanick, C., Lyon, A., Darnell, D., Locke, J., Puspitasari, A., Marriott, B. R., Dorsey, C. N., Larson, M., Jackson, C., et al. (2018) · Implementation Science · doi:10.1186/s13012-018-0714-0 · Cited by 22 · see citing works
Motamedi, M., Lau, A. S., Lind, T., Lui, J. H., Rodriguez, A., Smith, A., & Brookman-Frazee, L. (2021) · Implementation Research and Practice · doi:10.1177/2633489520982903 · Cited by 17 · see citing works
Ten years of implementation outcome research: a scoping review protocol
Lengnick-Hall, R., Proctor, E. K., Bunger, A. C., & Gerke, D. R. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-049339 · Cited by 14 · see citing works
- Authors
- Williams, N. J.
- Affiliations
- Boise State University (US)
- Corresponding author
- Williams, Nathaniel J.
- Venue
- Administration and Policy in Mental Health and Mental Health Services Research
- Published
- 2015-10-16
- Volume
- 43
- Issue
- 5
- Pages
- 783-798
- ISSN
- 0894-587X (ISSN-L)
- 1573-3289
- PMC
- PMC4834058
- OpenAlex
- W2284982684
- Semantic Scholar
- 69db89a38d1cda11ec0579716e79c3427877cb3e
- Cited by
- 164
- References
- 80
- Citations by year
- 2026: 9 · 2025: 17 · 2024: 29 · 2023: 12 · 2022: 26 · 2021: 26 · 2020: 16 · 2019: 12 · 2018: 9 · 2017: 5 · 2016: 1 · 2015: 1
- FWCI
- 0.96
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Administration and Policy in Mental Health and Mental Health Services Research (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Scholar Works (Boise State University) (submittedVersion)
- Funders
- National Institute of Mental Health (F31MH099846)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- implementation strategies, multilevel mediation, implementation outcomes, mental health services, implementation research
- MeSH terms
- Delivery of Health Care, Humans, Mental Health Services, Evidence-Based Practice, Multilevel Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
