Mechanisms of Change in the ARC Organizational Strategy: Increasing Mental Health Clinicians’ EBP Adoption Through Improved Organizational Culture and Capacity
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
The development of efficient and scalable implementation strategies in mental health is restricted by poor understanding of the change mechanisms that increase clinicians’ evidence-based practice (EBP) adoption. This study tests the cross-level change mechanisms that link an empirically-supported organizational strategy for supporting implementation (labeled ARC for Availability, Responsiveness, and Continuity) to mental health clinicians’ EBP adoption and use. Four hundred seventy-five mental health clinicians in 14 children’s mental health agencies were randomly assigned to the ARC intervention or a control condition. Measures of organizational culture, clinicians’ intentions to adopt EBPs, and job-related EBP barriers were collected before, during, and upon completion of the three-year ARC intervention. EBP adoption and use were assessed at 12-month follow-up. Multilevel mediation analyses tested changes in organizational culture, clinicians’ intentions to adopt EBPs, and job-related EBP barriers as linking mechanisms explaining the effects of ARC on clinicians’ EBP adoption and use. ARC increased clinicians’ EBP adoption (OR = 3.19, p = .003) and use (81 vs. 56 %, d = .79, p = .003) at 12-month follow-up. These effects were mediated by improvement in organizational proficiency culture leading to increased clinician intentions to adopt EBPs and by reduced job-related EBP barriers. A combined mediation analysis indicated the organizational culture-EBP intentions mechanism was the primary carrier of ARC’s effects on clinicians’ EBP adoption and use. ARC increases clinicians’ EBP adoption and use by creating proficient organizational cultures that increase clinicians’ intentions to adopt EBPs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The theory of planned behavior
Ajzen, I. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90020-t · Cited by 90,587
Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.5.879 · Cited by 82,194
Michie, S., van Stralen, M. M., & West, R. (2011) · Implementation Science · doi:10.1186/1748-5908-6-42 · Cited by 13,697
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0319-7 · Cited by 8,721 · cited 3×
“Guided by the taxonomy of implementation outcomes proposed by Proctor et al. (2010) EBP adoption was conceptualized in this study as any action to try or employ an EBP including a clinician’s initial implementation of an EBP.”
InfluentialConfidence Limits for the Indirect Effect: Distribution of the Product and Resampling Methods
MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3901_4 · Cited by 7,806 · cited 2×
“Monte Carlo methods have been recommended by methodologists because they provide more accurate confidence limits and exhibit higher statistical power (while maintaining Type I error rates) than normal theory approaches such as the Sobel test (Hayes and Scharkow 2013; MacKinnon et al. 2004).”
Intention—Behavior Relations: A Conceptual and Empirical Review
Sheeran, P. (2002) · European Review of Social Psychology · doi:10.1080/14792772143000003 · Cited by 3,688
Answers to 20 questions about interrater reliability and interrater agreement
LeBreton, J. M., & Senter, J. L. (2008) · Organizational Research Methods · doi:10.1177/1094428106296642 · Cited by 3,608 · cited 2×
“7 is recommended to support aggregation to the unit level (LeBreton and Senter 2008).”
Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0327-7 · Cited by 3,240 · cited 2×
“For example, implementation models such as the Exploration, Preparation/Adoption, Implementation, and Sustainment model proposed by Aarons et al. (2011) suggest adoption occurs relatively early in the implementation process following an exploration phase and preceding active implementation and…”
Preacher, K. J., & Kelley, K. (2011) · Psychological Methods · doi:10.1037/a0022658 · Cited by 3,060 · cited 1×
“The ratio of ARC’s indirect effect to its total effect was PM = .88, suggesting improved proficiency culture accounted for 88 % of ARC’s total effect on clinicians’ EBP intentions (Preacher and Kelley 2011).”
rwg: an assessment of within-group interrater agreement
James, L. R., Demaree, R. G., & Wolf, G. (1993) · Journal of Applied Psychology · doi:10.1037/0021-9010.78.2.306 · Cited by 2,221 · cited 1×
“Clinicians’ responses to the proficiency culture items were aggregated to the agency level based on a referentshift consensus composition model (Chan 1998) and evidence of adequate within-agency agreement using the rwg(j) index (James et al. 1993).”
MethodGrand challenges in global mental health
Collins, P. Y., Patel, V., Joestl, S. S., March, D., Insel, T. R., Daar, A. S., Bordin, I. A., Costello, E. J., Durkin, M., Fairburn, C., et al. (2011) · Nature · doi:10.1038/475027a · Cited by 2,043 · cited 2×
“Significant deficits remain in the development of implementation strategies that increase the use of evidence-based practices (EBPs) in routine mental health care (Collins et al. 2011; Insel 2009; NIMH 2008).”
Advantages of Monte Carlo confidence intervals for indirect effects
Preacher, K. J., & Selig, J. P. (2012) · Communication Methods and Measures · doi:10.1080/19312458.2012.679848 · Cited by 1,908
Tests of the three-path mediated effect
Taylor, A. B., MacKinnon, D. P., & Tein, J.-Y. (2008) · Organizational Research Methods · doi:10.1177/1094428107300344 · Cited by 1,209
Chaudoir, S. R., Dugan, A. G., & Barr, C. H. (2013) · Implementation Science · doi:10.1186/1748-5908-8-22 · Cited by 800
Glisson, C., Landsverk, J., Schoenwald, S., Kelleher, K., Hoagwood, K. E., Mayberg, S., Green, P., & The Research Network on Youth Mental Health (2008) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-007-0148-5 · Cited by 608 · cited 6×
“…health agencies’ cultural norms and expectations (Glisson et al. 2008a, 2012a) and strong associations between organizational culture and clinicians’ practice attitudes, behaviors, and clinical outcomes (Aarons et al. 2012; Glisson et al. 2008b; Olin et al. 2014; Williams and Glisson 2013, 2014a).”
InfluentialSelf-citationEhrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2014) · Implementation Science · doi:10.1186/s13012-014-0157-1 · Cited by 502 · cited 2×
“For example, the EBP implementation climate construct focuses more narrowly on the implementation of EBPs (Ehrhart et al. 2014).”
Insel, T. R. (2009) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2008.540 · Cited by 459 · cited 2×
“Significant deficits remain in the development of implementation strategies that increase the use of evidence-based practices (EBPs) in routine mental health care (Collins et al. 2011; Insel 2009; NIMH 2008).”
Glisson, C., Dukes, D., & Green, P. (2006) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2005.12.010 · Cited by 298
Self-citationAarons, G. A., Sommerfeld, D. H., & Walrath-Greene, C. M. (2009) · Implementation Science · doi:10.1186/1748-5908-4-83 · Cited by 276
Novins, D. K., Green, A. E., Legha, R. K., & Aarons, G. A. (2013) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2013.07.012 · Cited by 270
A Systematic Review of Strategies for Implementing Empirically Supported Mental Health Interventions
Powell, B. J., Proctor, E. K., & Glass, J. E. (2014) · Research on Social Work Practice · doi:10.1177/1049731513505778 · Cited by 268
Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy.
Glisson, C., Schoenwald, S. K., Hemmelgarn, A., Green, P., Dukes, D., Armstrong, K. S., & Chapman, J. E. (2010) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0019160 · Cited by 243
Self-citationWilliams, N. J. (2016) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-015-0693-2 · Cited by 164
Self-citationSchoenwald, S. K., Chapman, J. E., Kelleher, K., Hoagwood, K. E., Landsverk, J., Stevens, J., Glisson, C., Rolls-Reutz, J., & The Research Network on Youth Mental Health (2008) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-007-0147-6 · Cited by 149 · cited 4×
“Similar to national samples (Schoenwald et al. 2008), clinicians working in the agencies delivered a variety of mental health services to youth (e.”
InfluentialSelf-citationCounselor assessments of training and adoption barriers
Bartholomew, N. G., Joe, G. W., Rowan-Szal, G. A., & Simpson, D. D. (2007) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2007.01.005 · Cited by 86 · cited 3×
“provide clinical supervision that supports EBPs (Bartholomew et al. 2007; Raghavan et al. 2007).”
Reliability of Therapist Self-Report on Treatment Targets and Focus in Family-Based Intervention
Hogue, A., Dauber, S., Henderson, C. E., & Liddle, H. A. (2014) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-013-0520-6 · Cited by 52 · cited 2×
“A third caveat concerns the reliability of the self-report measures of EBP adoption and use given evidence that observational and self-report measures of clinicians’ EBP adoption do not always correspond (Hogue et al. 2014; Hurlburt et al. 2010).”
Cited by
Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda
Powell, B. J., Fernandez, M. E., Williams, N. J., Aarons, G. A., Beidas, R. S., Lewis, C. C., McHugh, S. M., & Weiner, B. J. (2019) · Frontiers in Public Health · doi:10.3389/fpubh.2019.00003 · Cited by 775
Self-citationLi, S.-A., Jeffs, L., Barwick, M., & Stevens, B. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0734-5 · Cited by 433
“[73] reported an increase in EBP uptake through improved organizational culture among mental health clinicians in 14 children’s mental health agencies.”
BackgroundA systematic review of empirical studies examining mechanisms of implementation in health
Lewis, C. C., Boyd, M. R., Walsh-Bailey, C., Lyon, A. R., Beidas, R., Mittman, B., Aarons, G. A., Weiner, B. J., & Chambers, D. A. (2020) · Implementation Science · doi:10.1186/s13012-020-00983-3 · Cited by 363
Williams, N. J., & Beidas, R. S. (2019) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12960 · Cited by 233
“To bring our suggestions to life, we share a recently published study with community clinicians serving youth in outpatient mental health clinics (Williams et al., 2017).”
BackgroundSelf-citationMeasuring implementation outcomes: An updated systematic review of measures’ psychometric properties
Mettert, K., Lewis, C., Dorsey, C., Halko, H., & Weiner, B. (2020) · Implementation Research and Practice · doi:10.1177/2633489520936644 · Cited by 162
“…possible score = 36), with ratings of “3—good” and 2—adequate” for internal consistency, “3—good” and “0—no evidence” for convergent validity, “0—no evidence” and “3—good” for predictive validity,, and “3—good” and “4—”excellent” for norms, respectively (Ruzek et al., 2016; Williams et al., 2017).”
Metadata
- Authors
- Williams, N. J., Glisson, C., Hemmelgarn, A., & Green, P.
- Venue
- Administration and Policy in Mental Health and Mental Health Services Research
- Published
- 2016-05-28
- Volume
- 44
- Issue
- 2
- Pages
- 269-283
- ISSN
- 0894-587X (ISSN-L)
- 1573-3289
- OpenAlex
- W2407484481
- Cited by
- 86
- References
- 26
- FWCI
- 13.37
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.