The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership
Open accessAbstract
BACKGROUND: In healthcare and allied healthcare settings, leadership that supports effective implementation of evidenced-based practices (EBPs) is a critical concern. However, there are no empirically validated measures to assess implementation leadership. This paper describes the development, factor structure, and initial reliability and convergent and discriminant validity of a very brief measure of implementation leadership: the Implementation Leadership Scale (ILS). METHODS: Participants were 459 mental health clinicians working in 93 different outpatient mental health programs in Southern California, USA. Initial item development was supported as part of a two United States National Institutes of Health (NIH) studies focused on developing implementation leadership training and implementation measure development. Clinician work group/team-level data were randomly assigned to be utilized for an exploratory factor analysis (n = 229; k = 46 teams) or for a confirmatory factor analysis (n = 230; k = 47 teams). The confirmatory factor analysis controlled for the multilevel, nested data structure. Reliability and validity analyses were then conducted with the full sample. RESULTS: The exploratory factor analysis resulted in a 12-item scale with four subscales representing proactive leadership, knowledgeable leadership, supportive leadership, and perseverant leadership. Confirmatory factor analysis supported an a priori higher order factor structure with subscales contributing to a single higher order implementation leadership factor. The scale demonstrated excellent internal consistency reliability as well as convergent and discriminant validity. CONCLUSIONS: The ILS is a brief and efficient measure of unit level leadership for EBP implementation. The availability of the ILS will allow researchers to assess strategic leadership for implementation in order to advance understanding of leadership as a predictor of organizational context for implementation. The ILS also holds promise as a tool for leader and organizational development to improve EBP implementation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Subscale internal consistencies ranged from 0.95 to 0.98.”
From Leading the charge in the education sector: development and validation of the School Implementation Leadership Scale (SILS) (2022) · cites this paper as [9]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Our preliminary results also signal a need for new instrumentation targeting non-provider stakeholders such as leaders and external change agents (e.g., implementation practitioners or intermediaries), particularly in light of research identifying the role they play in implementation success (e.g., [25]).”
From The Society for Implementation Research Collaboration Instrument Review Project: A methodology to promote rigorous evaluation (2016) · cites this paper as [25]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Designed to be brief and pragmatic [29], the original ILS contained 12 items loading onto 4 subscales: Proactive Leadership (anticipating and addressing implementation challenges), Knowledgeable Leadership (deep understanding of EBP and implementation issues), Supportive Leadership (support for EBP adoption/use), and Perseverant Leadership (consistent and responsive to challenges).”
From Leading the charge in the education sector: development and validation of the School Implementation Leadership Scale (SILS) (2022) · cites this paper as [9]
The cited source says what the claim attributes to it.
In the citing paper
“The original ILS [9] and original SILS adaptation [66] are 12-item instruments developed to assess strategic leadership for EBP implementation.”
From Leading the charge in the education sector: development and validation of the School Implementation Leadership Scale (SILS) (2022) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788 · cited 1×
“08 indicate acceptable model fit [41-44].”
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532 · cited 1×
“We first examined the amount of dependency among observations within groups using intraclass correlations (ICC, type 1) [46].”
MethodHu, L.-T., & Bentler, P. M. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.424 · Cited by 12,298 · cited 1×
“08 indicate acceptable model fit [41-44].”
Organizational Culture and Leadership
Tierney, W. G., & Schein, E. H. (1986) · The Academy of Management Review · doi:10.2307/258322 · Cited by 9,601
Evaluating the use of exploratory factor analysis in psychological research.
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.3.272 · Cited by 9,587 · cited 1×
“Principal axis factoring was selected for factor extraction because it allows for consideration of both systematic and random error [37] and Promax oblique rotation was utilized for factor rotation as we assumed that derived factors would be correlated.”
MethodProctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0319-7 · Cited by 8,721
Self-citationA Rationale and Test for the Number of Factors in Factor Analysis
Horn, J. L. (1965) · Psychometrika · doi:10.1007/bf02289447 · Cited by 8,714 · cited 1×
“Parallel analysis was based on estimation of 1,000 random data matrices with values that correspond to the 95 percentile of the distribution of random data eigenvalues [39,40].”
MethodDETERMINANTS OF INNOVATIVE BEHAVIOR: A PATH MODEL OF INDIVIDUAL INNOVATION IN THE WORKPLACE.
Scott, S. G., & Bruce, R. A. (1994) · Academy of Management Journal · doi:10.2307/256701 · Cited by 6,068 · cited 1×
“Leaders can positively or negatively impact the capacity to foster change and innovation [7-10] and therefore are instrumental in facilitating a positive climate for innovation and positive attitudes toward EBP during implementation”
BackgroundEstimating within-group interrater reliability with and without response bias
James, L. R., Demaree, R. G., & Wolf, G. (1984) · Journal of Applied Psychology · doi:10.1037/0021-9010.69.1.85 · Cited by 4,774 · cited 3×
“We next examined the average agreement within clinical work group for individual items and scales using awg(1) and awg(J), respectively [48-50].”
BackgroundMethodOrganizational culture and leadership
Tohidi, H., & Jabbari, M. M. (2012) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2011.12.156 · Cited by 4,303 · cited 2×
“First, as part of a study focused on developing an intervention to improve leadership for evidencebased practice implementation [18], the investigative team developed items based on review of literature relating leader behaviors to implementation and organizational climate and culture change [32,33].”
Background
Research works printed but not linked: 4
- Massoud M, Nielsen G, Nolan K, Nolan TW, Schall MW, Sevin C: A Framework for Spread: From Local Improvements to System-wide Change. 2006, Cambridge, MA: Institute for Healthcare Improvement
- Dunn G, Everitt B, Pickles A: Modeling Covariances and Latent Variables Using EQS. 1993, London: Chapman & Hall/CRC Press
- Cochran WG: Sampling Techniques. 1977, New York: Wiley and Sons, 3
- Kouzes JM, Posner BZ: The Leadership Challenge. 2007, San Francisco, CA: Jossey-Bass
Cited by
Psychometric assessment of three newly developed implementation outcome measures
Weiner, B. J., Lewis, C. C., Stanick, C., Powell, B. J., Dorsey, C. N., Clary, A. S., Boynton, M. H., & Halko, H. (2017) · Implementation Science · doi:10.1186/s13012-017-0635-3 · Cited by 2,819
“There are several examples of recent efforts to develop pragmatic measures for implementation constructs such as organizational readiness for change [36], implementation leadership [37], and implementation climate [38, 39].”
BackgroundAn introduction to implementation science for the non-specialist
Bauer, M. S., Damschroder, L., Hagedorn, H., Smith, J., & Kilbourne, A. M. (2015) · BMC Psychology · doi:10.1186/s40359-015-0089-9 · Cited by 1,611
“Explicit operationalization of theoretical constructs can also guide development of robust quantitative measures [50, 51].”
BackgroundSystematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework
Moullin, J. C., Dickson, K. S., Stadnick, N. A., Rabin, B., & Aarons, G. A. (2019) · Implementation Science · doi:10.1186/s13012-018-0842-6 · Cited by 1,223
“The Implementation Leadership Scale (ILS): development of a brief measure of unit level implementation leadership.”
Self-citationEhrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2014) · Implementation Science · doi:10.1186/s13012-014-0157-1 · Cited by 502
“We also focused the items in this measure on the work group level, as we have argued in our other research [43-45] for the critical role of the first-level supervisor for implementation success.”
ResultSelf-citationTen recommendations for using implementation frameworks in research and practice
Moullin, J. C., Dickson, K. S., Stadnick, N. A., Albers, B., Nilsen, P., Broder-Fingert, S., Mukasa, B., & Aarons, G. A. (2020) · Implementation Science Communications · doi:10.1186/s43058-020-00023-7 · Cited by 488
“, implementation leadership [76], implementation climate [77, 78], or implementation intentions [79]).”
BackgroundSelf-citation
Metadata
- Authors
- Aarons, G. A., Ehrhart, M. G., & Farahnak, L. R.
- Affiliations
- University of California San Diego (US)
- San Diego State University (US)
- Corresponding author
- Aarons, Gregory A
- Venue
- Implementation Science
- Published
- 2014-04-14
- Volume
- 9
- Issue
- 1
- Pages
- 45-45
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC4022333
- OpenAlex
- W2120503228
- Cited by
- 559
- References
- 56
- Citations by year
- 2026: 40 · 2025: 54 · 2024: 72 · 2023: 72 · 2022: 59 · 2021: 66 · 2020: 40 · 2019: 48 · 2018: 36 · 2017: 23 · 2016: 20 · 2015: 18 · 2014: 10
- FWCI
- 42.75
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Implementation Science (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Open Scholarship Institutional Repository (Washington University in St. Louis) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- Funders
- National Institute of Mental Health (R21MH098124, P30MH074678, R21MH082731, R01MH072961, R25MH080916)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Social Sciences › Education › Innovations in Medical Education
- Keywords
- implementation leadership, scale development, evidence-based practice implementation, mental health professionals, proactive leadership, supportive leadership, organizational context, leader development, organizational development
- MeSH terms
- Adult, Aged, Attitude of Health Personnel, California, Cooperative Behavior, Factor Analysis, Statistical, Female, Humans, Leadership, Male, Middle Aged, Organizational Innovation, Psychometrics, Reproducibility of Results, Young Adult, Translational Research, Biomedical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership. https://app.cochat.ai/papers/10.1186/1748-5908-9-45
- The paper
- Aarons, G. A., Ehrhart, M. G., & Farahnak, L. R. (2014). The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership. Implementation Science. https://doi.org/10.1186/1748-5908-9-45
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.