Cross-Validation of the Implementation Leadership Scale (ILS) in Child Welfare Service Organizations
Abstract
The Implementation Leadership Scale (ILS) is a brief, pragmatic, and efficient measure that can be used for research or organizational development to assess leader behaviors and actions that actively support effective implementation of evidence-based practices (EBPs). The ILS was originally validated with mental health clinicians. This study validates the ILS factor structure with providers in community-based organizations (CBOs) providing child welfare services. Participants were 214 service providers working in 12 CBOs that provide child welfare services. All participants completed the ILS, reporting on their immediate supervisor. Confirmatory factor analyses were conducted to examine the factor structure of the ILS. Internal consistency reliability and measurement invariance were also examined. Confirmatory factor analyses showed acceptable fit to the hypothesized first- and second-order factor structure. Internal consistency reliability was strong and there was partial measurement invariance for the first-order factor structure when comparing child welfare and mental health samples. The results support the use of the ILS to assess leadership for implementation of EBPs in child welfare organizations.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The ILS has been successfully validated in child welfare, substance use treatment, and acute care.
“Subsequent studies have begun this work, including successful validation of the instrument in child welfare [30], substance use treatment [31], and acute care [32];”
From Leading the charge in the education sector: development and validation of the School Implementation Leadership Scale (SILS) (2022) · cites this paper as [30]
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788 · see citing works
Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance
Cheung, G. W., & Rensvold, R. B. (2002) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem0902_5 · Cited by 16,280 · see citing works
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496 · see citing works
Longshore, J. M., & Bass, B. M. (1987) · Academy of Management Review · doi:10.2307/258081 · Cited by 12,424 · see citing works
Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance
Chen, F. F. (2007) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705510701301834 · Cited by 12,217 · see citing works
Measurement Invariance, Factor Analysis and Factorial Invariance
Meredith, W. (1993) · Psychometrika · doi:10.1007/bf02294825 · Cited by 4,536 · see citing works
Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0327-7 · Cited by 3,240 · cited 3× · see citing works
“, system) context of service delivery (Aarons et al., 2011).”
BackgroundInfluentialSelf-citationProctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-008-0197-4 · Cited by 1,894 · see citing works
Self-citationAarons, G. A. (2004) · Mental Health Services Research · doi:10.1023/b:mhsr.0000024351.12294.65 · Cited by 1,272 · see citing works
Self-citationKnowledge Translation in Health Care: Moving from Evidence to Practice
(2009) · International Journal of Health Care Quality Assurance · doi:10.1108/ijhcqa.2009.06222fae.002 · Cited by 689 · see citing works
Aarons, G. A., Ehrhart, M. G., & Farahnak, L. R. (2014) · Implementation Science · doi:10.1186/1748-5908-9-45 · Cited by 559 · see citing works
Self-citationEhrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2014) · Implementation Science · doi:10.1186/s13012-014-0157-1 · Cited by 502 · see citing works
Self-citationAarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Hurlburt, M. S. (2015) · Implementation Science · doi:10.1186/s13012-014-0192-y · Cited by 435 · see citing works
Self-citationTesting differences between nested covariance structure models: Power analysis and null hypotheses.
Maccallum, R., Browne, M., & Cai, L. (2006) · Psychological methods · doi:10.1037/1082-989x.11.1.19 · Cited by 374 · see citing works
Aarons, G. A., & Sommerfeld, D. H. (2012) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2012.01.018 · Cited by 325 · see citing works
Self-citationGlasgow, R. E., & Riley, W. T. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.03.010 · Cited by 318 · see citing works
Instrumentation issues in implementation science
Martinez, R. G., Lewis, C. C., & Weiner, B. J. (2014) · Implementation Science · doi:10.1186/s13012-014-0118-8 · Cited by 243 · see citing works
Michaelis, B., Stegmaier, R., & Sonntag, K. (2010) · Journal of Managerial Psychology · doi:10.1108/02683941011035304 · Cited by 219 · see citing works
Nelson, T. D., & Steele, R. G. (2007) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-006-0111-x · Cited by 216 · see citing works
Michaelis, B., Stegmaier, R., & Sonntag, K. (2009) · Journal of Change Management · doi:10.1080/14697010903360608 · Cited by 216 · see citing works
Context matters: measuring implementation climate among individuals and groups
Jacobs, S., Weiner, B. J., & Bunger, A. C. (2014) · Implementation Science · doi:10.1186/1748-5908-9-46 · Cited by 189 · see citing works
Ployhart, R. E., & Oswald, F. L. (2004) · Organizational Research Methods · doi:10.1177/1094428103259554 · Cited by 183 · see citing works
The Role of Organizational Processes in Dissemination and Implementation Research
Aarons, G. A., Moullin, J. C., & Ehrhart, M. G. (2017) · Oxford University Press eBooks · doi:10.1093/oso/9780190683214.003.0008 · Cited by 84 · see citing works
Self-citationRaghavan, R., Inoue, M., Ettner, S. L., Hamilton, B. H., & Landsverk, J. (2010) · American Journal of Public Health · doi:10.2105/ajph.2008.151472 · Cited by 83 · see citing works
Organizational Measurement and the Implementation of Innovations in Mental Health Services
Kimberly, J. R., & Cook, J. M. (2007) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-007-0143-x · Cited by 72 · see citing works
Ehrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2015) · Implementation Science · doi:10.1186/s13012-015-0255-8 · Cited by 71 · see citing works
Self-citationThe Role of Organizational Processes in Dissemination and Implementation Research
Aarons, G. A., Horowitz, J. D., Dlugosz, L. R., & Ehrhart, M. G. (2012) · Dissemination and Implementation Research in HealthTranslating Science to Practice · doi:10.1093/acprof:oso/9780199751877.003.0007 · Cited by 52 · see citing works
Self-citationResearch in Organizational Change and Development
Noumair, D. A., & Shani, A. B. R. (2014) · Emerald Group Publishing Limited eBooks · doi:10.1108/s0897-3016201422 · Cited by 30 · see citing works
The State of Parent Training: Program Offerings and Empirical Support
Kohl, P. L., Schurer, J., & Bellamy, J. L. (2009) · Families in Society The Journal of Contemporary Social Services · doi:10.1606/1044-3894.3894 · Cited by 24 · see citing works
Roesch, S. C., Norman, G. J., Merz, E. L., Sallis, J. F., & Patrick, K. (2013) · Journal of Applied Social Psychology · doi:10.1111/jasp.12000 · Cited by 17 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Lyon, A. R., Cook, C. R., Brown, E. C., Locke, J., Davis, C., Ehrhart, M., & Aarons, G. A. (2018) · Implementation Science · doi:10.1186/s13012-017-0705-6 · Cited by 199 · see citing works
Self-citationHagermoser Sanetti, L. M., & Collier-Meek, M. A. (2019) · Journal of School Psychology · doi:10.1016/j.jsp.2019.07.008 · Cited by 129 · see citing works
Gifford, W., Graham, I. D., Ehrhart, M. G., Davies, B., & Aarons, G. A. (2017) · Journal of Healthcare Leadership · doi:10.2147/jhl.s125558 · Cited by 84 · see citing works
Self-citationAarons, G. A., Ehrhart, M. G., Torres, E. M., Finn, N. K., & Beidas, R. S. (2016) · Psychiatric Services · doi:10.1176/appi.ps.201600062 · Cited by 68 · see citing works
Self-citationLyon, A. R., Whitaker, K. M., Locke, J. J., Cook, C. R., King, K. M., Duong, M. T., Davis, C. J., Weist, M. D., Ehrhart, M. G., & Aarons, G. A. (2018) · Implementation Science · doi:10.1186/s13012-018-0721-1 · Cited by 64 · see citing works
Self-citationGuerrero, E. G., Padwa, H., Fenwick, K. M., Harris, L. M., & Aarons, G. A. (2015) · Implementation Science · doi:10.1186/s13012-016-0438-y · Cited by 60 · see citing works
Self-citationAarons, G. A., Ehrhart, M. G., Torres, E. M., Finn, N. K., & Roesch, S. C. (2016) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2016.05.004 · Cited by 57 · see citing works
Self-citationWilliams, N. J., Hugh, M. L., Cooney, D., Worley, J. A., & Locke, J. J. (2022) · Behavior Therapy · doi:10.1016/j.beth.2022.03.001 · Cited by 49 · see citing works
Shuman, C. J., Ehrhart, M. G., Torres, E. M., Veliz, P., Kath, L. M., VanAntwerp, K., Banaszak‐Holl, J., Titler, M. G., & Aarons, G. A. (2020) · Worldviews on Evidence-Based Nursing · doi:10.1111/wvn.12402 · Cited by 46 · see citing works
Self-citationImplementation leadership: A concept analysis
Castiglione, S. A. (2019) · Journal of Nursing Management · doi:10.1111/jonm.12899 · Cited by 43 · see citing works
Lewis, C. C., Darnell, D. A., Kerns, S. E. U., Monroe‐DeVita, M., Landes, S. J., Lyon, A. R., Stanick, C., Dorsey, S., Locke, J. J., Marriott, B. R., et al. (2016) · Implementation Science · doi:10.1186/s13012-016-0428-0 · Cited by 43 · see citing works
Self-citationPsychometric properties of leadership scales for health professionals: a systematic review
Carlson, M. A., Morris, S., Day, F. L., Dadich, A., Ryan, A., Fradgley, E. A., & Paul, C. L. (2021) · Implementation Science · doi:10.1186/s13012-021-01141-z · Cited by 37 · see citing works
Lyon, A. R., Corbin, C. M., Brown, E. C., Ehrhart, M. G., Locke, J., Davis, C., Picozzi, E., Aarons, G. A., & Cook, C. R. (2022) · Implementation Science · doi:10.1186/s13012-022-01222-7 · Cited by 34 · see citing works
Self-citationBunger, A. C., Chuang, E., Girth, A. M., Lancaster, K. E., Gadel, F., Himmeger, M., Saldana, L., Powell, B. J., & Aarons, G. A. (2020) · Implementation Science · doi:10.1186/s13012-020-01016-9 · Cited by 29 · see citing works
Self-citationBraathu, N., Laukvik, E. H., Egeland, K. M., & Skar, A. S. (2022) · BMC Psychology · doi:10.1186/s40359-022-00725-8 · Cited by 31 · see citing works
Hu, J., Gifford, W., Ruan, H., Harrison, D. M., Li, Q., Ehrhart, M. G., & Aarons, G. A. (2019) · Journal of Nursing Management · doi:10.1111/jonm.12768 · Cited by 24 · see citing works
Self-citationValidating the Implementation Leadership Scale in Chinese nursing context: A cross‐sectional study
Hu, J., Gifford, W., Ruan, H., Harrison, D. M., Li, Q., Ehrhart, M. G., Harrison, M., Barrowman, N., & Aarons, G. A. (2021) · Nursing Open · doi:10.1002/nop2.888 · Cited by 16 · see citing works
Self-citationKolko, D. J., McGuier, E. A., Turchi, R. M., Thompson, E., Iyengar, S., Smith, S. N., Hoagwood, K., Liebrecht, C., Bennett, I. M., Powell, B. J., et al. (2022) · Implementation Science · doi:10.1186/s13012-022-01195-7 · Cited by 16 · see citing works
En Route to Implementation Science 3.0
Albers, B., Shlonsky, A., & Mildon, R. L. (2020) · Springer eBooks · doi:10.1007/978-3-030-03874-8_1 · Cited by 15 · see citing works
Validity and reliability of the Greek version of Implementation Leadership Scale (ILS)
Mandrou, E., Tsounis, A., & Sarafis, P. (2020) · BMC Psychology · doi:10.1186/s40359-020-00413-5 · Cited by 16 · see citing works
Shapiro, V. B., Ziemer, K. L., Accomazzo, S., & Kim, B. E. (2019) · Contemporary School Psychology · doi:10.1007/s40688-019-00230-7 · Cited by 13 · see citing works
Størksen, H. T., Haga, S., Slinning, K., & Drozd, F. (2019) · JMIR Mental Health · doi:10.2196/15149 · Cited by 8 · see citing works
McCarthy, S., & Griffiths, L. J. (2021) · Human Services Organizations Management Leadership & Governance · doi:10.1080/23303131.2021.1929626 · Cited by 9 · see citing works
Zhang, Y., Liu, W., Yan, Y.-T., Porr, C., Zhang, Y., & Wei, H. (2018) · Evaluation & the Health Professions · doi:10.1177/0163278718801439 · Cited by 9 · see citing works
Factors that predict child welfare caseworker referrals to an evidence-based parenting program
Myers, C., Garcia, A. R., Beidas, R. S., & Yang, Z. (2020) · Children and Youth Services Review · doi:10.1016/j.childyouth.2020.104750 · Cited by 8 · see citing works
The Role of Organizational Processes in Dissemination and Implementation Research
Aarons, G. A., Ehrhart, M. G., Lengnick‐Hall, R., & Moullin, J. C. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780197660690.003.0009 · Cited by 7 · see citing works
Self-citationCastiglione, S. A., Fréchette, J., & Agnihotram, V. R. (2023) · SAGE Open Nursing · doi:10.1177/23779608231216161 · Cited by 5 · see citing works
Sklar, M., Ehrhart, M. G., Ramirez, N., Carandang, K. M., Kuhn, N., Day, A., Aarons, G. A., & Williams, N. J. (2024) · Implementation Research and Practice · doi:10.1177/26334895241236680 · Cited by 6 · see citing works
Self-citationLlarena, M., Rogers, H. L., Macía, P., Pablo, S., González, M., Montejo, M., Paniagua, N., Benito, J., Rueda-Etxebarria, M., Santos, B., et al. (2023) · Implementation Science Communications · doi:10.1186/s43058-023-00495-3 · Cited by 5 · see citing works
Clark, S. L., Akin, B. A., Vanchy, P., Byers, K. D., & Diaz, A. L. (2024) · Human Services Organizations Management Leadership & Governance · doi:10.1080/23303131.2024.2392821 · Cited by 2 · see citing works
Validation of the extended version of the Implementation Leadership Scale (ILS-X)
Ehrhart, M. G., Williams, N. J., Sklar, M., Vega, N. R., Kandah, A., & Aarons, G. A. (2025) · Implementation Science Communications · doi:10.1186/s43058-025-00786-x · Cited by 1 · see citing works
Self-citationYıldız, A. (2024) · BMC Health Services Research · doi:10.1186/s12913-024-11721-6 · Cited by 3 · see citing works
Santos, W. J., Hutchinson, A. M., Rader, T., Graham, I. D., Watkins, V. J., de Cândido, L. K., Greenough, M. J., & Squires, J. E. (2023) · BMC Research Notes · doi:10.1186/s13104-023-06294-2
Santos, W. J., Hutchinson, A. M., Rader, T., Graham, I. D., Watkins, V. J., de Cândido, L. K., Greenough, M. J., & Squires, J. E. (2024) · Figshare · doi:10.6084/m9.figshare.26574445
Santos, W. J., Hutchinson, A. M., Rader, T., Graham, I. D., Watkins, V. J., de Cândido, L. K., Greenough, M. J., & Squires, J. E. (2024) · Figshare · doi:10.6084/m9.figshare.26574454
Santos, W. J., Hutchinson, A. M., Rader, T., Graham, I. D., Watkins, V. J., de Cândido, L. K., Greenough, M. J., & Squires, J. E. (2024) · Figshare · doi:10.6084/m9.figshare.26574460
Santos, W. J., Hutchinson, A. M., Rader, T., Graham, I. D., Watkins, V., de Cândido, L. K., Greenough, M. J., & Squires, J. E. (2024) · Figshare · doi:10.6084/m9.figshare.26574448
Santos, W. J., Hutchinson, A. M., Rader, T., Graham, I. D., Watkins, V. J., de Cândido, L. K., Greenough, M. J., & Squires, J. E. (2024) · Figshare · doi:10.6084/m9.figshare.26574457
Lyon, A. R., Corbin, C. M., Brown, E. C., Ehrhart, M. G., Locke, J. J., Davis, C. J., Picozzi, E., Aarons, G. A., & Cook, C. R. (2022) · Figshare · doi:10.6084/m9.figshare.20342346
Self-citationCarlson, M. A., Morris, S., Day, F., Dadich, A., Ryan, A., Fradgley, E. A., & Paul, C. (2021) · Figshare · doi:10.6084/m9.figshare.16530009
Llarena, M., Rogers, H. L., Macia, P., Pablo, S., González, M., Montejo, M., Paniagua, N., Benito, J., Rueda, M., Santos, B., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26616964
Santos, W. J., Hutchinson, A. M., Rader, T., Graham, I. D., Watkins, V. J., de Cândido, L. K., Greenough, M. J., & Squires, J. E. (2024) · Figshare · doi:10.6084/m9.figshare.26574451
Marriott, J. P., Shanks, C. B., Calloway, E. E., Yaroch, A. L., Prickitt, J., Meléndrez, B., Knauer, N., & Houghtaling, B. (2026) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-026-01928-7
McCormick, K. A., Stanton, M. C., Strohl, K., & Ali, S. (2026) · Journal of the International Association of Providers of AIDS Care (JIAPAC) · doi:10.1177/23259582261470700
Kuha, S., Miettunen, J., & Kanste, O. I. (2026) · Worldviews on Evidence-Based Nursing · doi:10.1111/wvn.70163
- Authors
- Finn, N. K., Torres, E. M., Ehrhart, M. G., Roesch, S. C., & Aarons, G. A.
- Affiliations
- University of California San Diego (US)
- San Diego State University (US)
- Corresponding author
- Aarons, Gregory A.
- Venue
- Child Maltreatment
- Publisher
- SAGE Publications
- Published
- 2016-07-24
- Volume
- 21
- Issue
- 3
- Pages
- 250-255
- ISSN
- 1077-5595 (ISSN-L)
- 1552-6119
- OpenAlex
- W2308360502
- Cited by
- 45
- References
- 30
- Citations by year
- 2026: 3 · 2025: 1 · 2024: 10 · 2023: 4 · 2022: 5 · 2021: 4 · 2020: 5 · 2019: 5 · 2018: 3 · 2017: 1 · 2016: 3 · 2015: 1
- FWCI
- 7.45
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Child Maltreatment (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of Mental Health (R21MH098124, P30MH074678, R21MH082731, R01MH072961, R25MH080916)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Public Administration › Social Work Education and Practice
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- implementation leadership, evidence-based practice implementation, child welfare services, community organizations, leadership behavior, factor structure, measurement invariance, cross-validation
- MeSH terms
- Child Protective Services, Adult, Child, Child Welfare, Factor Analysis, Statistical, Female, Humans, Leadership, Male, Middle Aged, United States, Reproducibility of Results, Evidence-Based Practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review