The 4KEEPS study: identifying predictors of sustainment of multiple practices fiscally mandated in children’s mental health services
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Research to date has largely focused on predictors of adoption and initial implementation of evidence-based practices (EBPs), yet sustained implementation is crucial to deliver a return on investments in dissemination. Furthermore, most studies focus on single EBPs, limiting opportunities to study the fit between practice characteristics EBPs and implementation contexts. METHODS/DESIGN: This observational study will characterize implementation sustainment and identify organizational and therapist characteristics that predict sustainment of multiple practices being implemented within a fiscal mandate in the largest public mental health system in the USA. Specific aims are to (1) characterize sustainment outcomes (volume/penetration, EBP concordant care); (2) use mixed methods to characterize inner context (agency- and therapist-level) factors and early implementation conditions; and (3) identify inner context factors and early implementation conditions that predict sustainment outcomes. This study will undertake original data collection and analysis of existing data sources to achieve its aims. Archived reports and documents will be used to characterize early implementation conditions in 102 agencies. Administrative claims data will be used to characterize volume and penetration outcomes over 8 years. Therapist and program manager surveys will be administered to characterize sustained EBP concordant care and inner context determinants of sustainment. An in-depth study in a subset of agencies will yield interview data and recordings of treatment sessions for validation of the EBP concordant care scale. DISCUSSION: This project will yield new understanding of whether and how multiple EBPs can be sustained in public mental health systems undergoing a policy-driven community implementation effort. We will produce generalizable models for characterizing sustainment, including feasible and flexible measurement of practice across multiple EBPs. The findings will inform the development of implementation interventions to promote sustained delivery of EBPs to maximize their public health impact.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0319-7 · Cited by 8,721 · cited 3×
“It can be measured as the number of eligible individuals who use/deliver a treatment, divided by the number of eligible individuals [17].”
BackgroundInfluentialAdvancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0327-7 · Cited by 3,240 · cited 2×
“’s [15] EPIS Framework to the LACDMH Prevention and Early Intervention (PEI) timeline Lau and Brookman-Frazee Implementation Science (2016) 11:31 Page 3 of 8”
MethodWiltsey Stirman, S., Kimberly, J., Cook, N., Calloway, A., Castro, F., & Charns, M. (2012) · Implementation Science · doi:10.1186/1748-5908-7-17 · Cited by 1,355 · cited 2×
“[18] reviewed 125 studies examining sustainment across service settings and interventions and highlighted a general lack of methodological rigor.”
BackgroundAarons, G. A. (2004) · Mental Health Services Research · doi:10.1023/b:mhsr.0000024351.12294.65 · Cited by 1,272
McHugh, R. K., & Barlow, D. H. (2010) · American Psychologist · doi:10.1037/a0018121 · Cited by 909
An Agenda for Research on the Sustainability of Public Health Programs
Scheirer, M. A., & Dearing, J. W. (2011) · American Journal of Public Health · doi:10.2105/ajph.2011.300193 · Cited by 775 · cited 1×
“These costly efforts provide natural laboratories to identify determinants of the sustainment of EBPs, an understudied topic in implementation science [10].”
BackgroundStirman, S. W., Miller, C. J., Toder, K., & Calloway, A. (2013) · Implementation Science · doi:10.1186/1748-5908-8-65 · Cited by 691 · cited 1×
“[52, 53] framework of adaptations and modifications of EBPs.”
BackgroundChorpita, B. F., & Daleiden, E. L. (2009) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0014565 · Cited by 687 · cited 1×
“intervention as patient needs vary [3, 11, 12].”
Lau, A. S. (2006) · Clinical Psychology: Science and Practice · doi:10.1111/j.1468-2850.2006.00042.x · Cited by 624 · cited 1×
“Therapists may adapt by reframing interventions, incorporating techniques or components to promote acceptance or address group-specific needs [40].”
BackgroundSelf-citationImplementing computerized technology: An organizational analysis.
Klein, K. J., Conn, A. B., & Sorra, J. S. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.5.811 · Cited by 615 · cited 1×
“Organizational support for practices predicts adoption and initial implementation [23, 25], and organizational climate is associated with provider willingness to deliver practices [24, 26].”
Background
Cited by
Quantitative measures of health policy implementation determinants and outcomes: a systematic review
Allen, P., Pilar, M., Walsh-Bailey, C., Hooley, C., Mazzucca, S., Lewis, C. C., Mettert, K. D., Dorsey, C. N., Purtle, J., Kepper, M. M., et al. (2020) · Implementation Science · doi:10.1186/s13012-020-01007-w · Cited by 164
Becan, J. E., Bartkowski, J. P., Knight, D. K., Wiley, T. R. A., DiClemente, R., Ducharme, L., Welsh, W. N., Bowser, D., McCollister, K., Hiller, M., et al. (2018) · Health & Justice · doi:10.1186/s40352-018-0068-3 · Cited by 119
“However, there is a precedent for studies examining implementation and sustainment of multiple EBPs as a function of large scale policy initiatives (Lau & Brookman-Frazee, 2016).”
BackgroundSklar, M., Ehrhart, M. G., & Aarons, G. A. (2021) · Psychiatric Rehabilitation Journal · doi:10.1037/prj0000480 · Cited by 85
Nevedal, A. L., Reardon, C. M., Jackson, G. L., Cutrona, S. L., White, B., Gifford, A. L., Orvek, E., DeLaughter, K., White, L., King, H. A., et al. (2020) · Implementation Science Communications · doi:10.1186/s43058-020-00053-1 · Cited by 44
Ramos, G., Brookman-Frazee, L., Kodish, T., Rodriguez, A., & Lau, A. S. (2021) · Cultural Diversity & Ethnic Minority Psychology · doi:10.1037/cdp0000357 · Cited by 30
“Recruitment and Procedures Data for the current study were drawn from the Knowledge Exchange on Evidence-Based Practice Sustainment (4KEEPS) project (Lau & Brookman-Frazee, 2016), which examined Cultur Divers Ethnic Minor Psychol.”
MethodSelf-citation
Metadata
- Authors
- Lau, A. S., & Brookman-Frazee, L.
- Venue
- Implementation Science
- Published
- 2015-12
- Volume
- 11
- Issue
- 1
- Pages
- 31-31
- ISSN
- 1748-5908 (ISSN-L)
- OpenAlex
- W2294413879
- Cited by
- 80
- References
- 21
- FWCI
- 11.69
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The 4KEEPS study: identifying predictors of sustainment of multiple practices fiscally mandated in children’s mental health services. https://app.cochat.ai/papers/10.1186/s13012-016-0388-4
- The paper
- Lau, A. S., & Brookman-Frazee, L. (2015). The 4KEEPS study: identifying predictors of sustainment of multiple practices fiscally mandated in children’s mental health services. Implementation Science. https://doi.org/10.1186/s13012-016-0388-4
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.