The DOI matches its Crossref record; 162 of the 171 references it cites were found in a registry or an index; 18 of the 120 sources its claims cite were checked for support.
Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors
Open accessAbstract
Implementation science is a quickly growing discipline. Lessons learned from business and medical settings are being applied but it is unclear how well they translate to settings with different historical origins and customs (e.g., public mental health, social service, alcohol/drug sectors). The purpose of this paper is to propose a multi-level, four phase model of the implementation process (i.e., Exploration, Adoption/Preparation, Implementation, Sustainment), derived from extant literature, and apply it to public sector services. We highlight features of the model likely to be particularly important in each phase, while considering the outer and inner contexts (i.e., levels) of public sector service systems.
Overview
CoChat Summary
TL;DR
This conceptual paper proposes a multi-level, four-phase model of evidence-based practice (EBP) implementation — Exploration, Adoption/Preparation, Implementation, and Sustainment — derived from extant literature and applied specifically to public sector services such as child welfare and specialty mental health. It argues that different outer-context (sociopolitical/funding, interorganizational networks, leadership) and inner-context (organizational characteristics, culture/climate, individual adopter characteristics) factors matter differentially across phases, and that sustainment can recursively inform future implementation efforts.
Key findings
- The model divides implementation into four phases (Exploration, Adoption/Preparation, Active Implementation, Sustainment) and holds that "different aspects of the outer and inner context may be more prominent or manifest differently during different phases" — a feature the authors say few prior models explicitly recognize.
- Exploration is shaped by outer-context forces (sociopolitical/funding, client advocacy, interorganizational networks) and inner-context factors including absorptive capacity, readiness for change, and receptive context; the authors note child welfare agencies often have "varied levels of education and considerable workloads" and few venues for knowledge sharing.
- Sustainment is described as under-studied: "there has been little empirical work examining factors that either facilitate or limit sustainability of EBPs in public sector social services," and the authors state "we currently lack comprehensive models of factors that support maintenance or sustainment."
- Sustainment factors discussed include ongoing fidelity monitoring and support, a critical mass of EBP use, staffing/selection practices, leadership and culture-embedding mechanisms, and funding; the paper cites turnover rates in public sector service organizations "range from 20% to 50% annually" as a sustainment challenge.
Methods
- Design: conceptual/theoretical paper advancing a framework, not an empirical study; the model is "derived from extant literature" and illustrated with examples primarily from child welfare and specialty mental health, with the claim that many factors are "common to public sector services in general."
- Data/sample: none — the paper synthesizes prior implementation models and literature (e.g., RE-AIM, ORC, ARC, QUERI, CDT, IHI collaborative) rather than collecting data.
- Analysis: narrative synthesis organized by implementation phase and by outer vs. inner context levels, presented in two conceptual figures (Fig. 1 global factors; Fig. 2 phases and factors).
Limitations
- Stated: the authors note "few, if any, good measures of the process or progression through the phases" exist, and that process-based measures would be needed to assess status, progression, milestones, and speed of phase traversal.
- Stated: the paper repeatedly acknowledges thin empirical support — "there are few data supporting the importance of the other drivers of EBP exploration suggested by the literature," and consensus on core implementation drivers exists but "the body of concrete research evidence for each core driver in many cases remains to be developed."
- Evident from the text: the model is explicitly a hypothesis-generating framework ("variables hypothesized to play important roles"), its emphasis is skewed toward child welfare with mental health examples, and it offers no empirical test of the proposed phase-specific relationships.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
16 Valid · 9 Not valid · 60 Not checked · 33 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Federal and state agencies may fund implementation demonstration projects and research studies to facilitate the implementation of EBPs, however, there has been little empirical work examining factors that either facilitate or limit sustainability of EBPs in public sector social services (Greenhalgh et al. 2004).”
§ Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors > Sustainment Phase
What the sources say
- Partly supportedcites Greenhalgh et al. 2004 sourceExperimental — accuracy not yet measured
The review says the literature on the sustainability of service innovations was very sparse (passage 1, page unavailable). It focused particularly on health care, however, and does not establish that empirical work on factors affecting sustainability of evidence-based practices was scarce specifically in public-sector social services. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The policies and funding needed for sustained use of practices typically follow from leaders who develop initiatives or set visions that can be maintained over time (Hoagwood et al. 2006).”
§ Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors > Sustainment Phase > Outer Context
What the sources say
- Partly supportedcites Hoagwood et al. 2006 sourceExperimental — accuracy not yet measured
The Discussion (page unavailable) says innovative leaders aided implementation and that all site Directors made plans for sustainability. The paper does not say that policies and funding typically follow from leaders’ initiatives or visions; it instead notes that continuation in Israel depended on securing funding. (LLM, conf=0.83)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Conversely, severe budget restrictions discourage the exploration of innovations since exploration demands staff time, which may be limited when work forces are cut due to funding restrictions”
§ Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors > Exploration Phase1 > The Outer Context
What the sources say
- Partly supportedcites Greenhalgh et al. 2004 sourceExperimental — accuracy not yet measured
The paper’s discussion of organizational innovativeness (section labeled Abstract; page unavailable) associates slack resources with innovation, and its ‘Dedicated Time and Resources’ passage says adequate, continuing resources make assimilation more likely. It does not specifically say that severe budget restrictions lead to workforce cuts that leave staff too little time to explore innovations. (LLM, conf=0.86)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Interorganizational forces, which focus on how individual organizations relate to, partner with, and compete with one another also play a potentially powerful role in adoption decisions.”
§ Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors > Adoption Decision/Preparation Phase > Outer Context
What the sources say
- Partly supportedcites Chamberlain et al. 2008 sourceExperimental — accuracy not yet measured
The conceptual-model passage (Abstract; page unavailable) identifies prior cross-agency collaboration and positive relationships between counties and the California Institute for Mental Health as factors expected to influence adoption. This supports the claim about relationships and partnerships, but our copy does not address competition between organizations as an adoption influence. (LLM, conf=0.84)
- Not checked — no full text of the sourcecites Glisson and Schoenwald 2005 source
- Not matchedcites Institute for Healthcare Improvement [IHI] 2003
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Organizations that start with good knowledge/skills, can incorporate new knowledge, are highly specialized, and have mechanisms in place to spread knowledge throughout the organization, are much more likely to explore EBPs and eventually initiate them”
§ Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors > Exploration Phase1 > The Inner Context
What the sources say
- Not checked — no full text of the sourcecites Damanpour 1991 source
- Not checked — no full text of the sourcecites Ferlie and Shortell 2001 source
- Partly supportedcites Greenhalgh et al. 2004 sourceExperimental — accuracy not yet measured
The passages on structural determinants and absorptive capacity (indexed as Abstract; page unavailable) link specialization, an existing knowledge base, and knowledge sharing to greater organizational innovativeness and assimilation. They do not establish that organizations with these features are *much* more likely to explore and then initiate evidence-based practices specifically; the paper also describes assimilation as a process that can move back and forth between stages. (LLM, conf=0.84)
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
57 more claims · 2 with their cited sources checked
Create a free account to see all 118 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 31
- It may state two separate claims as one. 35
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Specifically, unlike traditional feasibility and pilot studies that may include the preliminary testing of interventions on individual clinical or public health outcomes, implementation feasibility and pilot studies that explore strategies to improve intervention implementation often require assessing changes across multiple levels including individuals (e.g. service providers or clinicians) and”
From Guidance for conducting feasibility and pilot studies for implementation trials (2020) · cites this paper as [21]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“we propose that the concept of the individual might incorporate: capability, capacity, motivation (including recognising a need for change), resilience, acceptability, feelings, knowledge and beliefs (including self efficacy) about the intervention/evidence, position and fit within the organization/social system, and approach to decision-making (e.g., experimentation, use of information).”
From The role of evidence, context, and facilitation in an implementation trial: implications for the development of the PARIHS framework (2013) · cites this paper as [8]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Implementation strategies are too often designed in an unsystematic manner and fail to address key contextual determinants (13–16).”
From Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda (2019) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Even if strategies are meticulously described in a study protocol or trial registry, it is often unrealistic to expect that they will not need to be altered as determinants emerge across implementation phases (13, 93, 94).”
From Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda (2019) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Extent, quality, and timeliness of the completion of activities related to specific implementation process strategies”
From Beyond “implementation strategies”: classifying the full range of strategies used in implementation science and practice (2017) · cites this paper as [30]
Experimental — accuracy not yet measured
34 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors > References
“Data Path Design. (2010). ASWB projects.”
§ Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors > References
“Retrieved June 10, 2009”
Citations
Cites
Rogers, E. M., Singhal, A., & Quinlan, M. M. (2019) · An Integrated Approach to Communication Theory and Research · doi:10.4324/9780203710753-35 · Cited by 37,322
The fifth discipline, the art and practice of the learning organization
Senge, P. M. (1991) · Performance + Instruction · doi:10.1002/pfi.4170300510 · Cited by 20,060
Organizational culture and leadership
(1993) · Long Range Planning · doi:10.1016/0024-6301(93)90120-5 · Cited by 16,462
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496
Valente, T. W. (2003) · Genetics in Medicine · doi:10.1097/01.gim.0000061743.67794.c4 · Cited by 13,723
(1994) · Cited by 13,126
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0319-7 · Cited by 8,721
Self-citationDiffusion of Innovations in Service Organizations: Systematic Review and Recommendations
GREENHALGH, T., ROBERT, G., MACFARLANE, F., BATE, P., & KYRIAKIDOU, O. (2004) · The Milbank Quarterly · doi:10.1111/j.0887-378x.2004.00325.x · Cited by 7,448 · cited 11×
“Beginning at the broadest level, the state and federal sociopolitical and funding contexts influence the exploration of innovative interventions or practices (Davies and Nutley 2008; Ganju 2003; Hoagwood 2003).”
BackgroundInfluentialEvaluating the public health impact of health promotion interventions: the RE-AIM framework.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999) · American Journal of Public Health · doi:10.2105/ajph.89.9.1322 · Cited by 7,217 · cited 2×
“The RE-AIM model (Glasgow et al. 1999; Glasgow et al. 2006) explicitly identifies the ‘‘maintenance’’ phase of implementation as critical in sustaining effective health interventions.”
MethodOrganizational innovation: A meta-analysis of effects of determinants and moderators
Damanpour, F. (1991) · Academy of Management Journal · doi:10.2307/256406 · Cited by 6,542 · cited 9×
“Because the implementation literature has been developed in areas where the adoption of innovation is driven by profit or science, there is little attention to this first phase in the extant literature (Damanpour 1991).”
BackgroundMethodInfluential
Research works printed but not linked: 8
- Aos, S. (2006). Intensive family preservation programs: Program fidelity influences effectiveness revised (No. 06-02-3901). Olympia: Washington State Institute for Public Policy.
- California Institute for Mental Health [CIMH]. (2006). Community Development Team model: Supporting the model adherent implementation of programs and practices. Sacramento: California Institute for Mental Health.
- Chadwick Center. (2004). Closing the quality chasm in child abuse treatment: Identifying and disseminating best practices. San Diego, CA: Children’s Hospital-San Diego, Chadwich Center for Children and Families.
- Davies, H. T. O., & Nutley, S. M. (2008). Learning more about how research-based knowledge gets used: Guidance in the development of new empirical research. New York, NY: William T. Grant Foundation.
- Institute of Medicine [IOM]. (2007). The state of quality improvement and implementation research: Expert views. Workshop summary. Washington, DC: The National Academies Press.
- National Association of Public Child Welfare Administrators [NAPCWA]. (2005). Guide for child welfare administrators on evidence based practice. Washington DC: APHSA.
- National Institute of Mental Health [NIMH]. (2001). Blueprint for change: Research on child and adolescent mental health. Bethesda, MD: National Institutes of Health.
- Srivastava, S. K., & Bathla, A. (1996). Correlational study of organizational climate and work adjustment among industrial workers. Social Science International, 12(1), 52–60.
Cited by
Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-013-0528-y · Cited by 12,038
Self-citationThe updated consolidated framework for implementation research based on user feedback
Damschroder, L. J., Reardon, C. M., Widerquist, M. A. O., & Lowery, J. (2022) · Implementation Science · doi:10.1186/s13012-022-01245-0 · Cited by 4,452
Implementation strategies: recommendations for specifying and reporting
Proctor, E. K., Powell, B. J., & McMillen, J. C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-139 · Cited by 2,585
“[78] also note four”
The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change
Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-117 · Cited by 1,724
“Sustainment The continued use of an intervention within practice [10].”
BackgroundWiltsey Stirman, S., Baumann, A. A., & Miller, C. J. (2019) · Implementation Science · doi:10.1186/s13012-019-0898-y · Cited by 1,329
Metadata
- Authors
- Aarons, G. A., Hurlburt, M., & Horwitz, S. M.
- Affiliations
- University of California San Diego (US)
- University of Southern California (US)
- Rady Children's Hospital-San Diego (US)
- Corresponding author
- Aarons, Gregory A.
- Venue
- Administration and Policy in Mental Health and Mental Health Services Research
- Published
- 2010-12-14
- Volume
- 38
- Issue
- 1
- Pages
- 4-23
- ISSN
- 0894-587X (ISSN-L)
- 1573-3289
- PMC
- PMC3025110
- OpenAlex
- W2044858486
- Cited by
- 3,240
- References
- 171
- Citations by year
- 2026: 306 · 2025: 331 · 2024: 333 · 2023: 311 · 2022: 307 · 2021: 274 · 2020: 251 · 2019: 257 · 2018: 170 · 2017: 168 · 2016: 111 · 2015: 143 · 2014: 107 · 2013: 96 · 2012: 48
- FWCI
- 93.26
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc
- Locations
- Administration and Policy in Mental Health and Mental Health Services Research (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- National Institute of Mental Health (P30MH074678, R01MH072961)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Keywords
- evidence-based practice implementation, implementation science, public service sector, conceptual model, multi-level model, implementation stages, exploration, adoption, implementation, program sustainability, public mental health services, social services, drug and alcohol services, organizational context
- MeSH terms
- Child, Child Welfare, Diffusion of Innovation, Health Plan Implementation, Humans, Mental Health Services, Social Work, United States, Models, Organizational, Evidence-Based Practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors. https://app.cochat.ai/papers/10.1007/s10488-010-0327-7
- The paper
- Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors. Administration and Policy in Mental Health and Mental Health Services Research. https://doi.org/10.1007/s10488-010-0327-7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

