The DOI matches its Crossref record; 89 of the 91 references it cites were found in a registry or an index; 4 of the 62 sources its claims cite were checked for support.
“Scaling-out” evidence-based interventions to new populations or new health care delivery systems
Open accessAbstract
BACKGROUND: Implementing treatments and interventions with demonstrated effectiveness is critical for improving patient health outcomes at a reduced cost. When an evidence-based intervention (EBI) is implemented with fidelity in a setting that is very similar to the setting wherein it was previously found to be effective, it is reasonable to anticipate similar benefits of that EBI. However, one goal of implementation science is to expand the use of EBIs as broadly as is feasible and appropriate in order to foster the greatest public health impact. When implementing an EBI in a novel setting, or targeting novel populations, one must consider whether there is sufficient justification that the EBI would have similar benefits to those found in earlier trials. DISCUSSION: In this paper, we introduce a new concept for implementation called "scaling-out" when EBIs are adapted either to new populations or new delivery systems, or both. Using existing external validity theories and multilevel mediation modeling, we provide a logical framework for determining what new empirical evidence is required for an intervention to retain its evidence-based standard in this new context. The motivating questions are whether scale-out can reasonably be expected to produce population-level effectiveness as found in previous studies, and what additional empirical evaluations would be necessary to test for this short of an entirely new effectiveness trial. We present evaluation options for assessing whether scaling-out results in the ultimate health outcome of interest. CONCLUSION: In scaling to health or service delivery systems or population/community contexts that are different from the setting where the EBI was originally tested, there are situations where a shorter timeframe of translation is possible. We argue that implementation of an EBI in a moderately different setting or with a different population can sometimes "borrow strength" from evidence of impact in a prior effectiveness trial. The collection of additional empirical data is deemed necessary by the nature and degree of adaptations to the EBI and the context. Our argument in this paper is conceptual, and we propose formal empirical tests of mediational equivalence in a follow-up paper.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This conceptual paper introduces “scaling-out” as the deliberate implementation, testing, improvement, and sustainment of evidence-based interventions (EBIs) in new populations and/or new delivery systems. It argues that under some conditions, scale-out can “borrow strength” from prior effectiveness trials if core elements, mechanisms, and mediational pathways are retained or empirically tested.
Key findings
- The paper defines three scale-out types: Type I (population fixed, different delivery system), Type II (delivery system fixed, different population), and Type III (different population and delivery system); it focuses mainly on Types I and II.
- It proposes that EBIs in moderately different settings or populations can sometimes borrow strength from prior effectiveness evidence, with additional empirical data needed depending on the nature and degree of adaptations; this requires retaining core elements, preserving mechanisms of action, ensuring sufficient organizational/system support, and testing “mediational equivalence.”
- It outlines four levels of evidence for scale-out evaluations, from level 0 (minimal or no new empirical evidence) to level 3 (full randomized hybrid trial), mapped to RE-AIM domains; Table 3 gives typical evaluation levels by whether population and delivery system are same or different.
- Examples include PrEP, where adherence reduced HIV risk by close to 90% in trials, but CDC guidelines did not recommend PrEP for at-risk adolescents due to lack of efficacy/safety evidence; adolescents aged 13–19 accounted for 4% of new HIV diagnoses in 2015. SafeCare was scaled out in one US county with population 3.2 million using an interagency collaborative team.
Methods
- Conceptual/theoretical paper; no empirical sample, data collection, or primary analysis.
- Uses external validity theories (Cook’s proximal similarity and heterogeneous irrelevancies), multilevel mediation modeling, RE-AIM, and implementation frameworks including EPIS, the Dynamic Sustainability Framework, and CFIR.
- Develops a logical framework, tables of scale-out types, four evidence levels, and a sequential mediation schematic; formal statistical tests of mediational equivalence are deferred to a follow-up paper.
Limitations
- The paper states its argument is conceptual and that formal empirical tests of mediational equivalence are proposed in a follow-up paper.
- It states that details remain unanswered regarding how best to assess changes in complex implementation systems, and that rigorous non-inferiority statistical methods require additional development.
- Evident from the text: the proposed framework and evidence levels are not empirically validated here; Type III scale-out is only mentioned and not developed; examples are illustrative rather than evaluated.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
34 Not checked yet · 34 Own interpretation · 2 Partly supported · 2 No basis given · accuracy not yet measured
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“As one example, improvement in the parent-child relationship through parenting interventions has been shown to relate to lower drug abuse, HIV sex risk behavior, and internalizing behavioral symptoms for adolescents [24, 44–46].”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Assuming a conceptual theory of mediation holds for scaling-out
What the sources say
- Partly supportedcites [24] Perrino T, Pantin H, Prado G, Huang S, Brincks A, Howe G, Beardslee W, Sandler I, Brown CH. Preventing internalizing symptoms among Hispanic adolescents: a synthesis across Familias Unidas trials. Prev Sci. 2014;15:917–28. sourceExperimental — accuracy not yet measured
The Results (Moderated Mediation Analysis; page not provided) link improved parent–adolescent communication to reduced internalizing symptoms, particularly among Hispanic adolescents with poorer communication at baseline. The Familias Unidas Intervention section reports reductions in substance use and sexual risk behavior, but this paper does not establish that improvements in the parent–child relationship accounted for those two outcomes. (LLM, conf=0.86)
- Not checked yetcites [44] Prado G, Cordova D, Cano N, Arzon M, Pantin H, Brown CH. Drug abuse preventive interventions for Hispanic youth: State of the science and implications for future research. In: Drug Use Trajectories Among Minority Youth. Dordrecht: Springer; 2016. p. 347–65. source
- Partly supportedcites [45] Estrada Y, Rosen A, Huang S, Tapia M, Sutton M, Willis L, Quevedo A, Condo C, Vidot DC, Pantin H. Efficacy of a brief intervention to reduce substance use and human immunodeficiency virus infection risk among Latino youth. J Adolesc Health. 2015;57:651–7. sourceExperimental — accuracy not yet measured
The Results (Tests of intervention effects; Moderating effects, page unavailable) report improved positive parenting and reduced illicit drug use among adolescents with low-to-moderate parental involvement at baseline. The Discussion links improved family functioning with less drug use and unprotected sex. The paper does not report a reduction in internalizing symptoms, nor does it establish that the observed parenting improvement caused the reductions in risk behavior. (LLM, conf=0.86)
- Not checked yetcites [46] Brincks A, Perrino T, Howe G, Pantin H, Prado G, Huang S, Cruden G, Brown CH. Preventing youth internalizing symptoms through the Familias Unidas intervention: examining variation in response. Prev Sci. 2016. https://link.springer.com/article/10.1007%2Fs11121-016-0666-z .
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, PrEP delivery requires a higher level of engagement of healthcare providers than has typically been engaged for HIV primary prevention; in fact, it more closely mimics the types of care an individual living with HIV received than the less medicalized services more typically provided for HIV prevention.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Two types of scaling-out: population fixed, different system, and delivery system fixed, different population, scaling-out > Type I: population fixed, different delivery system scaling-out
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Indeed, if we can legitimately borrow strength from previous studies and a modest amount of empirical evidence, this could accelerate and expand benefit to populations that have experienced health disparities that might never be included in a rigorous randomized effectiveness trial [6].”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Defining a new concept for implementation—“scaling-out”
What the sources say
- Not checked yetcites [6] Brown CH, Beardslee W. Realizing population-level improvements for all children’s cognitive, affective, and behavioral health. Am J Prev Med. 2016;51:S101–5. source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Therefore, it would be reasonable to expect beneficial outcomes from this intervention when an intervention increases the mean on parent-child communication to the level found in previous studies.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Assuming a conceptual theory of mediation holds for scaling-out
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Logically, we have no justification to anticipate health impact in scale-out if we are unwilling to embrace these three fundamental premises either based on the strength of generalizability of existing evidence or new data.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Defining a new concept for implementation—“scaling-out” > Empirical evidence needed for scaling-out
plain- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Concern regarding the effectiveness of an intervention when adapted has been addressed for a number of specific EBIs, including child maltreatment interventions [50], substance abuse treatment [51], child anxiety interventions [52], HIV treatment [53], school-based social competence interventions [54], psychological treatments for a variety of disorders [55, 56], and health risk prevention program”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Considerations for scaling-out > Intervention adaptation
What the sources say
- Not checked yetcites [50] Aarons GA, Palinkas LA. Implementation of evidence-based practice in child welfare: service provider perspectives. Adm Policy Ment Health Ment Health Serv Res. 2007;34:411–9. source
- Not checked yetcites [51] Backer TE. The failure of success: challenges of disseminating effective substance abuse prevention programs. J Community Psychol. 2000;28:363–73. source
- Not checked yetcites [52] Kendall PC, Gosch E, Furr JM, Sood E. Flexibility within fidelity. J Am Acad Child Adolesc Psychiatry. 2008;47:987–93. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Because few effectiveness trials of these programs exist in primary care, the evidence for the effectiveness of such parenting programs while being delivered through primary care is relatively limited [29].”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Two types of scaling-out: population fixed, different system, and delivery system fixed, different population, scaling-out > Type I: population fixed, different delivery system scaling-out
What the sources say
- Not checked yetcites [29] Asarnow JR, Rozenman M, Wiblin J, Zeltzer L. Integrated medical-behavioral care compared with usual primary care for child and adolescent behavioral health: a meta-analysis. JAMA Pediatr. 2015;169:929–37. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“However, this perspective is not universally shared [8], and there are examples of EBI scale-ups where health outcomes did not improve as intended [9].”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Defining a new concept for implementation—“scaling-out” > Scaling-out vs scaling-up
What the sources say
- Not checked yetcites [9] Klein RA, Ratliff KA, Vianello M, Adams RB, Bahnik S, Bernstein MJ, Bocian K, Brandt MJ, Brooks B, Brumbaugh CC, et al. Investigating variation in replicability: a "many labs" replication project. Soc Psychol. 2014;45:142–52. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It would therefore take far less time and expense to conduct such an evaluation compared to the original effectiveness trial.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Evaluation options for scaling-out > EBI replicability or effectiveness
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Second, the underlying mechanism of action regarding how core elements affect health outcomes remains the same, which relies on analyses of such mechanisms [13].”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Defining a new concept for implementation—“scaling-out” > Empirical evidence needed for scaling-out
What the sources say
- Not checked yetcites [13] MacKinnon DP. Introduction to statistical mediation analysis. New York: Routledge; 2008.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 33
- It may state two separate claims as one. 15
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“To close this discussion of a framework for asserting that a scaling-out is expected to share the original version’s impact on health outcomes by relying on previous studies and new empirical data, we note that more rapid implementation is especially important to the delivery of effective interventions to minorities and other populations experiencing health or health service disparities that would”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Conclusion
What the sources say
- Partly supportedcites [90] Brown CH, Mohr DC, Gallo CG, Mader C, Palinkas L, Wingood G, Prado G, Kellam SG, Pantin H, Poduska J, et al. A computational future for preventing HIV in minority communities: how advanced technology can improve implementation of effective programs. J Acquir Immune Defic Syndr. 2013;63:S72–84. sourceExperimental — accuracy not yet measured
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
- Partly supportedcites [91] Perrino T, Beardslee W, Bernal G, Brincks A, Cruden G, Howe G, Murry V, Pantin H, Prado G, Sandler I. Toward scientific equity for the prevention of depression and depressive symptoms in vulnerable youth. Prev Sci. 2015;16:642–51. sourceExperimental — accuracy not yet measured
The Implementation Science Research section (page unavailable) says implementation research can help make evidence-based interventions accessible to vulnerable groups in communities affected by health disparities. The Dissemination section (page unavailable) also says too few efficacious interventions have been widely adopted. Neither section says that more rapid implementation is especially important because these populations would otherwise miss out during the research needed to demonstrate effectiveness; the paper’s discussion concerns prevention for vulnerable youth specifically. (LLM, conf=0.85)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
- Partly supported
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We note that the logic behind borrowing strength from previous studies requires that we justify that key elements of the intervention still exist and are delivered with fidelity, that the delivery system retains critical components of the implementation strategy, and that broader ecological systems are still supportive of the delivery and sustainment of this intervention.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Defining a new concept for implementation—“scaling-out” > Scaling-out vs scaling-up
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In delivering PrEP to prevent HIV incidence, adherence to the medication would be the most important intermediate outcome.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Assuming a conceptual theory of mediation holds for scaling-out
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
In the paper
“Typically, a more limited evaluation would collect a small amount of—or even no—health outcome data, but a substantial amount of implementation process and output data, what others have called implementation outcomes [10].”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Evaluation options for scaling-out > EBI replicability or effectiveness
What the sources say
- Not checked yetcites [10] Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health Ment Health Serv Res. 2011;38:65–76. source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“When an EBI is implemented with fidelity in a setting that is identical to or very similar to where it was previously tested and found to be effective, it is reasonable to anticipate that the EBI would provide similar benefits to those found earlier.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Introduction
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
In the paper
“Staffing, training, and costs required to deliver PrEP in this setting are complex and need to be addressed for successful implementation in STD clinics [31].”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Two types of scaling-out: population fixed, different system, and delivery system fixed, different population, scaling-out > Type I: population fixed, different delivery system scaling-out
What the sources say
- Not checked yetcites [31] Weiss G. Implementing PrEP in STD clinics: findings from a 2015 assessment of local health department engagement in PrEP implementation. In: 2016 National STD Prevention Conference. Atlanta: CDC; 2016.
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Given that state laws and regulations vary in terms of parental consent requirements for medical services, PrEP implementation in this new population requires different strategies and research that can inform effective delivery [35–37].”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Two types of scaling-out: population fixed, different system, and delivery system fixed, different population, scaling-out > Type II: delivery system fixed, different population scaling-out
What the sources say
- Not checked yetcites [35] Culp L, Caucci L. State adolescent consent laws and implications for HIV pre-exposure prophylaxis. Am J Prev Med. 2013;44:S119–24. source
- Not checked yetcites [36] Mustanski B, Fisher CB. HIV rates are increasing in gay/bisexual teens: IRB barriers to research must be resolved to bend the curve. Am J Prev Med. 2016;51:249–52. source
- Not checked yetcites [37] Hosek S, Celum C, Wilson CM, Kapogiannis B, Delany-Moretlwe S, Bekker L-G. Preventing HIV among adolescents with oral PrEP: observations and challenges in the United States and South Africa. J Int AIDS Soc. 2016;19(7(Suppl 6)):21107. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Cook’s principle of causal explanation [5] requires investigation into the causal mediating mechanisms that underlie a relationship of interest, explaining how and why an effect occurs.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Assuming a conceptual theory of mediation holds for scaling-out
What the sources say
- Not checked yetcites [5] Cook TD. Meta-analysis: its potential for causal description and causal explanation within program evaluation. In: Social prevention and the social sciences: theoretical controversies, research problems, and evaluation strategies. Berlin: Walter de Gruyter; 1991. p. 245–85.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With a lack of empirical data in the new setting, the burden of proof necessarily rests on similarity with previous work.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Evaluation options for scaling-out > Levels of evidence for scale-out evaluations
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
None of the sources cited for this claim have been checked yet.
In the paper
“we focus on three key interconnected implementation/sustainment components: (1) the alignment of the surrounding ecological context including characteristics of the target population as well as broader cultural and contextual factors (e.g., policies, funding), (2) the alignment of the health or service delivery system and organizations where it”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Considerations for scaling-out > Intervention adaptation
What the sources say
- Not checked yetcites [47] Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health Ment Health Serv Res. 2011;38:4–23. source
- Not checked yetcites [40] Chambers DA, Glasgow R, Stange K. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8:117. source
- Not checked yetcites [48] Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:1. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We note that the logic behind borrowing strength from previous studies requires that we justify that key elements of the intervention still exist and are delivered with fidelity, that the delivery system retains critical components of the implementation strategy, and that broader ecological systems are still supportive of the delivery and sustainment of this intervention.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Defining a new concept for implementation—“scaling-out” > Scaling-out vs scaling-up
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
None of the sources cited for this claim have been checked yet.
In the paper
“This “adaptation-fidelity” tension is a critical component of scaling-out in that it is addressed head on as proposed in more dynamic models of implementation process [60–62].”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Considerations for scaling-out > Intervention adaptation
What the sources say
- Not checked yetcites [60] Stirman SW, Gutner CA, Langdon K, Graham JR. Bridging the gap between research and practice in mental health service settings: An overview of developments in implementation theory and research. Behav Ther. 2016;47(6):920–36. source
- Not checked yetcites [61] Kemp L. Adaption and fidelity: a recipe analogy for achieving both in population scale implementation. Prev Sci. 2016;17:429–38. source
- Not checked yetcites [62] Chambers DA, Norton WE. The adaptome: advancing the science of intervention adaptation. Am J Prev Med. 2016;51:S124–31. source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Overly strict intervention fidelity may be at odds with effective implementation of EBIs in real-world practice settings outside highly controlled efficacy trials.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Level 0 depends almost exclusively on the assumption that the new implementation will follow in the same footsteps as the previous intervention did when it produced findings of health impact.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Evaluation options for scaling-out > Levels of evidence for scale-out evaluations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- The authors state this as an assumption: The new implementation will follow in the same footsteps as the previous intervention did when it produced findings of health impact.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If testing of this new scale-out requires the full empirical evaluation that would be required for establishing an EBI, this would be exceptionally costly, time consuming, and would delay implementation, especially to populations underrepresented in scientific trials or in settings where its delivery could reasonably produce benefit.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Defining a new concept for implementation—“scaling-out”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We note that an actual design may involve different levels of assessment across the columns.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Evaluation options for scaling-out > Levels of evidence for scale-out evaluations
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We note that the logic behind borrowing strength from previous studies requires that we justify that key elements of the intervention still exist and are delivered with fidelity, that the delivery system retains critical components of the implementation strategy, and that broader ecological systems are still supportive of the delivery and sustainment of this intervention.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Defining a new concept for implementation—“scaling-out” > Scaling-out vs scaling-up
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We describe four levels of evidence and recommend that evidence of the effectiveness of scaling-out can be supported if we establish that mediational pathways have equivalent strength as they did in the original trials.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Defining a new concept for implementation—“scaling-out” > Scaling-out vs scaling-up
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
In the paper
“An expectation of beneficial impact when scaling-up relies upon Cook’s principle of proximal similarity [5] because a nearly identical intervention is delivered in the same way to a similar population.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Defining a new concept for implementation—“scaling-out” > Scaling-out vs scaling-up
What the sources say
- Not checked yetcites [5] Cook TD. Meta-analysis: its potential for causal description and causal explanation within program evaluation. In: Social prevention and the social sciences: theoretical controversies, research problems, and evaluation strategies. Berlin: Walter de Gruyter; 1991. p. 245–85.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Third, there must be sufficient organizational or system support to deliver the intervention as intended to sufficient numbers of the target population.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Defining a new concept for implementation—“scaling-out” > Empirical evidence needed for scaling-out
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For all types of scaling-out, it may be possible to use the identical EBI (e.g., delivered using the same manualized version and/or dosage). However, we more often need to adapt the intervention to fit the new population and/or new delivery system.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Considerations for scaling-out > Intervention adaptation
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In all variants of scaling-out, there is more concern about the impact on effectiveness and health outcomes, as there is more uncertainty whether the empirically supported causal association between intervention and outcome found in previous studies will hold when adapted and tested under yet-unstudied conditions.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Defining a new concept for implementation—“scaling-out” > Scaling-out vs scaling-up
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
In the paper
“First, even as the EBI is adapted to new settings or populations, it still must retain its core elements [12].”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Defining a new concept for implementation—“scaling-out” > Empirical evidence needed for scaling-out
What the sources say
- Not checked yetcites [12] Mohr DC, Schueller SM, Riley WT, Brown CH, Cuijpers P, Duan N, Kwasny MJ, Stiles-Shields C, Cheung K. Trials of intervention principles: evaluation methods for evolving behavioral intervention technologies. J Med Internet Res. 2015;17:e166. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Assuming the conceptual theory of mediation holds in a scaled-out adaptation, relying on Cook’s principles of heterogeneous irrelevancies and proximal similarity, then we could assess impact on the mediating variable using a level 1 design for assessing a proxy outcome.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Assuming a conceptual theory of mediation holds for scaling-out
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
- The authors state this as an assumption: The conceptual theory of mediation holds in a scaled-out adaptation.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We can also infer that positive effects would be expected to translate into corresponding benefit on the distal outcome as well.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Assuming a conceptual theory of mediation holds for scaling-out
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Intuitively one would expect that the more similar the delivery system, the broader the contextual setting, the intervention, and the population, the more we should be able to rely on previous evidence, and the less new empirical evidence should be needed, to anticipate a successful scaling-out.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Defining a new concept for implementation—“scaling-out” > Empirical evidence needed for scaling-out
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“Discussions of this issue most commonly pertain to adaptations to fit an EBI for a particular cultural group [58].”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Considerations for scaling-out > Core elements of the intervention
What the sources say
- Not checked yetcites [58] Castro FG, Barrera M Jr, Martinez CR Jr. The cultural adaptation of prevention interventions: resolving tensions between fidelity and fit. Prev Sci. 2004;5:41–5. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“However, adaptations to accommodate an EBI are likely to be needed at the service system or organization levels, and often to the EBI itself [47, 60].”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Considerations for scaling-out > Core elements of the intervention
What the sources say
- Not checked yetcites [47] Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health Ment Health Serv Res. 2011;38:4–23. source
- Not checked yetcites [60] Stirman SW, Gutner CA, Langdon K, Graham JR. Bridging the gap between research and practice in mental health service settings: An overview of developments in implementation theory and research. Behav Ther. 2016;47(6):920–36. source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Recent methodologic developments have addressed challenging issues in causality for single [77] and multiple randomized trials [25], evolving interventions [12], natural experiments [78, 79], and multilevel [78] as well as multidimensional situations [80].”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Discussion > Scaling-out
What the sources say
- Not checked yetcites [77] VanderWeele TJ. Mediation analysis: a practitioner's guide. Annu Rev Public Health. 2016;37:17–32. source
- Not checked yetcites [25] Huang S, MacKinnon DP, Perrino T, Gallo CG, Cruden G, Brown CH. A statistical method for synthesizing mediation analyses using the product of coefficient approach across multiple trials. Stat Methods Appl. 2016;25:565–79. source
- Not checked yetcites [12] Mohr DC, Schueller SM, Riley WT, Brown CH, Cuijpers P, Duan N, Kwasny MJ, Stiles-Shields C, Cheung K. Trials of intervention principles: evaluation methods for evolving behavioral intervention technologies. J Med Internet Res. 2015;17:e166. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For type III scaling-out where both population and service delivery are different (bottom right), logic would require more empirical examination that the mediational pathways remain in this new context compared to types I or II scaling-out. Potentially, proxy measures could be used to assess these mediational relationships, and generally some measures of the distal health outcome would be required”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Discussion > Scaling-out
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
In the paper
“Major implementation strategy adaptations often occur, and indeed are sometimes required to support implementation in delivery systems with widely different system and/or organizational cultures and climates, readiness, and resources [81], to populations having widely different histories, norms and values.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Discussion > Scaling-out
What the sources say
- Not checked yetcites [81] Czaja SJ, Valente TW, Nair SN, Villamar J, Brown CH. Characterizing implementation strategies using a systems engineering survey and interview tool: a comparison across 10 prevention programs for drug abuse and HIV sexual risk behaviors. Implement Sci. 2016;11:70. source
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“At the heart of the scale-out approach is borrowing strength through the use of mediational modeling.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Discussion > Scaling-out
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It seems natural to require much more evidence when both population and delivery system change; doing so would demand new empirical evidence beyond that for either delivery system or population fixed scaling-out.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Discussion > Scaling-out
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We take a mechanistic approach in suggesting that by testing underlying mechanisms or mediators of effects, the efficiency of testing EBIs for new populations or new service systems can be streamlined through a greater understanding of and use of prior data to borrow strength from previous effectiveness studies.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Discussion
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Scaling-out, while allowing for appropriate system, organization, and intervention adaptation, necessitates a better understanding of how to facilitate delivery of EBIs with appropriate content adherence and competence in delivery”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Considerations for scaling-out > Intervention adaptation
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
None of the sources cited for this claim have been checked yet.
In the paper
“That is, there are multiple implementation strategies that, in specific circumstances can effectively address different barriers to implementation, as well as the same barrier (e.g., financing) [83–85].”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Discussion > Scaling-out
What the sources say
- Not checked yetcites [83] Eaton J, McCay L, Semrau M, Chatterjee S, Baingana F, Araya R, Ntulo C, Thornicroft G, Saxena S. Scale-up of services for mental health in low-income and middle-income countries. Lancet. 2011;378:1592–603. source
- Not checked yetcites [84] Paina L, Peters DH. Understanding pathways for scaling-up health services through the lens of complex adaptive systems. Health Policy Plan. 2012;27:365–73. source
- Not checked yetcites [85] Mangham LJ, Hanson K. Scaling-up in international health: what are the key issues? Health Policy Plan. 2010;25:85–96. source
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Under this principle, generalization is justified when all relevant properties of the causal association, such as prototypical and necessary components of an EBI, are adequately represented in the new context.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Considerations for scaling-out > Core elements of the intervention
What the sources say
- Not checked yetcites [5] Cook TD. Meta-analysis: its potential for causal description and causal explanation within program evaluation. In: Social prevention and the social sciences: theoretical controversies, research problems, and evaluation strategies. Berlin: Walter de Gruyter; 1991. p. 245–85.
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Scaling-out provides an opportunity to use a strategic approach to improve the efficiency of moving an EBI from one setting to another and/or from one population to another.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Discussion
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Mediational analysis has an extensive history [74, 75] but has had increased interest in implementation science [76].”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Discussion > Scaling-out
What the sources say
- Not checked yetcites [74] MacKinnon DP. Introduction to statistical mediation analysis. Mahwah, NJ: Lawrence Erlbaum Associates; 2008.
- Not checked yetcites [75] MacKinnon DP, Kisbu-Sakarya Y, Gottschall AC. 16 developments in mediation analysis. The Oxford Handbook of Quantitative Methods in Psychology: Vol 2: Statistical Analysis. 2013;2:338.
- Not checked yetcites [76] Anselmi L, Binyaruka P, Borghi J. Understanding causal pathways within health systems policy evaluation through mediation analysis: an application to payment for performance (P4P) in Tanzania. Implement Sci. 2017;12:10. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Similar scaling-out approaches are possible when implementing other types of interventions including evidence-based decision support systems, such as Communities that Care [68–72], which helps communities decide which among a menu of EBIs are most appropriate to implement, rather than a single intervention.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Discussion > Scaling-out
What the sources say
- Not checked yetcites [68] Hawkins JD, Oesterle S, Brown EC, Abbott RD, Catalano RF. Youth problem behaviors 8 years after implementing the communities that care prevention system: a community-randomized trial. JAMA Pediatr. 2014;168:122–9. source
- Not checked yetcites [69] Hawkins JD, Catalano Jr RF. Communities that care: action for drug abuse prevention. San Francisco: Jossey-Bass; 1992.
- Not checked yetcites [70] Fagan AA, Hanson K, Hawkins JD, Arthur MW. Translational research in action: implementation of the communities that care prevention system in 12 communities. J Community Psychol. 2009;37:809–29. source
- Not checked yetcites [71] Brown EC, Hawkins JD, Arthur MW, Briney JS, Fagan AA. Prevention service system transformation using communities that care. J Community Psychol. 2011;39:183–201. source
- Not checked yetcites [72] Brown E, Hawkins JD, Rhew I, Shapiro V, Abbott R, Oesterle S, Arthur M, Briney J, Catalano R. Prevention system mediation of communities that care effects on youth outcomes. Prev Sci. 2014;15:623–32. source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For type II delivery system fixed, different population scaling-out (lower left), we would typically want to assess reach with high accuracy due to the focus on a new population, but may make the case that assessing implementation fidelity, adoption, and sustainment could be done with proxy measures.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Discussion > Scaling-out
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For type I population fixed, different delivery system scaling-out (upper right), the mediational relationships are again expected to hold, reach could be measured with a proxy, but because the delivery system changes we would need stronger evidence around implementation fidelity, adoption, and sustainment.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Discussion > Scaling-out
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The less system/organization adaptation needed, the more readily an EBI can be assimilated.”
§ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems > Considerations for scaling-out > Core elements of the intervention
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper relies on this without support.
Adolescents aged 13–19 who engage in risky sexual behavior and/or injection drugs are at increased risk of HIV infection and accounted for 4% of new HIV diagnoses in 2015.
“However, adolescents between the ages of 13–19 who engage in risky sexual behavior and/or injection drugs are also at increased risk of HIV infection, and accounted for 4% of new HIV diagnoses in 2015.”
The authors state this as an assumption.
The conceptual theory of mediation holds in a scaled-out adaptation.
“Assuming the conceptual theory of mediation holds in a scaled-out adaptation, relying on Cook’s principles of heterogeneous irrelevancies and proximal similarity, then we could assess impact on the mediating variable using a level 1 design for assessing a proxy outcome.”
The authors state this as an assumption.
The new implementation will follow in the same footsteps as the previous intervention did when it produced findings of health impact.
“Level 0 depends almost exclusively on the assumption that the new implementation will follow in the same footsteps as the previous intervention did when it produced findings of health impact.”
The paper relies on this without support.
Overly strict intervention fidelity may be at odds with effective implementation of EBIs in real-world practice settings outside highly controlled efficacy trials.
“However, there is mounting evidence that overly strict intervention fidelity may be at odds with effective implementation of EBIs in real-world practice settings (i.e., outside of highly controlled efficacy trials), thus, raising concern about the balance between delivering EBIs with fidelity and making adaptations believed to be necessary for usual care contexts.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A hybrid type 2 design is ideal when studying interventions that already have evidence of effectiveness in other settings or populations, but not in either the context or population in the current trial resulting in uncertainty that there would be a similar clinical benefit (Aarons et al., 2017).”
From An introduction to effectiveness-implementation hybrid designs (2019) · cites this paper as Aarons et al., 2017
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“During scale-up or scale-out, other changes may be necessary, particularly for contexts where the intervention is likely to reach populations that may differ from the population that received the intervention during implementation [43].”
From The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions (2019) · cites this paper as [43]
Citations
Cites
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496
(1979) · Medical Entomology and Zoology · Cited by 11,252
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0319-7 · Cited by 8,721
Self-citationEvaluating the public health impact of health promotion interventions: the RE-AIM framework.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999) · American Journal of Public Health · doi:10.2105/ajph.89.9.1322 · Cited by 7,217
Introduction to Statistical Mediation Analysis
MacKinnon, D. P. (2012) · doi:10.4324/9780203809556 · Cited by 5,618
Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012) · Medical Care · doi:10.1097/mlr.0b013e3182408812 · Cited by 4,058
Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0327-7 · Cited by 3,240 · cited 2×
“However, adaptations to accommodate an EBI are likely to be needed at the service system or organization levels, and often to the EBI itself [47, 60].”
BackgroundMethodSelf-citationImplementation strategies: recommendations for specifying and reporting
Proctor, E. K., Powell, B. J., & McMillen, J. C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-139 · Cited by 2,585
Mediation Analysis: A Practitioner's Guide
VanderWeele, T. J. (2016) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-032315-021402 · Cited by 1,985
The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change
Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-117 · Cited by 1,724
Research works printed but not linked: 3
- Institute of Medicine. National Research Council: preventing mental, emotional, and behavioral disorders among young people: progress and possibilities. Washington, DC: The National Academies Press; 2009.
- U.S. Public Health Service. In: Services USDoHaH, editor. Preexposure prophylaxis for the prevention of HIV infection in the United States - 2014: a clinical practice guideline. Atlanta: Centers for Disease Control and Prevention; 2014.
- MacKinnon DP. Introduction to statistical mediation analysis. Mahwah, NJ: Lawrence Erlbaum Associates; 2008.
Cited by
Sorted by citations
Items 401–405 of 405
Seff, I., Deitch, J., Schulhofer, L., Rodriguez, C., Pena, E. D. R., Roa, A. H., & Stark, L. (2026) · Women s Health · doi:10.1177/17455057261495653
Bruzios, K. E., Hugh, M. L., Joshi, M., Gaias, L. M., Pullmann, M., Reyes, R., Cueto, A. P. M., Goosey, R., Landa, Y., Corbin, C. M., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-11001359/v1
Braaten, K., Grieve, N. J., Weatherson, K. A., Cranston, K. D., PERCIVAL, S., Bourne, J. E., Young, R., & Jung, M. E. (2026) · International Journal of Behavioral Medicine · doi:10.1007/s12529-026-10493-z
Schwarz, A. F., van Bottenburg, M., & Simons, M. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001792
Fontaine, G., Nicholls, W., Rudge, T., Day, E., Taylor, N., Robles, A. T., Marshall, A. D., Lafferty, L., Silk, D., Comben, S., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-11176279/v1
Metadata
- Authors
- Aarons, G. A., Sklar, M., Mustanski, B., Benbow, N., & Brown, C. H.
- Affiliations
- University of California San Diego (US)
- Brown University (US)
- Northwestern University (US)
- Corresponding author
- Aarons, Gregory A.
- Venue
- Implementation Science
- Published
- 2017-09-06
- Volume
- 12
- Issue
- 1
- Pages
- 111-111
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC5588712
- OpenAlex
- W2751779860
- Semantic Scholar
- fa16633491dbb36e730405e32620ef57ee0196f9
- Cited by
- 399
- References
- 91
- Citations by year
- 2026: 25 · 2025: 52 · 2024: 41 · 2023: 41 · 2022: 56 · 2021: 66 · 2020: 51 · 2019: 51 · 2018: 16
- FWCI
- 5.69
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Implementation Science (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- eScholarship (California Digital Library) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- Funders
- National Institute on Drug Abuse (P30DA027828, R01DA038466)
- National Institute of Mental Health (R01MH072961, R01MH092950)
- Agency for Healthcare Research and Quality (F32HS024192)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- horizontal scaling, evidence-based interventions, generalizability, intervention adaptation, health care delivery systems, multilevel mediation, implementation fidelity, intervention generalizability
- MeSH terms
- Delivery of Health Care, Health Plan Implementation, Humans, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
