Efficacy vs Effectiveness Trial Results of an Indicated “Model” Substance Abuse Program: Implications for Public Health
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Objectives. The US Department of Education requires schools to choose substance abuse and violence prevention programs that meet standards of effectiveness. The Substance Abuse and Mental Health Services Agency certifies “model” programs that meet this standard. We compared findings from a large, multisite effectiveness trial of 1 model program to its efficacy trial findings, upon which the certification was based. Methods. 1370 high-risk youths were randomized to experimental or control groups across 9 high schools in 2 large urban school districts. We used intent-to-treat and on-treatment approaches to examine baseline equivalence, attrition, and group differences in outcomes at the end of the program and at a 6-month follow-up. Results. Positive efficacy trial findings were not replicated in the effectiveness trial. All main effects were either null or worse for the experimental than for the control group. Conclusions. These findings suggest that small efficacy trials conducted by developers provide insufficient evidence of effectiveness. Federal agencies and public health scientists must work together to raise the standards of evidence and ensure that data from new trials are incorporated into ongoing assessments of program effects.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reducing Risks for Mental Disorders: Frontiers for Preventive Intervention Research
(1994) · doi:10.17226/2139 · Cited by 2,073
Flay, B. R. (1986) · Preventive Medicine · doi:10.1016/0091-7435(86)90024-1 · Cited by 723
Post-randomisation exclusions: the intention to treat principle and excluding patients from analysis
Fergusson, D., Aaron, S. D., Guyatt, G., & Hébert, P. (2002) · BMJ · doi:10.1136/bmj.325.7365.652 · Cited by 708
Glasgow, R. E., Klesges, L. M., Dzewaltowski, D. A., Bull, S. S., & Estabrooks, P. (2004) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm2701_2 · Cited by 635
The Intention-to-Treat Principle: How to Assess the True Effect of Choosing a Medical Treatment
Detry, M. A., & Lewis, R. J. (2014) · JAMA · doi:10.1001/jama.2014.7523 · Cited by 634
Cited by
Gottfredson, D. C., Cook, T. D., Gardner, F. E. M., Gorman-Smith, D., Howe, G. W., Sandler, I. N., & Zafft, K. M. (2015) · Prevention Science · doi:10.1007/s11121-015-0555-x · Cited by 627
Evidence-based Kernels: Fundamental Units of Behavioral Influence
Embry, D. D., & Biglan, A. (2008) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-008-0036-x · Cited by 396
“A fifth concern is that existing programs have limited effectiveness, modest effect sizes, scalability concerns, weak generalization, difficulty with maintenance or sus- tainability, and even iatrogenic effects (e.g., Hallfors et al. 2006; St. Pierre et al. 2005; Sanchez et al. 2007).”
BackgroundComputer-delivered interventions to reduce college student drinking: A meta-analysis
Carey, K. B., Scott‐Sheldon, L. A. J., Elliott, J. C., Bolles, J. R., & Carey, M. P. (2009) · Addiction · doi:10.1111/j.1360-0443.2009.02691.x · Cited by 250
“These trends suggest as evaluation trials become larger and greater dissemination is achieved, the likelihood of detecting differences between CDIs and alternatives is reduced (consistent with the transition from efficacy to effectiveness [34]).”
BackgroundThe Tug-of-War: Fidelity Versus Adaptation Throughout the Health Promotion Program Life Cycle
Bopp, M., Saunders, R. P., & Lattimore, D. (2013) · The Journal of Primary Prevention · doi:10.1007/s10935-013-0299-y · Cited by 150
“There are exceptions to this [e.g., CATCH (McGraw et al., 1996) and Reconnecting Youth (Hallfors et al., 2006; Sanchez et al., 2007)], which are likely related to the influence of contextual factors.”
BackgroundEisner, M., Nagin, D., Ribeaud, D., & Malti, T. (2012) · Prevention Science · doi:10.1007/s11121-011-0266-x · Cited by 89
Metadata
- Authors
- Hallfors, D., Cho, H., Sanchez, V., Khatapoush, S., Kim, H. M., & Bauer, D.
- Venue
- American Journal of Public Health
- Publisher
- American Public Health Association
- Published
- 2006-12
- Volume
- 96
- Issue
- 12
- Pages
- 2254-2259
- ISSN
- 0090-0036 (ISSN-L)
- 1541-0048
- OpenAlex
- W2051668009
- Cited by
- 88
- References
- 5
- FWCI
- 6.42
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.