Standards of Evidence: Criteria for Efficacy, Effectiveness and Dissemination
The DOI matches its Crossref record; 131 of the 138 references it cites were found in a registry or an index.
Abstract
Ever increasing demands for accountability, together with the proliferation of lists of evidence-based prevention programs and policies, led the Society for Prevention Research to charge a committee with establishing standards for identifying effective prevention programs and policies. Recognizing that interventions that are effective and ready for dissemination are a subset of effective programs and policies, and that effective programs and policies are a subset of efficacious interventions, SPR's Standards Committee developed overlapping sets of standards. We designed these Standards to assist practitioners, policy makers, and administrators to determine which interventions are efficacious, which are effective, and which are ready for dissemination. Under these Standards, an efficacious intervention will have been tested in at least two rigorous trials that (1) involved defined samples from defined populations, (2) used psychometrically sound measures and data collection procedures; (3) analyzed their data with rigorous statistical approaches; (4) showed consistent positive effects (without serious iatrogenic effects); and (5) reported at least one significant long-term follow-up. An effective intervention under these Standards will not only meet all standards for efficacious interventions, but also will have (1) manuals, appropriate training, and technical support available to allow third parties to adopt and implement the intervention; (2) been evaluated under real-world conditions in studies that included sound measurement of the level of implementation and engagement of the target audience (in both the intervention and control conditions); (3) indicated the practical importance of intervention outcome effects; and (4) clearly demonstrated to whom intervention findings can be generalized. An intervention recognized as ready for broad dissemination under these Standards will not only meet all standards for efficacious and effective interventions, but will also provide (1) evidence of the ability to "go to scale"; (2) clear cost information; and (3) monitoring and evaluation tools so that adopting agencies can monitor or evaluate how well the intervention works in their settings. Finally, the Standards Committee identified possible standards desirable for current and future areas of prevention science as the field develops. If successful, these Standards will inform efforts in the field to find prevention programs and policies that are of proven efficacy, effectiveness, or readiness for adoption and will guide prevention scientists as they seek to discover, research, and bring to the field new prevention programs and policies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Only four of the studies achieved ‘fair’ or ‘good’ methodological quality [65, 66] plus a moderate to high level of evidence [1, 69].”
From Incorporating animal-assisted therapy in mental health treatments for adolescents: A systematic review of canine assisted psychotherapy (2019) · cites this paper as [1]
In the citing paper
“They did not consider standards for research on broad dissemination efforts, but rather assumed that once an intervention had cleared all hurdles related to efficacy and effectiveness, further rigorous research on its effects was not necessary as long as fidelity to the proven model was demonstrated.”
From Standards of Evidence for Efficacy, Effectiveness, and Scale-up Research in Prevention Science: Next Generation (2015) · cites this paper as Flay et al. 2005
In the citing paper
“They therefore provided additional criteria that must be met in order to justify a decision to broadly disseminate.”
From Standards of Evidence for Efficacy, Effectiveness, and Scale-up Research in Prevention Science: Next Generation (2015) · cites this paper as Flay et al. 2005
In the citing paper
“The earlier standards instead recommended that monitoring and evaluation tools suitable for use by practitioners be made available to adopting organizations so that they could demonstrate to their constituencies that their prevention dollars were well spent.”
From Standards of Evidence for Efficacy, Effectiveness, and Scale-up Research in Prevention Science: Next Generation (2015) · cites this paper as Flay et al. 2005
In the citing paper
“Flay et al. (2005) defined efficacy trials as studies of programs or policies delivered under optimal conditions, and effectiveness trials as studies conducted in real-world conditions.”
From Standards of Evidence for Efficacy, Effectiveness, and Scale-up Research in Prevention Science: Next Generation (2015) · cites this paper as Flay et al. 2005
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Baron, R. M., & Kenny, D. A. (1986) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.51.6.1173 · Cited by 91,532
Hierarchical Linear Models: Applications and Data Analysis Methods.
Longford, N. T., Bryk, A. S., & Raudenbush, S. W. (1993) · Contemporary Sociology · doi:10.2307/2075823 · Cited by 18,724
(2002) · Cited by 13,211
Missing data: our view of the state of the art
Schafer, J. L., & Graham, J. W. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.2.147 · Cited by 11,638
(1979) · Medical Entomology and Zoology · Cited by 11,252
Gottfredson, M. R., & Hirschi, T. (1990) · doi:10.1515/9781503621794 · Cited by 10,127
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436 · cited 1×
“Otherwise, claims of effectiveness are likely to be biased or untrue and it is necessary to consider competing explanations that could lead to the expected outcome (Bertrand et al., 2002; Holland, 1986; Manski, 1995; Rubin, 1974; Shadish et al., 2002).”
BackgroundQuasi-experimentation: Design and analysis issues for field settings
Shrout, P. E. (1980) · Evaluation and Program Planning · doi:10.1016/0149-7189(80)90063-4 · Cited by 6,010
Statistics and Causal Inference
Holland, P. W. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478354 · Cited by 5,293 · cited 1×
“Otherwise, claims of effectiveness are likely to be biased or untrue and it is necessary to consider competing explanations that could lead to the expected outcome (Bertrand et al., 2002; Holland, 1986; Manski, 1995; Rubin, 1974; Shadish et al., 2002).”
BackgroundRosenbaum, P. R., & Rubin, D. B. (1985) · The American Statistician · doi:10.2307/2683903 · Cited by 4,774
Research works printed but not linked: 6
- Biglan, A., Brennan, P. A., Foster, S. L., Holder, H. D., Miller, T. L., Cunningham, P. B., et al. (2004). Helping adolescents at risk: Prevention of multiple problems of youth. New York: Guilford.
- Boruch, R. F. (2005b). Comments on Jacob and Ludwig. In D. Ravitch (Ed.), Brookings papers on educational policy (pp. 67–73). Washington, DC: Brookings Institution Press.
- Greenberg, M., Domitrovich, C., Graczyk, P., & Zins, J. (2001). The study of implementation in school-based preventive interventions: Theory, research and practice. Washington, DC: Center for Mental Health Services, Substance Abuse and Mental Health Administration, U.S. Department of Health and Human Services.
- Jacob, B., & Ludwig, J. (2005). Can the federal government improve government research? In D. Ravitch (Ed.), Brookings papers on educational policy (pp. 47–66). Washington, DC: Brookings Institution Press.
- Kellam (2000). Community and institutional partnerships for school violence prevention. In Preventing School Violence: Plenary Papers of the 1999 Conference on Criminal Justice. Research and Evaluation–Enhancing Policy and Practice Through Research, Volume 2 NCJ 180972 (pp. 1–21). Washington, DC.
- US Department of Education. (1998). Safe and drug-free schools program: Notice of final principles of effectiveness. Federal Register 63(104), 29901–29906.
Cited by
Systematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,245
(2009) · Cited by 2,013
School Bullying: Development and Some Important Challenges
Olweus, D. (2013) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050212-185516 · Cited by 1,395
Single-Case Intervention Research Design Standards
Kratochwill, T. R., Hitchcock, J. H., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2013) · Remedial and Special Education · doi:10.1177/0741932512452794 · Cited by 1,331
“Third, SCDs have contributed greatly to the “evidencebased practice” movement (Flay et al., 2005), and we deemed that inclusion of these methodologies was important for advancing the scientific knowledge base on educational practices in schools.”
BackgroundThe Effectiveness of Telemental Health: A 2013 Review
Hilty, D. M., Ferrer, D. C., Parish, M. B., Johnston, B., Callahan, E. J., & Yellowlees, P. M. (2013) · Telemedicine and e-Health · doi:10.1089/tmj.2013.0075 · Cited by 1,048
Metadata
- Authors
- Flay, B. R., Biglan, A., Boruch, R. F., Castro, F. G., Gottfredson, D., Kellam, S., Mościcki, E. K., Schinke, S., Valentine, J. C., & Ji, P.
- Affiliations
- University of Illinois Chicago (US)
- Oregon Research Institute (US)
- University of Pennsylvania (US)
- Arizona State University (US)
- University of Maryland, College Park (US)
- American Institutes for Research (US)
- National Institute of Mental Health (US)
- Columbia University (US)
- Duke University (US)
- ORCID
- Venue
- Prevention Science
- Published
- 2005-05-16
- Volume
- 6
- Issue
- 3
- Pages
- 151-175
- ISSN
- 1389-4986 (ISSN-L)
- 1573-6695
- OpenAlex
- W2015939282
- Cited by
- 1,338
- References
- 138
- Citations by year
- 2026: 21 · 2025: 21 · 2024: 35 · 2023: 43 · 2022: 59 · 2021: 50 · 2020: 108 · 2019: 75 · 2018: 85 · 2017: 71 · 2016: 91 · 2015: 103 · 2014: 93 · 2013: 76 · 2012: 102
- FWCI
- 70.60
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Prevention Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- evidence standards, prevention programs, prevention policy, efficacy, effectiveness, dissemination, evidence-based practice, implementation fidelity, generalizability, cost information, monitoring and evaluation tools, iatrogenic disease, long-term follow-up
- MeSH terms
- Clinical Trials as Topic, Humans, Policy Making, Preventive Medicine, United States, Treatment Outcome, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Standards of Evidence: Criteria for Efficacy, Effectiveness and Dissemination. https://app.cochat.ai/papers/10.1007/s11121-005-5553-y
- The paper
- Flay, B. R., Biglan, A., Boruch, R. F., Castro, F. G., Gottfredson, D., Kellam, S., Mościcki, E. K., Schinke, S., Valentine, J. C., & Ji, P. (2005). Standards of Evidence: Criteria for Efficacy, Effectiveness and Dissemination. Prevention Science. https://doi.org/10.1007/s11121-005-5553-y
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.