Examining the effects of prevention programs on the incidence of new cases of mental disorders: the lack of statistical power
Abstract
Objective In the past few decades about 1,000 controlled studies have examined the effects of mental health prevention programs, but few studies have examined the effects of such programs on the incidence of new cases of mental disorders defined according to diagnostic criteria. A major reason why so few studies have examined this important question is that very large numbers of subjects are needed to provide sufficient statistical power for these studies. In this article this power problem is explored. Method Power calculations are presented for studies examining universal prevention (aimed at the general population regardless of risk status), selective prevention (aimed at high-risk groups), and indicated prevention (aimed at subjects who have some symptoms of a disorder without meeting full diagnostic criteria). Results Studies examining universal prevention are hardly feasible, as the number of subjects required amounts to tens of thousands at least. Research examining selective prevention is more feasible, but the number of subjects needed for these studies is still very high. Studies of indicated prevention are possible. Three major studies of indicated prevention examining the effects on the incidence of new cases of mental disorders are described. Conclusions There are several strategies for increasing statistical power in prevention studies: 1) focus on high-incidence groups (by concentrating on indicated prevention, by targeting high-risk groups with multiple risk f…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The evidence for universal interventions is generally weak.
“The evidence for universal interventions is generally weak, but conducting studies with a large enough sample size and consequently sufficient power to detect the very small effect sizes that are achievable if the interventions do work is challenging. 101”
From The Lancet Psychiatry Commission on youth mental health (2024) · cites this paper as [101]
Conducting studies with a large enough sample size and sufficient power to detect the very small effect sizes achievable if universal interventions do work is challenging.
“The evidence for universal interventions is generally weak, but conducting studies with a large enough sample size and consequently sufficient power to detect the very small effect sizes that are achievable if the interventions do work is challenging. 101”
From The Lancet Psychiatry Commission on youth mental health (2024) · cites this paper as [101]
Cites
Reducing Risks for Mental Disorders: Frontiers for Preventive Intervention Research
(1994) · doi:10.17226/2139 · Cited by 2,073 · see citing works
The benefits of interventions for work-related stress
(2001) · American Journal of Public Health · doi:10.2105/ajph.91.2.270 · Cited by 999 · see citing works
Primary prevention mental health programs for children and adolescents: A meta-analytic review
Durlak, J. A., & Wells, A. M. (1997) · American Journal of Community Psychology · doi:10.1023/a:1024654026646 · Cited by 963 · see citing works
Clarke, G. N., Hornbrook, M., Lynch, F., Polen, M., Gale, J., Beardslee, W., O'Connor, E., & Seeley, J. (2001) · Archives of General Psychiatry · doi:10.1001/archpsyc.58.12.1127 · Cited by 615 · see citing works
The Course of Depression in Adult Outpatients: Results From the Medical Outcomes Study
Wells, K. B. (1992) · Archives of General Psychiatry · doi:10.1001/archpsyc.1992.01820100032007 · Cited by 475 · see citing works
Zlotnick, C., Johnson, S. L., Miller, I. W., Pearlstein, T., & Howard, M. (2001) · American Journal of Psychiatry · doi:10.1176/appi.ajp.158.4.638 · Cited by 304 · see citing works
Child Sexual Abuse Prevention Programs: A Meta-Analysis
Davis, M. K., & Gidycz, C. A. (2000) · Journal of Clinical Child Psychology · doi:10.1207/s15374424jccp2902_11 · Cited by 255 · see citing works
Prevention of child sexual abuse victimization: a meta-analysis of school programs
Rispens, J., Aleman, A., & Goudena, P. P. (1997) · Child Abuse & Neglect · doi:10.1016/s0145-2134(97)00058-6 · Cited by 186 · see citing works
Child sexual abuse prevention: research review and recommendations
Berrick, J. D., & Barth, R. P. (1992) · Social Work Research and Abstracts · doi:10.1093/swra/28.4.6 · Cited by 71 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Amminger, G. P., Schäfer, M. R., Papageorgiou, K., Klier, C. M., Cotton, S. M., Harrigan, S. M., Mackinnon, A., McGorry, P. D., & Berger, G. E. (2010) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2009.192 · Cited by 896 · see citing works
Harvey, S. B., Modini, M., Joyce, S., Milligan-Saville, J. S., Tan, L., Mykletun, A., Bryant, R. A., Christensen, H., & Mitchell, P. B. (2017) · Occupational and Environmental Medicine · doi:10.1136/oemed-2016-104015 · Cited by 799 · see citing works
The prevention of depressive symptoms in children and adolescents: a meta-analytic review
Horowitz, J. L., & Garber, J. (2006) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.74.3.401 · Cited by 803 · see citing works
The Lancet Psychiatry Commission on youth mental health
McGorry, P. D., Mei, C., Dalal, N., Alvarez-Jimenez, M., Blakemore, S.-J., Browne, V., Dooley, B., Hickie, I. B., Jones, P. B., McDaid, D., et al. (2024) · The Lancet Psychiatry · doi:10.1016/s2215-0366(24)00163-9 · Cited by 663 · see citing works
Cuijpers, P., & Smit, F. (2004) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2004.00301.x · Cited by 595 · see citing works
Self-citationBeyond the “at risk mental state” concept: transitioning to transdiagnostic psychiatry
McGorry, P. D., Hartmann, J. A., Spooner, R., & Nelson, B. (2018) · World Psychiatry · doi:10.1002/wps.20514 · Cited by 488 · see citing works
Preventing the Onset of Depressive Disorders: A Meta-Analytic Review of Psychological Interventions
Cuijpers, P., van Straten, A., Smit, F., Mihalopoulos, C., & Beekman, A. (2008) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2008.07091422 · Cited by 456 · see citing works
Self-citationBorderline personality disorder in adolescence
Kaess, M., Brunner, R., & Chanen, A. (2014) · Pediatrics · doi:10.1542/peds.2013-3677 · Cited by 386 · see citing works
Prevention of major depression
Muñoz, R. F., Cuijpers, P., Smit, F., Barrera, A. Z., & Leykin, Y. (2010) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-033109-132040 · Cited by 326 · see citing works
Self-citationTransdiagnostic clinical staging in youth mental health: a first international consensus statement
Shah, J. L., Scott, J., McGorry, P. D., Cross, S. P. M., Keshavan, M. S., Nelson, B., Wood, S. J., Marwaha, S., Yung, A. R., Scott, E. M., et al. (2020) · World Psychiatry · doi:10.1002/wps.20745 · Cited by 291 · see citing works
Chanen, A. M., & McCutcheon, L. (2013) · British Journal of Psychiatry · doi:10.1192/bjp.bp.112.119180 · Cited by 262 · see citing works
Deady, M., Choi, I., Calvo, R. A., Glozier, N., Christensen, H., & Harvey, S. B. (2017) · BMC Psychiatry · doi:10.1186/s12888-017-1473-1 · Cited by 222 · see citing works
Moving towards a population health approach to the primary prevention of common mental disorders
Jacka, F. N., Mykletun, A., & Berk, M. (2012) · BMC Medicine · doi:10.1186/1741-7015-10-149 · Cited by 235 · see citing works
“Beyond such trials in clinical samples, proving the effectiveness of universal prevention is also problematic, given the substantial statistical power required [42].”
BackgroundApplying clinical staging to young people who present for mental health care
Hickie, I. B., Scott, E. M., Hermens, D. F., Naismith, S. L., Guastella, A. J., Kaur, M., Sidis, A., Whitwell, B., Glozier, N., Davenport, T., et al. (2013) · Early Intervention in Psychiatry · doi:10.1111/j.1751-7893.2012.00366.x · Cited by 229 · see citing works
Horowitz, J. L., Garber, J., Ciesla, J. A., Young, J. F., & Mufson, L. (2007) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.75.5.693 · Cited by 236 · see citing works
Mykletun, A. (2006) · American Journal of Psychiatry · doi:10.1176/appi.ajp.163.8.1412 · Cited by 237 · see citing works
Tan, L., Wang, M.-J., Modini, M., Joyce, S., Mykletun, A., Christensen, H., & Harvey, S. B. (2014) · BMC Medicine · doi:10.1186/1741-7015-12-74 · Cited by 208 · see citing works
Glozier, N., Martiniuk, A., Patton, G., Ivers, R., Li, Q., Hickie, I., Senserrick, T., Woodward, M., Norton, R., & Stevenson, M. (2010) · Sleep · doi:10.1093/sleep/33.9.1139 · Cited by 177 · see citing works
Okereke, O. I., Reynolds, C. F., Mischoulon, D., Chang, G., Vyas, C. M., Cook, N. R., Weinberg, A., Bubes, V., Copeland, T., Friedenberg, G., et al. (2020) · JAMA · doi:10.1001/jama.2020.10224 · Cited by 171 · see citing works
Ratheesh, A., Davey, C., Hetrick, S., Alvarez‐Jimenez, M., Voutier, C., Bechdolf, A., McGorry, P. D., Scott, J., Berk, M., & Cotton, S. M. (2017) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.12686 · Cited by 142 · see citing works
Tandon, S. D., Leis, J. A., Mendelson, T., Perry, D. F., & Kemp, K. (2014) · Maternal and Child Health Journal · doi:10.1007/s10995-013-1313-y · Cited by 142 · see citing works
Linardon, J., Gleeson, J., Yap, K., Murphy, K., & Brennan, L. (2019) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2018.1517389 · Cited by 134 · see citing works
“At present, “pure” prevention trials on third-wave interventions have not been conducted (i.e. trials that evaluate whether intervention programs actually prevent illness onset; Cuijpers, 2003).”
BackgroundOkereke, O. I., Vyas, C. M., Mischoulon, D., Chang, G., Cook, N. R., Weinberg, A., Bubes, V., Copeland, T., Friedenberg, G., Lee, I.-M., et al. (2021) · JAMA · doi:10.1001/jama.2021.21187 · Cited by 121 · see citing works
Sportel, B. E., de Hullu, E., de Jong, P. J., & Nauta, M. H. (2013) · PLoS ONE · doi:10.1371/journal.pone.0064355 · Cited by 119 · see citing works
“Definitive conclusions on the efficacy of early interventions such as described here should be drawn on the basis of meta-analysis of multiple studies [47].”
BackgroundSijbrandij, M., Horn, R., Esliker, R., O’May, F., Reiffers, R., Ruttenberg, L., Stam, K., de Jong, J., & Ager, A. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17020484 · Cited by 115 · see citing works
The prevention of schizophrenia
Yung, A. R., Killackey, E., Hetrick, S. E., Parker, A. G., Schultze-Lutter, F., Klosterkoetter, J., Purcell, R., & Mcgorry, P. D. (2007) · International Review of Psychiatry · doi:10.1080/09540260701797803 · Cited by 104 · see citing works
Randomized controlled trial to prevent postpartum depression in adolescent mothers
Phipps, M. G., Raker, C. A., Ware, C. F., & Zlotnick, C. (2013) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2012.12.036 · Cited by 99 · see citing works
Prevention, innovation and implementation science in mental health: the next wave of reform
McGorry, P. (2013) · British Journal of Psychiatry · doi:10.1192/bjp.bp.112.119222 · Cited by 56 · see citing works
Ultra-high-risk paradigm: lessons learnt and new directions
McGorry, P. D., & Mei, C. (2018) · Evidence Based Mental Health · doi:10.1136/ebmental-2018-300061 · Cited by 52 · see citing works
McGorry, P. (2013) · The Canadian Journal of Psychiatry · doi:10.1177/070674371305800105 · Cited by 41 · see citing works
Roberts, C. M., Kane, R. T., Rooney, R. M., Pintabona, Y., Baughman, N., Hassan, S., Cross, D., Zubrick, S. R., & Silburn, S. R. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01392 · Cited by 36 · see citing works
Meijer, J. L., Bockting, C. L., Beijers, C., Verbeek, T., Stant, A. D., Ormel, J., Stolk, R. P., de Jonge, P., van Pampus, M. G., & Burger, H. (2011) · Trials · doi:10.1186/1745-6215-12-157 · Cited by 28 · see citing works
Social and clinical interventions after conflict or other large disaster
van Ommeren, M., Morris, J., & Saxena, S. (2008) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2008.06.021 · Cited by 16 · see citing works
Scott, J., Crouse, J. J., Ho, N., Iorfino, F., Martin, N., Parker, R., McGrath, J., Gillespie, N. A., Medland, S., & Hickie, I. B. (2021) · PLOS ONE · doi:10.1371/journal.pone.0252550 · Cited by 15 · see citing works
Mor, S., Botella, C., Campos, D., Tur, C., Castilla, D., Soler, C., & Quero, S. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100387 · Cited by 6 · see citing works
- Authors
- Cuijpers, P.
- Affiliations
- Trimbos Institute (NL)
- Corresponding author
- Cuijpers, Pim
- ORCID
- Cuijpers, Pim
- Venue
- American Journal of Psychiatry
- Published
- 2003-08-01
- Volume
- 160
- Issue
- 8
- Pages
- 1385-1391
- ISSN
- 0002-953X (ISSN-L)
- 1535-7228
- OpenAlex
- W2134404467
- Cited by
- 217
- References
- 9
- Citations by year
- 2026: 5 · 2025: 4 · 2024: 5 · 2023: 25 · 2022: 10 · 2021: 10 · 2020: 10 · 2019: 11 · 2018: 8 · 2017: 14 · 2016: 6 · 2015: 8 · 2014: 9 · 2013: 11 · 2012: 11
- FWCI
- 10.19
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Journal of Psychiatry (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Social Psychology › Mental Health Treatment and Access
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- mental health prevention programs, statistical power, sample size determination, universal prevention, selective prevention, indicated prevention, high-risk groups, cumulative meta-analysis
- MeSH terms
- Adolescent, Mental Disorders, Child, Data Interpretation, Statistical, Female, Humans, Male, Preventive Health Services, Primary Prevention, Research Design, Risk Factors, Meta-Analysis as Topic, Cognitive Behavioral Therapy, Incidence, Randomized Controlled Trials as Topic, Patient Selection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review