Do larger studies find smaller effects? The example of studies for the prevention of conduct disorder
Abstract
Introduction There is some emerging evidence in medicine that larger clinical trials tend to be associated with smaller effect sizes. Much of the evidence-base currently informing practice in Child Psychiatry relies on relatively small trials. We therefore investigated the relationship between trial size and effect size in research within a key area of child mental health.Method A recent systematic review of 20 trials of prevention of conduct disorder was subjected to meta-regression, to examine the relationship between study size and effect size, and to explore hypothesised confounding variables.Results In this sample of studies, reported effect size was inversely related to sample size. This effect is not explained by year of publication, intervention type or quality of methodology in the study.Discussion Our finding is consistent with other reports in the literature. The origin of this effect is not yet clear. However if replicated it clearly has significant implications for the way trials in child mental health are interpreted.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Large trial sizes generally produce small effects.
“Large trial sizes generally produce small effects,31”
From Parent-mediated communication-focused treatment in children with autism (PACT): a randomised controlled trial (2010) · cites this paper as [31]
Cites
Olds, D. L. (1997) · JAMA · doi:10.1001/jama.1997.03550080047038 · Cited by 1,372 · see citing works
Significant Benefits: The High/Scope Perry Preschool Study Through Age 27
Schweinhart, L. J., Barnes, H., Weikart, D. P., & Ypsilanti, M. (1993) · Cited by 1,217 · see citing works
Preventing Adolescent Health-Risk Behaviors by Strengthening Protection During Childhood
Hawkins, J. D., Catalano, R. F., Kosterman, R., Abbott, R. D., & Hemphill, S. A. (1999) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/archpedi.153.3.226 · Cited by 1,034 · see citing works
Preventing conduct problems in Head Start children: Strengthening parenting competencies.
Webster-Stratton, C. H. (1998) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.66.5.715 · Cited by 852 · see citing works
Preventing conduct problems in Head Start children: Strengthening parenting competencies.
Webster-Stratton, C. H. (1998) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.66.5.715 · Cited by 624 · see citing works
Cartwright‐Hatton, S., Roberts, C., Chitsabesan, P., Fothergill, C., & Harrington, R. (2004) · British Journal of Clinical Psychology · doi:10.1348/0144665042388928 · Cited by 449 · see citing works
Self-citationKellam, S. G., Rebok, G. W., Ialongo, N. S., & Mayer, L. S. (1994) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.1994.tb01161.x · Cited by 407 · see citing works
Tremblay, R. E., Pagani, L. S., Mâsse, L. C., Vitaro, F., & al, E. (1995) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.63.4.560 · Cited by 407 · see citing works
Initial impact of the fast track prevention trial for conduct problems: II. Classroom effects.
(1999) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.67.5.648 · Cited by 402 · see citing works
A thirty-year follow-up of treatment effects.
McCord, J. (1978) · American Psychologist · doi:10.1037//0003-066x.33.3.284 · Cited by 396 · see citing works
A thirty-year follow-up of treatment effects.
McCord, J. (1978) · American Psychologist · doi:10.1037/0003-066x.33.3.284 · Cited by 391 · see citing works
MCCARTON, C. M. (1997) · JAMA · doi:10.1001/jama.277.2.126 · Cited by 366 · see citing works
Tremblay, R. E., Pagani-Kurtz, L., Mâsse, L. C., et al. (1995) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.63.4.560 · Cited by 303 · see citing works
Initial Impact of the Fast Track Prevention Trial for Conduct Problems: II. Classroom Effects
Bierman, K. L., Coie, J. D., Dodge, K. A., Greenberg, M. T., Lochman, J. E., McMahon, R. J., & Pinderhughes, E. E. (1999) · Journal of Consulting and Clinical Psychology · Cited by 290 · see citing works
Dolan, L. J., Kellam, S. G., Brown, C. H., Werthamer‐Larsson, L., Rebok, G. W., Mayer, L. S., Laudolff, J., Turkkan, J. S., Ford, C., & Wheeler, L. (1993) · Journal of Applied Developmental Psychology · doi:10.1016/0193-3973(93)90013-l · Cited by 280 · see citing works
Cognitive)ehavioral intervention with aggressive boys: Three-year follow-up and preventive effects.
Lochman, J. E. (1992) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.60.3.426 · Cited by 250 · see citing works
Results at Age 8 Years of Early Intervention for Low-Birth-Weight Premature Infants
MCCARTON, C. M. (1997) · JAMA · doi:10.1001/jama.1997.03540260040033 · Cited by 250 · see citing works
Effects of Family Support Intervention: A Ten-Year Follow-Up
Seitz, V., Rosenbaum, L. K., & Apfel, N. H. (1985) · Child Development · doi:10.2307/1129727 · Cited by 248 · see citing works
Lochman, J. E., Coie, J. D., Underwood, M. K., & Terry, R. (1993) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.61.6.1053 · Cited by 246 · see citing works
Cognitive-behavioral intervention with aggressive boys: Three-year follow-up and preventive effects.
Lochman, J. E. (1992) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.60.3.426 · Cited by 240 · see citing works
Structural family versus psychodynamic child therapy for problematic Hispanic boys.
Szapocznik, J., Rio, A. T., Murray, E. J., Cohen, R. E., Scopetta, M. A., Rivas-Vazquez, A. A., Hervis, O. E., Posada, V., & Kurtines, W. M. (1989) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.57.5.571 · Cited by 234 · see citing works
Structural family versus psychodynamic child therapy for problematic Hispanic boys.
Szapocznik, J., Rio, A. T., Murray, E. J., et al. (1989) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.57.5.571 · Cited by 172 · see citing works
Bierman, K. L., Miller, C. L., & Stabb, S. D. (1987) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.55.2.194 · Cited by 160 · see citing works
Lochman, J. E., Coie, J. D., Underwood, M. K., & Terry, R. (1993) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.61.6.1053 · Cited by 140 · see citing works
Bierman, K. L., Miller, C. L., & Stabb, S. D. (1987) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.55.2.194 · Cited by 133 · see citing works
Primary prevention of behavior problems in mexican‐american children
Johnson, D. L., & Walker, T. B. (1987) · American Journal of Community Psychology · doi:10.1007/bf00915208 · Cited by 112 · see citing works
(1992) · Cited by 109 · see citing works
Strayhorn, J. M., & Weidman, C. S. (1991) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-199101000-00021 · Cited by 106 · see citing works
Can intervention early prevent crime later? The abecedarian project compared with other programs
Clarke, S. H., & Campbell, F. A. (1998) · Early Childhood Research Quarterly · doi:10.1016/s0885-2006(99)80042-8 · Cited by 73 · see citing works
Schweinhart, L. J., & Weikart, D. P. (1997) · Cited by 65 · see citing works
Helping Children Adjust—a Tri-Ministry Study: II. Program Effects
Hundert, J., Boyle, M. H., Cunningham, C. E., Duku, E., Heale, J., McDonald, J., Offord, D. R., & Racine, Y. A. (1999) · Journal of Child Psychology and Psychiatry · doi:10.1111/1469-7610.00524 · Cited by 55 · see citing works
Statistics and the nature of depression
Dunn, G. D., Sham, P. C., & Hand, D. J. (1993) · Psychological Medicine · doi:10.1017/s0033291700026350 · Cited by 39 · see citing works
The impact of socioeconomic status on teenage mothers and children who received early intervention
Stone, W. L., Bendell, R. D., & Field, T. M. (1988) · Journal of Applied Developmental Psychology · doi:10.1016/0193-3973(88)90008-1 · Cited by 34 · see citing works
The prevention of conduct disorder: a review of successful and unsuccessful experiments
LeMarquand, D. G., Tremblay, R. E., & Vitaro, F. (2000) · Cambridge University Press eBooks · doi:10.1017/cbo9780511543852.017 · Cited by 14 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Green, J., Charman, T., McConachie, H., Aldred, C., Slonims, V., Howlin, P., Le Couteur, A., Leadbitter, K., Hudry, K., Byford, S., et al. (2010) · The Lancet · doi:10.1016/s0140-6736(10)60587-9 · Cited by 857 · see citing works
Self-citationMenting, A. T. A., Orobio de Castro, B., & Matthys, W. (2013) · Clinical Psychology Review · doi:10.1016/j.cpr.2013.07.006 · Cited by 417 · see citing works
Moreno, S. G., Sutton, A. J., Turner, E. H., Abrams, K. R., Cooper, N. J., Palmer, T. M., & Ades, A. E. (2009) · BMJ · doi:10.1136/bmj.b2981 · Cited by 184 · see citing works
A meta-analysis of cue reactivity in tobacco cigarette smokers
Betts, J. M., Dowd, A. N., Forney, M., Hetelekides, E., & Tiffany, S. T. (2021) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntaa147 · Cited by 72 · see citing works
- Authors
- McMahon, B., Holly, L., Harrington, R., Roberts, C., & Green, J.
- Affiliations
- University of Manchester (GB)
- Corresponding author
- Green, Jonathan
- Venue
- European Child & Adolescent Psychiatry
- Published
- 2008-04-21
- Volume
- 17
- Issue
- 7
- Pages
- 432-437
- ISSN
- 1018-8827 (ISSN-L)
- 1435-165X
- OpenAlex
- W2159468524
- Cited by
- 20
- References
- 36
- Citations by year
- 2026: 1 · 2020: 2 · 2018: 4 · 2015: 1 · 2014: 2 · 2013: 3 · 2012: 1
- FWCI
- 0.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- European Child & Adolescent Psychiatry (publishedVersion)
- PubMed (publishedVersion)
- Research Explorer (The University of Manchester) (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- sample size, effect size, child mental health, clinical trials, meta-regression, trial interpretation, confounding variables
- MeSH terms
- Data Interpretation, Statistical, Humans, Regression Analysis, Confounding Factors, Epidemiologic, Randomized Controlled Trials as Topic, Sample Size, Conduct Disorder
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
