Implications of clinical trial design on sample size requirements
Open accessAbstract
The primary goal in designing a randomized controlled clinical trial (RCT) is to minimize bias in the estimate of treatment effect. Randomized group assignment, double-blinded assessments, and control or comparison groups reduce the risk of bias. The design must also provide sufficient statistical power to detect a clinically meaningful treatment effect and maintain a nominal level of type I error. An attempt to integrate neurocognitive science into an RCT poses additional challenges. Two particularly relevant aspects of such a design often receive insufficient attention in an RCT. Multiple outcomes inflate type I error, and an unreliable assessment process introduces bias and reduces statistical power. Here we describe how both unreliability and multiple outcomes can increase the study costs and duration and reduce the feasibility of the study. The objective of this article is to consider strategies that overcome the problems of unreliability and multiplicity.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The low power of completed trials increases the likelihood of a type II statistical error of concluding that a drug with true efficacy did not have beneficial effect.
“Our analysis shows that only one in 17 completed trials with available results had sufficient power to detect a medium (d = 0.5) effect size, thus increasing the likelihood of a type II statistical error of concluding that a drug with true efficacy did not have beneficial effect.54,55”
From Clinical trials of potential cognitive-enhancing drugs in schizophrenia: what have we learned so far? (2013) · cites this paper as [54]
Large samples imply multi-site trials, which present investigators and sponsors with specific challenges.
“Such large samples imply multi-site trials, which present investigators and sponsors with specific challenges (for review, see ref.54).”
From Clinical trials of potential cognitive-enhancing drugs in schizophrenia: what have we learned so far? (2013) · cites this paper as [54]
Only one in 17 completed trials with available results had sufficient power to detect a medium (d = 0.5) effect size.
“Our analysis shows that only one in 17 completed trials with available results had sufficient power to detect a medium (d = 0.5) effect size, thus increasing the likelihood of a type II statistical error of concluding that a drug with true efficacy did not have beneficial effect.54,55”
From Clinical trials of potential cognitive-enhancing drugs in schizophrenia: what have we learned so far? (2013) · cites this paper as [54]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · see citing works
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771 · see citing works
Effectiveness of Antipsychotic Drugs in Patients with Chronic Schizophrenia
Lieberman, J. A., Stroup, T. S., McEvoy, J. P., Swartz, M. S., Rosenheck, R. A., Perkins, D. O., Keefe, R. S. E., Davis, S. M., Davis, C. E., Lebowitz, B. D., et al. (2005) · New England Journal of Medicine · doi:10.1056/nejmoa051688 · Cited by 5,892 · see citing works
A sharper Bonferroni procedure for multiple tests of significance
HOCHBERG, Y. (1988) · Biometrika · doi:10.1093/biomet/75.4.800 · Cited by 4,829 · see citing works
The MATRICS Consensus Cognitive Battery, part 1: test selection, reliability, and validity
Nuechterlein, K. H., Green, M. F., Kern, R. S., Baade, L. E., Barch, D. M., Cohen, J. D., Essock, S., Fenton, W. S., Frese, F. J., Gold, J. M., et al. (2008) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2007.07010042 · Cited by 2,342 · see citing works
Size of Treatment Effects and Their Importance to Clinical Research and Practice
KRAEMER, H. C., & Kupfer, D. J. (2005) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.09.014 · Cited by 850 · see citing works
. Caution regarding the use of pilot studies to guide power calculations for study proposals.
Kraemer, H. C., Mintz, J., Noda, A., Tinklenberg, J., & Yesavage, J. A. (2006) · Archives of General Psychiatry · doi:10.1001/archpsyc.63.5.484 · Cited by 702 · see citing works
The Analysis of Multiple Endpoints in Clinical Trials
Pocock, S. J., Geller, N. L., & Tsiatis, A. A. (1987) · Biometrics · doi:10.2307/2531989 · Cited by 524 · see citing works
Attrition in randomized controlled clinical trials: methodological issues in psychopharmacology
Leon, A. C., Mallinckrodt, C. H., Chuang-Stein, C., Archibald, D. G., Archer, G. E., & Chartier, K. (2006) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.10.020 · Cited by 303 · see citing works
Sixteen <i>S</i> ‐squared over <i>D</i> ‐squared: A relation for crude sample size estimates
Lehr, R. (1992) · Statistics in Medicine · doi:10.1002/sim.4780110811 · Cited by 253 · see citing works
Ury, H. K. (1976) · Technometrics · doi:10.2307/1267921 · Cited by 145 · see citing works
Barch, D. M., Yodkovik, N., Sypher-Locke, H., & Hanewinkel, M. J. (2008) · Journal of Abnormal Psychology · doi:10.1037/a0013944 · Cited by 113 · see citing works
More Reliable Outcome Measures Can Reduce Sample Size Requirements
Leon, A. C. (1995) · Archives of General Psychiatry · doi:10.1001/archpsyc.1995.03950220077014 · Cited by 71 · see citing works
Approximate multinormal probabilities applied to correlated multiple endpoints in clinical trials
Susan, J. (1991) · Statistics in Medicine · doi:10.1002/sim.4780100712 · Cited by 69 · see citing works
Multiplicity-Adjusted Sample Size Requirements
Leon, A. C. (2004) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v65n1111 · Cited by 44 · see citing works
A Comparison of Multiplicity Adjustment Strategies for Correlated Binary Endpoints
Leon, A. C., & Heo, M. (2005) · Journal of Biopharmaceutical Statistics · doi:10.1081/bip-200067922 · Cited by 29 · see citing works
Statistical power of multiplicity adjustment strategies for correlated binary endpoints
Leon, A. C., Heo, M., Teres, J. J., & Morikawa, T. (2007) · Statistics in Medicine · doi:10.1002/sim.2795 · Cited by 14 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Fleiss JL. The Design & Analysis of Clinical Experiments. New York, NY: John Wiley and Sons; 1986.
Cited by
Sorted by citations
Periodontitis and Cognitive Decline in Alzheimer’s Disease
Ide, M., Harris, M., Stevens, A., Sussams, R., Hopkins, V., Culliford, D., Fuller, J., Ibbett, P., Raybould, R., Thomas, R., et al. (2016) · PLOS ONE · doi:10.1371/journal.pone.0151081 · Cited by 461 · see citing works
Assessing cognitive function in clinical trials of schizophrenia
Barnett, J. H., Robbins, T. W., Leeson, V. C., Sahakian, B. J., Joyce, E. M., & Blackwell, A. D. (2010) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2010.01.012 · Cited by 173 · see citing works
Mazahery, H., Stonehouse, W., Delshad, M., Kruger, M. C., Conlon, C. A., Beck, K. L., & von Hurst, P. R. (2017) · Nutrients · doi:10.3390/nu9020155 · Cited by 146 · see citing works
Keefe, R. S. E., Buchanan, R. W., Marder, S. R., Schooler, N. R., Dugar, A., Zivkov, M., & Stewart, M. (2013) · Schizophrenia Bulletin · doi:10.1093/schbul/sbr153 · Cited by 138 · see citing works
Blinding in Clinical Trials: Seeing the Big Picture
Monaghan, T. F., Agudelo, C. W., Rahman, S. N., Wein, A. J., Lazar, J., Everaert, K., & Dmochowski, R. R. (2021) · Medicina · doi:10.3390/medicina57070647 · Cited by 143 · see citing works
Barch, D. M., Carter, C. S., Arnsten, A. F. T., Buchanan, R. W., Cohen, J. D., Geyer, M. A., Green, M. F., Krystal, J. H., Nuechterlein, K. H., Robbins, T. W., et al. (2008) · Schizophrenia Bulletin · doi:10.1093/schbul/sbn163 · Cited by 133 · see citing works
Treatment resistant schizophrenia and response to antipsychotics: A review
Suzuki, T., Remington, G. J., Mulsant, B. H., Rajji, T. K., Uchida, H., Graff‐Guerrero, A., & Mamo, D. C. (2011) · Schizophrenia Research · doi:10.1016/j.schres.2011.09.016 · Cited by 115 · see citing works
Test-retest reliability and validity of a frustration paradigm and irritability measures
Tseng, W., Moroney, E., Machlin, L., Roberson‐Nay, R., Hettema, J. M., Carney, D. M., Stoddard, J., Towbin, K. A., Pine, D. S., Leibenluft, E., et al. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.01.024 · Cited by 64 · see citing works
Schneider, K. L., Panza, E. A., Handschin, B., Ma, Y., Busch, A. M., Waring, M. E., Appelhans, B. M., Whited, M. C., Keeney, J., Kern, D., et al. (2015) · Behavior Therapy · doi:10.1016/j.beth.2015.10.005 · Cited by 61 · see citing works
Correll, C. U., Kishimoto, T., & Kane, J. M. (2011) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2011.13.2/ccorrell · Cited by 48 · see citing works
Kuo, L., Huang, H., Huang, H., Liang, J., Chiu, Y., Chen, S., Kwok, Y., Hsu, W., & Shyu, Y. L. (2012) · International Journal of Geriatric Psychiatry · doi:10.1002/gps.3853 · Cited by 44 · see citing works
Individualized Repetitive Transcranial Magnetic Stimulation Treatment in Chronic Tinnitus?
Kreuzer, P. M., Poeppl, T. B., Rupprecht, R., Vielsmeier, V., Lehner, A., Langguth, B., & Schecklmann, M. (2017) · Frontiers in Neurology · doi:10.3389/fneur.2017.00126 · Cited by 42 · see citing works
Samara, M. T., Cao, H., Helfer, B., Davis, J. M., & Leucht, S. (2014) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2014.03.012 · Cited by 31 · see citing works
Enhancing clinical trial design of interventions for posttraumatic stress disorder
Leon, A. C., & Davis, L. L. (2009) · Journal of Traumatic Stress · doi:10.1002/jts.20466 · Cited by 26 · see citing works
The effects of meditation on individuals facing loneliness: a scoping review
Saini, G. K., Haseeb, S. B., Taghi-Zada, Z., & Ng, J. Y. (2021) · BMC Psychology · doi:10.1186/s40359-021-00585-8 · Cited by 28 · see citing works
Casserly, B. P., Read, R. N., & Levy, M. M. (2011) · Critical Care Clinics · doi:10.1016/j.ccc.2010.12.011 · Cited by 25 · see citing works
Lenze, E. J., Dixon, D., Nowotny, P., Lotrich, F. E., Doré, P. M., Pollock, B. G., Hinrichs, A. L., & Butters, M. A. (2013) · International Journal of Neuropsychopharmacology · doi:10.1017/s1461145712000351 · Cited by 23 · see citing works
Perkins, L., Felstead, C., Stott, J. C. H., & Spector, A. E. (2021) · Journal of Clinical Nursing · doi:10.1111/jocn.16175 · Cited by 21 · see citing works
Inaccuracy in Clinical Trials: Effects and Methods to Control Inaccuracy
Kobak, K. A. (2010) · Current Alzheimer Research · doi:10.2174/156720510793499057 · Cited by 18 · see citing works
Kumar, S., Mulsant, B. H., Tsoutsoulas, C., Ghazala, Z., Voineskos, A. N., Bowie, C. R., & Rajji, T. K. (2016) · International Journal of Geriatric Psychiatry · doi:10.1002/gps.4547 · Cited by 14 · see citing works
The interactive computer interview for mania
Reilly-Harrington, N. A., DeBonis, D., Leon, A. C., Sylvia, L. G., Perlis, R. H., Lewis, D., & Sachs, G. S. (2010) · Bipolar Disorders · doi:10.1111/j.1399-5618.2010.00844.x · Cited by 12 · see citing works
Acute Pain Evaluation with Algoplus®Scale in Cambodian Patients
Pickering, M., Bunna, P., Rat, P., Madeline, G., Lebost, C., Serrié, A., & Pereira, B. (2013) · Pain Medicine · doi:10.1111/pme.12174 · Cited by 10 · see citing works
Kalembo, F. W., & Kendall, G. E. (2021) · Child & Family Social Work · doi:10.1111/cfs.12897 · Cited by 11 · see citing works
Arandjelović, O. (2015) · PLoS ONE · doi:10.1371/journal.pone.0131524 · Cited by 8 · see citing works
Group psychoeducation for forensic long-term patients with schizophrenia
Aho‐Mustonen, K. (2011) · UEF eRepo (University of Eastern Finland) · Cited by 8 · see citing works
A randomized pilot of eHealth mindful movement and breathing to improve gynecologic surgery outcomes
Sohl, S. J., Tooze, J. A., Levine, B. J., Zeidan, F. Z., Wheeler, A. E., Kelly, M., Shalowitz, D. I., & Danhauer, S. C. (2022) · Journal of Psychosocial Oncology · doi:10.1080/07347332.2022.2101908 · Cited by 6 · see citing works
Examining the Advantages and Disadvantages of Pilot Studies: Monte-Carlo Simulations
Nakazawa, M. (2012) · UNM’s Digital Repository (University of New Mexico) · Cited by 2 · see citing works
Rachmawati, Y. L., Rahmah, A. Y., Azzahra, N. F., Ananda, S. N., Elfath, B. A., & Yusof, Z. Y. M. (2025) · World Journal of Dentistry · doi:10.5005/jp-journals-10015-2622 · Cited by 1 · see citing works
Special Issues of Research Methodology in Bipolar Disorder Clinical Treatment Trials
Post, R. M. (2010) · Wiley eBooks · doi:10.1002/9780470749180.ch8 · Cited by 1 · see citing works
Assessment and treatment of cognitive impairment and related features
Harvey, P. D. (2013) · Cambridge University Press eBooks · doi:10.1017/cbo9781139003872.014
Sequential sample size re-estimation in clinical trials with multiple co-primary endpoints
Ntambwe, L. I. (2026) · Warwick Research Archive Portal (University of Warwick) · doi:10.82444/warw.33685981
The most effective surgical treatment option for congenital hydrocephalus: a review
Warren, D. (2011) · British Journal of Neuroscience Nursing · doi:10.12968/bjnn.2011.7.5.639
Vaughan, S. M. (2014) · Griffith Research Online · doi:10.25904/1912/3215
Monaghan, T. F., Wein, A. J., & Dmochowski, R. R. (2024) · Elsevier eBooks · doi:10.1016/b978-0-323-90186-4.00087-0
Wang, Y., Chaiard, J., Lirtmunlikaporn, S., & Suwankruhasn, N. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-100966
- Authors
- Leon, A. C.
- Affiliations
- Cornell University (US)
- Corresponding author
- Leon, A. C.
- ORCID
- Leon, A. C.
- Venue
- Schizophrenia Bulletin
- Publisher
- Oxford University Press (OUP)
- Published
- 2007-11-21
- Volume
- 34
- Issue
- 4
- Pages
- 664-669
- ISSN
- 0586-7614 (ISSN-L)
- 1745-1701
- PMC
- PMC2632464
- OpenAlex
- W2170636614
- Cited by
- 35
- References
- 17
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 1 · 2022: 1 · 2021: 4 · 2017: 3 · 2016: 2 · 2015: 2 · 2014: 2 · 2013: 2 · 2012: 3
- FWCI
- 1.87
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Schizophrenia Bulletin (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health
- National Institute of Mental Health (MH068638, MH 060447, P30MH068638, R01MH060447)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- clinical trial design, sample size
- MeSH terms
- Cognition Disorders, Double-Blind Method, Humans, Neuropsychological Tests, Pilot Projects, Psychometrics, Reaction Time, Research Design, Schizophrenia, Statistics as Topic, Reproducibility of Results, Randomized Controlled Trials as Topic, Treatment Outcome, Outcome Assessment, Health Care, Sample Size, Ethics, Research, Control Groups
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review