Sample size determinations in original research protocols for randomised clinical trials submitted to UK research ethics committees: review
Open accessAbstract
OBJECTIVES: To assess the completeness of reporting of sample size determinations in unpublished research protocols and to develop guidance for research ethics committees and for statisticians advising these committees. DESIGN: Review of original research protocols. STUDY SELECTION: Unpublished research protocols for phase IIb, III, and IV randomised clinical trials of investigational medicinal products submitted to research ethics committees in the United Kingdom during 1 January to 31 December 2009. MAIN OUTCOME MEASURES: Completeness of reporting of the sample size determination, including the justification of design assumptions, and disagreement between reported and recalculated sample size. RESULTS: 446 study protocols were reviewed. Of these, 190 (43%) justified the treatment effect and 213 (48%) justified the population variability or survival experience. Only 55 (12%) discussed the clinical importance of the treatment effect sought. Few protocols provided a reasoned explanation as to why the design assumptions were plausible for the planned study. Sensitivity analyses investigating how the sample size changed under different design assumptions were lacking; six (1%) protocols included a re-estimation of the sample size in the study design. Overall, 188 (42%) protocols reported all of the information to accurately recalculate the sample size; the assumed withdrawal or dropout rate was not given in 177 (40%) studies. Only 134 of the 446 (30%) sample size calculations could be accurately reproduced. Study size tended to be over-estimated rather than under-estimated. Studies with non-commercial sponsors justified the design assumptions used in the calculation more often than studies with commercial sponsors but less often reported all the components needed to reproduce the sample size calculation. Sample sizes for studies with non-commercial sponsors were less often reproduced. CONCLUSIONS: Most research protocols did not contain sufficient information to allow the sample size to be reproduced or the plausibility of the design assumptions to be assessed. Greater transparency in the reporting of the determination of the sample size and more focus on study design during the ethical review process would allow deficiencies to be resolved early, before the trial begins. Guidance for research ethics committees and statisticians advising these committees is needed.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Among randomised trial protocols that describe a sample size calculation, studies often do not state all components necessary to understand and reproduce the sample size (including the derivation of the target difference and the source of estimated values).
“Among randomised trial protocols that describe a sample size calculation, studies often do not state all components necessary to understand and reproduce the sample size (including the derivation of the target difference and the source of estimated values).200 297”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [297]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373 · see citing works
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835 · see citing works
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755 · see citing works
What Makes Clinical Research Ethical?
Emanuel, E. J. (2000) · JAMA · doi:10.1001/jama.283.20.2701 · Cited by 2,710 · see citing works
Wood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538 · see citing works
Sample Size Tables for Clinical Studies
Machin, D., Campbell, M. J., Tan, S. B., & Tan, S. H. (2008) · Wiley eBooks · doi:10.1002/9781444300710 · Cited by 1,336 · see citing works
McDonald, A. M., Knight, R. C., Campbell, M. K., Entwistle, V. A., Grant, A. M., Cook, J. A., Elbourne, D. R., Francis, D., Garcia, J., Roberts, I., et al. (2006) · Trials · doi:10.1186/1745-6215-7-9 · Cited by 946 · see citing works
Sample Size Calculations in Clinical Research
Chow, S., Wang, H., & Shao, J. (2003) · CRC Press eBooks · doi:10.1201/9780203911341 · Cited by 813 · see citing works
Sample sizes for clinical trials with normal data.
Julious, S. A. (2004) · Statistics in Medicine · doi:10.1002/sim.1783 · Cited by 605 · see citing works
Section XXIV. Partition Chromatographs
Haider, S. I., & Ashtok, A. (2009) · Biotechnology · doi:10.1201/9781420084498-31 · Cited by 590 · see citing works
Statistics and ethics in medical research: III How large a sample?
Altman, D. G. (1980) · BMJ · doi:10.1136/bmj.281.6251.1336 · Cited by 581 · see citing works
Statistical Principles for Clinical Trials
Kay, R. (1998) · Journal of International Medical Research · doi:10.1177/030006059802600201 · Cited by 573 · see citing works
Sample size calculations in randomised trials: mandatory and mystical
Schulz, K. F., & Grimes, D. A. (2005) · The Lancet · doi:10.1016/s0140-6736(05)61034-3 · Cited by 549 · see citing works
The Continuing Unethical Conduct of Underpowered Clinical Trials
Halpern, S. D. (2002) · JAMA · doi:10.1001/jama.288.3.358 · Cited by 548 · see citing works
Campbell, M. J., Julious, S. A., & Altman, D. G. (1995) · BMJ · doi:10.1136/bmj.311.7013.1145 · Cited by 476 · see citing works
Statistical Issues in Drug Development
Senn, S. (2007) · doi:10.1002/9780470723586 · Cited by 448 · see citing works
Acupuncture and Knee Osteoarthritis
Scharf, H., Mansmann, U., Streitberger, K., Witte, S., Krämer, J., Maier, C., Trampisch, H. J., & Victor, N. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-145-1-200607040-00005 · Cited by 419 · see citing works
Self-citationStatistical Issues in Drug Development
Senn, S. J. (2021) · Wiley eBooks · doi:10.1002/9781119238614 · Cited by 384 · see citing works
Reporting of sample size calculation in randomised controlled trials: review
Charles, P., Giraudeau, B., Dechartres, A., Baron, G., & Ravaud, P. (2009) · BMJ · doi:10.1136/bmj.b1732 · Cited by 274 · see citing works
Current sample size conventions: Flaws, harms, and alternatives
Bacchetti, P. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-17 · Cited by 233 · see citing works
Sample Sizes for Clinical Trials
Julious, S. A. (2009) · Chapman & Hall/CRC eBooks · doi:10.1201/9781584887409 · Cited by 227 · see citing works
Kieser, M., & Wassmer, G. (1996) · Biometrical Journal · doi:10.1002/bimj.4710380806 · Cited by 213 · see citing works
Chan, A.-W., Hrobjartsson, A., Jorgensen, K. J., Gotzsche, P. C., & Altman, D. G. (2008) · BMJ · doi:10.1136/bmj.a2299 · Cited by 202 · see citing works
Sample Size Calculations in Clinical Research
Bacchetti, P., & Leung, J. M. (2002) · Anesthesiology · doi:10.1097/00000542-200210000-00050 · Cited by 201 · see citing works
Savulescu, J., Chalmers, I., & Blunt, J. (1996) · BMJ · doi:10.1136/bmj.313.7069.1390 · Cited by 189 · see citing works
Sample size calculations: should the emperor's clothes be off the peg or made to measure?
Norman, G. R., Monteiro, S., & Salama, S. (2012) · BMJ · doi:10.1136/bmj.e5278 · Cited by 187 · see citing works
Bacchetti, P. (2005) · American Journal of Epidemiology · doi:10.1093/aje/kwi014 · Cited by 162 · see citing works
Simple procedures for blinded sample size adjustment that do not affect the type I error rate
Kieser, M., & Friede, T. (2003) · Statistics in Medicine · doi:10.1002/sim.1585 · Cited by 126 · see citing works
Optimism bias leads to inconclusive results—an empirical study
Djulbegović, B., Kumar, A., Magazin, A., Schroen, A. T., Soares, H. P., Hozo, I., Clarke, M. J., Sargent, D. J., & Schell, M. J. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.09.007 · Cited by 62 · see citing works
Julious, S. A., Campbell, M. J., & Altman, D. G. (1999) · Journal of Biopharmaceutical Statistics · doi:10.1081/bip-100101174 · Cited by 60 · see citing works
Tutorial in biostatistics - Sample sizes for clinical trials with Normal data
Julious, S. A. (2004) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · Cited by 55 · see citing works
Waiving confidentiality for the greater good
Chan, A., Upshur, R. E. G., Singh, J. A., Ghersi, D., Chapuis, F. R., & Altman, D. G. (2006) · BMJ · doi:10.1136/bmj.332.7549.1086 · Cited by 46 · see citing works
Acupuncture and Knee Osteoarthritis: A Three-Armed Randomized Trial
Berney, S. M. (2007) · Yearbook of Medicine · doi:10.1016/s0084-3873(08)70018-x · Cited by 38 · see citing works
Statistical Review by Research Ethics Committees
Williamson, P. R., Hutton, J. L., Bliss, J. M., Blunt, J., Campbell, M. J., & Nicholson, R. (2000) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/1467-985x.00152 · Cited by 34 · see citing works
Larivière, L., & Moréra, S. (2002) · Journal of Molecular Biology · doi:10.1016/s0022-2836(02)01091-4 · Cited by 31 · see citing works
Sample sizes for trials involving multiple correlated must‐win comparisons
Julious, S. A., & McIntyre, N. E. (2012) · Pharmaceutical Statistics · doi:10.1002/pst.515 · Cited by 26 · see citing works
Designing clinical trials with uncertain estimates of variability
Julious, S. A. (2004) · Pharmaceutical Statistics · doi:10.1002/pst.139 · Cited by 23 · see citing works
Cardiotocography v Doppler auscultation. All unbiased comparative studies should be published.
Chalmers, I. (2002) · PubMed · Cited by 4 · see citing works
Grant, S. (2002) · PubMed · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [3]Victor N. Prüfung der wissenschaftlichen Qualität und biometriespezifischer Anforderungen durch die Ethikkommissionen? Medizinrecht 1999; 9: 408-12.
Cited by
Sorted by citations
How to increase value and reduce waste when research priorities are set
Chalmers, I., Bracken, M. B., Djulbegovic, B., Garattini, S., Grant, J., Gülmezoglu, A. M., Howells, D. W., Ioannidis, J. P. A., & Oliver, S. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62229-1 · Cited by 1,436 · see citing works
Teare, M. D., Dimairo, M., Shephard, N., Hayman, A., Whitehead, A., & Walters, S. J. (2014) · Trials · doi:10.1186/1745-6215-15-264 · Cited by 773 · see citing works
SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials
Hróbjartsson, A., Boutron, I., Hopewell, S., Moher, D., Schulz, K. F., Collins, G. S., Tunn, R., Aggarwal, R., Berkwits, M., Berlin, J. A., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081660 · Cited by 171 · see citing works
More multiarm randomised trials of superiority are needed
Parmar, M. K. B., Carpenter, J., & Sydes, M. R. (2014) · The Lancet · doi:10.1016/s0140-6736(14)61122-3 · Cited by 127 · see citing works
Cook, J. A., Hislop, J., Altman, D. G., Fayers, P., Briggs, A. H., Ramsay, C. R., Norrie, J. D., Harvey, I. M., Buckley, B., Fergusson, D., et al. (2015) · Trials · doi:10.1186/s13063-014-0526-8 · Cited by 66 · see citing works
- Authors
- Clark, T., Berger, U., & Mansmann, U.
- Affiliations
- Institut für Medizinische Informationsverarbeitung, Biometrie und Epidemiologie (DE)
- Corresponding author
- Mansmann, U.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2013-03-21
- Volume
- 346
- Issue
- mar21 1
- Pages
- f1135-f1135
- ISSN
- 1756-1833 (ISSN-L)
- PMC
- PMC3604970
- OpenAlex
- W2045759042
- Cited by
- 68
- References
- 39
- Citations by year
- 2026: 1 · 2025: 5 · 2024: 5 · 2023: 3 · 2022: 4 · 2021: 6 · 2020: 4 · 2019: 7 · 2018: 6 · 2017: 3 · 2016: 6 · 2015: 10 · 2014: 7 · 2013: 1
- FWCI
- 1.57
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc
- Locations
- BMJ (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Open access LMU (Ludwid Maxmilian's Universitat Munchen) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- sample size determination, randomized controlled trials, research ethics committees, reporting completeness, design assumptions, sensitivity analysis, reproducibility
- MeSH terms
- United Kingdom, Humans, Research Design, Randomized Controlled Trials as Topic, Sample Size, Ethical Review, Ethics Committees, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
