The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
The Continuing Unethical Conduct of Underpowered Clinical Trials
Abstract
Despite long-standing critiques of the conduct of underpowered clinical trials, the practice not only remains widespread, but also has garnered increasing support. Patients and healthy volunteers continue to participate in research that may be of limited clinical value, and authors recently have offered 2 related arguments to support the validity and value of underpowered clinical trials: that meta-analysis may "save" small studies by providing a means to combine the results with those of other similar studies to enable estimates of an intervention's efficacy, and that although small studies may not provide a good basis for testing hypotheses, they may provide valuable estimates of treatment effects using confidence intervals. In this article, we examine these arguments in light of the distinctive moral issues associated with the conduct of underpowered trials, the disclosures that are owed to potential participants in underpowered trials so they may make autonomous enrollment decisions, and the circumstances in which the prospects for future meta-analyses may justify individually underpowered trials. We conclude that underpowered trials are ethical in only 2 situations: small trials of interventions for rare diseases in which investigators document explicit plans for including their results with those of similar trials in a prospective meta-analysis, and early-phase trials in the development of drugs or devices, provided they are adequately powered for defined purposes other than randomized treatment comparisons. In both cases, investigators must inform prospective subjects that their participation may only indirectly contribute to future health care benefits.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
What Makes Clinical Research Ethical?
Emanuel, E. J. (2000) · JAMA · doi:10.1001/jama.283.20.2701 · Cited by 2,710
Freiman, J. A., Chalmers, T. C., Smith, H., & Kuebler, R. R. (1978) · New England Journal of Medicine · doi:10.1056/nejm197809282991304 · Cited by 1,628
meta-analysis bias in location and selection of studies
Egger, M., & Smith, G. D. (1998) · BMJ · doi:10.1136/bmj.316.7124.61 · Cited by 1,374
Defining the clinically important difference in pain outcome measures
Farrar, J. T., Portenoy, R. K., Berlin, J. A., Kinman, J. L., & Strom, B. L. (2000) · Pain · doi:10.1016/s0304-3959(00)00339-0 · Cited by 1,071
The scandal of poor medical research.
Altman, D. G. (1994) · BMJ · doi:10.1136/bmj.308.6924.283 · Cited by 897
Publication bias—a problem in interpreting medical data
Begg, C. B., & Berlin, J. A. (1988) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2982993 · Cited by 814
A Comparison of Methotrexate with Placebo for the Maintenance of Remission in Crohn's Disease
Feagan, B. G., Fedorak, R. N., Irvine, E. J., Wild, G., Sutherland, L., Steinhart, A. H., Greenberg, G. R., Koval, J., Wong, C. J., Hopkins, M., et al. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006013422202 · Cited by 727
Statistical Problems in the Reporting of Clinical Trials
Pocock, S. J., Hughes, M. D., & Lee, R. J. (1987) · New England Journal of Medicine · doi:10.1056/nejm198708133170706 · Cited by 662
Goodman, S. N., & Berlin, J. A. (1994) · Annals of Internal Medicine · doi:10.7326/0003-4819-121-3-199408010-00008 · Cited by 654
Cited by
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883
A tutorial on pilot studies: the what, why and how
Thabane, L., Ma, J., Chu, R., Cheng, J., Ismaila, A., Rios, L. P., Robson, R., Thabane, M., Giangregorio, L., & Goldsmith, C. H. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-1 · Cited by 3,384
Sample size determination and power analysis using the G*Power software
Kang, H. (2021) · Journal of Educational Evaluation for Health Professions · doi:10.3352/jeehp.2021.18.17 · Cited by 2,572
“tors believe that underpowered research is unethical, except for small trials of interventions for rare diseases and early phase trials in the development of drugs or devices [5].”
BackgroundEllis, P. D. (2010) · doi:10.1017/cbo9780511761676 · Cited by 2,461
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2012) · International Journal of Surgery · doi:10.1016/j.ijsu.2011.10.001 · Cited by 2,345
Whitehead, A. L., Julious, S. A., Cooper, C. L., & Campbell, M. J. (2016) · Statistical Methods in Medical Research · doi:10.1177/0962280215588241 · Cited by 1,936
“Recruiting too few participants in a main trial means that the probability of finding a clinically relevant difference statistically significant is low and as a consequence, the chance of inconclusive results is high.(2,3) Conversely, if the sample size is too large, resources may be wasted, more patients than necessa…”
BackgroundLakens, D. (2022) · Collabra Psychology · doi:10.1525/collabra.33267 · Cited by 1,962
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.004 · Cited by 1,858
Increasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597
What is a pilot or feasibility study? A review of current practice and editorial policy
Arain, M., Campbell, M. J., Cooper, C. L., & Lancaster, G. A. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-67 · Cited by 1,483
“It is generally accepted that small, underpowered clinical trials are unethical [4].”
BackgroundFogel, D. B. (2018) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2018.08.001 · Cited by 1,035
Bayes factor design analysis: Planning for compelling evidence
Schönbrodt, F. D., & Wagenmakers, E.-J. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1230-y · Cited by 729
The impact of study size on meta-analyses: examination of underpowered studies in Cochrane reviews
Turner, R. M., Bird, S. M., & Higgins, J. P. T. (2013) · PLoS ONE · doi:10.1371/journal.pone.0059202 · Cited by 672
Maxwell, S. E. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.2.147 · Cited by 654
Sample size calculations in randomised trials: mandatory and mystical
Schulz, K. F., & Grimes, D. A. (2005) · The Lancet · doi:10.1016/s0140-6736(05)61034-3 · Cited by 549
In, J. (2017) · Korean Journal of Anesthesiology · doi:10.4097/kjae.2017.70.6.601 · Cited by 497
Effect of Communications Training on Medical Student Performance
Yedidia, M. J. (2003) · JAMA · doi:10.1001/jama.290.9.1157 · Cited by 481
Biau, D. J., Kernéis, S., & Porcher, R. (2008) · Clinical Orthopaedics and Related Research · doi:10.1007/s11999-008-0346-9 · Cited by 428
Carlisle, B., Kimmelman, J., Ramsay, T., & MacKinnon, N. (2015) · Clinical Trials · doi:10.1177/1740774514558307 · Cited by 367
Treatment of periodontal disease for glycaemic control in people with diabetes
Simpson, T. C., Needleman, I., Wild, S. H., Moles, D. R., & Mills, E. J. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004714.pub2 · Cited by 283
Mills (2010) · Clinical Epidemiology · doi:10.2147/clep.s9242 · Cited by 281
Reporting of sample size calculation in randomised controlled trials: review
Charles, P., Giraudeau, B., Dechartres, A., Baron, G., & Ravaud, P. (2009) · BMJ · doi:10.1136/bmj.b1732 · Cited by 274
Treatment of Pressure Ulcers: A Systematic Review
Reddy, M., Gill, S. S., Kalkar, S. R., Wu, W., Anderson, P. J., & Rochon, P. A. (2008) · JAMA · doi:10.1001/jama.2008.778 · Cited by 266
Hughes-Morley, A., Young, B., Waheed, W., Small, N., & Bower, P. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2014.10.005 · Cited by 132
A guide to prospective meta-analysis
Seidler, A. L., Hunter, K. E., Cheyne, S., Ghersi, D., Berlin, J. A., & Askie, L. (2019) · BMJ · doi:10.1136/bmj.l5342 · Cited by 125
Synthesizing Evidence: Shifting the Focus From Individual Studies to the Body of Evidence
Murad, M. H., & Montori, V. M. (2013) · JAMA · doi:10.1001/jama.2013.5616 · Cited by 104
A Guide for Calculating Study-Level Statistical Power for Meta-Analyses
Quintana, D. S. (2023) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459221147260 · Cited by 95
Haag, G., Diener, H.-C., May, A., Meyer, C., Morck, H., Straube, A., Wessely, P., & Evers, S. (2011) · The Journal of Headache and Pain · doi:10.1007/s10194-010-0266-4 · Cited by 78
“insufficient sample size can thus lose their ethical justification [74].”
BackgroundClark, T., Berger, U., & Mansmann, U. (2013) · BMJ · doi:10.1136/bmj.f1135 · Cited by 68
A brief summary of pilot and feasibility studies: Exploring terminology, aims, and methods
Donald, G. (2018) · European Journal of Integrative Medicine · doi:10.1016/j.eujim.2018.10.017 · Cited by 53
The conundrum of depression clinical trials: one size does not fit all
Khan, A., Mar, K. F., & Brown, W. A. (2018) · International Clinical Psychopharmacology · doi:10.1097/yic.0000000000000229 · Cited by 30
“Perhaps we were not heeding the risks of low statistical power well enough, as has been the case in many scientific fields (Cohen, 1962; Freiman et al., 1978; Sedlmeier and Gigerenzer, 1989; Halpern et al., 2002; Maxwell, 2004; Button et al., 2013).”
BackgroundEvidence appraisal: a scoping review, conceptual framework, and research agenda
Goldstein, A., Venker, E., & Weng, C. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx050 · Cited by 30
Pilot trials in physical activity journals: a review of reporting and editorial policy
Horne, E., Lancaster, G. A., Matson, R., Cooper, A., Ness, A., & Leary, S. (2018) · Pilot and Feasibility Studies · doi:10.1186/s40814-018-0317-1 · Cited by 23
Schuster, R., Schreyer, M. L., Kaiser, T., Berger, T., Klein, J. P., Moritz, S., Laireiter, A.-R., & Trutschnig, W. (2020) · Internet Interventions · doi:10.1016/j.invent.2020.100313 · Cited by 21
Doig, G. S. (2008) · JAMA · doi:10.1001/jama.2008.826 · Cited by 1
Metadata
- Authors
- Halpern, S. D.
- Venue
- JAMA
- Publisher
- American Medical Association (AMA)
- Published
- 2002-07-17
- Volume
- 288
- Issue
- 3
- Pages
- 358
- ISSN
- 0002-9955 (ISSN-L)
- 0098-7484
- OpenAlex
- W2017874571
- Cited by
- 547
- References
- 39
- FWCI
- 23.33
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.