The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Early stopping of randomized clinical trials for overt efficacy is problematic.
Abstract
Objective To illustrate controversial issues associated with stopping randomized controlled trials (RCTs) early for apparent benefit.Study design and setting The article presents our review of prior relevant work and our research group's reflections on early stopping.Results Compelling evidence suggests that trials stopped early for benefit systematically overestimate treatment effects, sometimes by a large amount. Unresolved controversies in trials stopped early for benefit include ethical and statistical problems in the interpretation of results.Conclusions The best strategy to minimize the problems associated with early stopping of RCTs for benefit is not to stop early. As an alternative, we suggest a threefold approach: a low P-value as the threshold for stopping at the time of interim analyses, not to look before a sufficiently large number of events has accrued and continuation of enrollment and follow-up for a further period.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The authors determined that a sufficient number of clusters had completed implementation of the planned intervention to allow analysis of the primary outcome but were unable to use the final 3 months of the stepped-wedge design (after the policy change) for the planned analysis.”
In the citing paper
“In a multicenter stepped-wedge cluster randomized pragmatic trial, regional government enacted a policy change partway through the rollout that mandated widespread use of a modified version of the experimental intervention as the new standard of care.”
In the citing paper
“The trialists identified this policy change as an extenuating circumstance leading to a premature and unanticipated modification to the implementation schedule at 9 months, before the planned completion at 12 months.”
In the citing paper
“The international survey yielded examples of trials that encountered extenuating circumstances including regulatory changes, new standards of care, and security threats (Table).”
In the citing paper
“The trial by Vaillancourt et al 20 on out-of-hospital cervical spine immobilization was stopped early when regional authorities enacted a new policy mandating a modified immobilization rule and adopting many aspects of the trial's experimental intervention as standard of care (Table).”
In the citing paper
“The investigators' mitigating strategies included omission of the final study period from the randomized clinical trial analysis, an extended observational data collection period for 3 months beyond the anticipated stop date, and a revised design and analysis plan to include 3 conditions: usual care, planned intervention, and an uncontrolled modified intervention condition.”
Citations
Cites
Peto, R., Pike, M. C., Armitage, P., Breslow, N. E., Cox, D. R., Howard, S. V., Mantel, N., McPherson, K., Peto, J., & Smith, P. G. (1977) · British Journal of Cancer · doi:10.1038/bjc.1977.1 · Cited by 8,285
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999) · The Lancet · doi:10.1016/s0140-6736(99)04149-5 · Cited by 4,826
A Multiple Testing Procedure for Clinical Trials
O'Brien, P. C., & Fleming, T. R. (1979) · Biometrics · doi:10.2307/2530245 · Cited by 3,311
Sleight, R. B., Sleight, P., Parish, S., Appleby, P. N., Collins, R., & Baigent, C. (1988) · PubMed · Cited by 2,868
急性心筋梗塞の疑いのある17187症例におけるストレプトキナーゼ静脈内投与,アスピリン経口投与,上記2つの併用または無処置の無作偽試験 ISIS-2
Dienstl, F., Backer, G. D., Cairns, J., Pedersen, F., Kala, R., Boissel, J., Schroeder, R., Horgan, J., & Tognoni, G. (1988) · The Lancet · Cited by 2,754
What Makes Clinical Research Ethical?
Emanuel, E. J. (2000) · JAMA · doi:10.1001/jama.283.20.2701 · Cited by 2,710
Auvert, B., Taljaard, D., Lagarde, E., Sobngwi-Tambekou, J., Sitta, R., & Puren, A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020298 · Cited by 2,451
Equipoise and the Ethics of Clinical Research
Freedman, B. (1987) · New England Journal of Medicine · doi:10.1056/nejm198707163170304 · Cited by 2,339
Discrete sequential boundaries for clinical trials
GORDON LAN, K. K., & DEMETS, D. L. (1983) · Biometrika · doi:10.1093/biomet/70.3.659 · Cited by 2,310
Discrete Sequential Boundaries for Clinical Trials
Lan, K. K. G., & DeMets, D. L. (1983) · Biometrika · doi:10.2307/2336502 · Cited by 2,310
Peto, R., Pike, M. C., Armitage, P., Breslow, N. E., Cox, D. R., Howard, S. V., Mantel, N., McPherson, K., Peto, J., & Smith, P. G. (1976) · British Journal of Cancer · doi:10.1038/bjc.1976.220 · Cited by 2,294
Male circumcision for HIV prevention in young men in Kisumu, Kenya: a randomised controlled trial.
Bailey, R. C., Moses, S., Parker, C. B., Agot, K., Maclean, I., Krieger, J. N., Williams, C. F., Campbell, R. T., & Ndinya-Achola, J. O. (2007) · The Lancet · doi:10.1016/s0140-6736(07)60312-2 · Cited by 2,292
Male circumcision for HIV prevention in men in Rakai, Uganda: a randomised trial.
Gray, R. H., Kigozi, G., Serwadda, D., Makumbi, F., Watya, S., Nalugoda, F., Kiwanuka, N., Moulton, L. H., Chaudhary, M. A., Chen, M. Z., et al. (2007) · The Lancet · doi:10.1016/s0140-6736(07)60313-4 · Cited by 2,226
Poldermans, D., Boersma, E. H., Bax, J. J., Thomson, I. R., van de Ven, L. L. M., Blankensteijn, J. D., Baars, H. F., Yo, T. I., Trocino, G., Vigna, C., et al. (1999) · New England Journal of Medicine · doi:10.1056/nejm199912093412402 · Cited by 1,480
Berry, D. A. (2006) · Nature Reviews Drug Discovery · doi:10.1038/nrd1927 · Cited by 728
Why do we need some large, simple randomized trials?
Yusuf, S., Collins, R., & Peto, R. (1984) · Statistics in Medicine · doi:10.1002/sim.4780030421 · Cited by 700
Repeated assessment of results in clinical trials of cancer treatment
Haybittle, J. L. (1971) · British Journal of Radiology · doi:10.1259/0007-1285-44-526-793 · Cited by 519
In Vitro Fertilization with Single Blastocyst-Stage versus Single Cleavage-Stage Embryos
Papanikolaou, E. G. E., Camus, M., Kolibianakis, E. M., Landuyt, L. V., Steirteghem, A. V., & Devroey, P. (2006) · New England Journal of Medicine · doi:10.1056/nejmoa053524 · Cited by 416
Multiplicity in randomised trials II: subgroup and interim analyses
Schulz, K. F., & Grimes, D. A. (2005) · The Lancet · doi:10.1016/s0140-6736(05)66516-6 · Cited by 357
A proposed charter for clinical trial data monitoring committees: helping them to do their job well
(2005) · The Lancet · doi:10.1016/s0140-6736(05)17965-3 · Cited by 324
Ethical issues in stopping randomized trials early because of apparent benefit.
Mueller, P. S., Montori, V. M., Bassler, D., Koenig, B. A., & Guyatt, G. H. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-146-12-200706190-00009 · Cited by 202
Evolution of treatment effects over time: Empirical insight from recursive cumulative metaanalyses
Ioannidis, J. P. A., & Lau, J. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.98.3.831 · Cited by 190
Effect sizes in cumulative meta-analyses of mental health randomized trials evolved over time
Trikalinos, T. A., Churchill, R., Ferri, M., Leucht, S., Tuunainen, A., Wahlbeck, K., & Ioannidis, J. P. A. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.02.018 · Cited by 177
Practical problems in interim analyses, with particular regard to estimation
Pocock, S., & Hughes, M. D. (1989) · Controlled Clinical Trials · doi:10.1016/0197-2456(89)90059-7 · Cited by 160
Peltoniemi, O. M., Kari, M. A., Heinonen, K., Saarela, T., Nikolajev, K., Andersson, S., Voutilainen, R., & Hallman, M. (2005) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2004.12.040 · Cited by 139
Stopping rules and estimation problems in clinical trials
Hughes, M. D., & Pocock, S. (1988) · Statistics in Medicine · doi:10.1002/sim.4780071204 · Cited by 128
Smith, M. R., Manola, J. B., Kaufman, D. S., Oh, W. K., Bubley, G. J., & Kantoff, P. W. (2006) · Journal of Clinical Oncology · doi:10.1200/jco.2005.03.7804 · Cited by 127
Trials stopped early: too good to be true?
Pocock, S., & White, I. R. (1999) · The Lancet · doi:10.1016/s0140-6736(98)00379-1 · Cited by 113
Data Safety and Monitoring Boards
Slutsky, A. S., & Lavery, J. V. (2004) · New England Journal of Medicine · doi:10.1056/nejmsb033476 · Cited by 101
Devereaux, P. J. (2004) · Canadian Medical Association Journal · doi:10.1503/cmaj.1031619 · Cited by 85
Current controversies in data monitoring for clinical trials
Pocock, S. (2006) · Clinical Trials · doi:10.1177/1740774506073467 · Cited by 82
Systematic reviewers neglect bias that results from trials stopped early for benefit
Bassler, D., González, I. F., Briel, M., Cook, D. J., Devereaux, P. J., Heels-Ansdell, D., Kirpalani, H. M., Meade, M. O., Montori, V. M., Rozenberg, A., et al. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.12.006 · Cited by 63
Sydes, M. R., Altman, D. G., Babikera, A. B., Parmar, M. K., Spiegelhalter, D. J., & DAMOCLES group (2004) · Clinical Trials · doi:10.1191/1740774504cn003oa · Cited by 49
Randomized Trials Stopped Early for Benefit: A Systematic Review
Montori, V. M., Devereaux, P. J., Adhikari, N. K. J., Burns, K. E. A., Eggert, C. H., Briel, M., Lacchetti, C., Leung, T. W., Darling, E., Bryant, D. M., et al. (2005) · JAMA · doi:10.1001/jama.294.17.2203 · Cited by 26
How Should Results from Completed Studies Influence Ongoing Clinical Trials?
Laupacis, A., Connolly, S. J., Gent, M., Roberts, R. S., Cairns, J. A., Joyner, C. D., & CAFA Study Group (1991) · Annals of Internal Medicine · doi:10.7326/0003-4819-115-10-818 · Cited by 21
Thistle, P., Spitzer, R. F., Glazier, R. H., Pilon, R., Arbess, G., Simor, A. E., Boyle, E., Chitsike, I., Chipato, T., Gottesman, M., et al. (2006) · Clinical Infectious Diseases · doi:10.1086/508869 · Cited by 21
The Promise of Single-Embryo Transfer
Schieve, L. A. (2006) · New England Journal of Medicine · doi:10.1056/nejme058321 · Cited by 17
Cyclooxygenase-2 Selective Inhibitors and Prostate Cancer: What Is the Clinical Benefit?
DuBois, R. N. (2006) · Journal of Clinical Oncology · doi:10.1200/jco.2006.05.9709 · Cited by 9
Cited by
Sorted by citations
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
Bassler, D. (2010) · JAMA · doi:10.1001/jama.2010.310 · Cited by 613
Orkin, A. M., Gill, P. J., Ghersi, D., Campbell, L., Sugarman, J., Emsley, R., Steg, P. G., Weijer, C., Simes, J., Rombey, T., et al. (2021) · JAMA · doi:10.1001/jama.2021.9941 · Cited by 299
Imberger, G. L., Thorlund, K., Gluud, C. N., & Wetterslev, J. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-011890 · Cited by 278
Transarterial (chemo)embolisation for unresectable hepatocellular carcinoma
Oliveri, R. S., Wetterslev, J., & Gluud, C. N. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004787.pub2 · Cited by 266
Metadata
- Authors
- Bassler, D., Montori, V. M., Briel, M., Glasziou, P., & Guyatt, G.
- Affiliations
- University Children's Hospital Tübingen (DE)
- University of Basel (CH)
- University of Oxford (GB)
- McMaster University (CA)
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2008-03
- Volume
- 61
- Issue
- 3
- Pages
- 241-246
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2042803707
- Cited by
- 130
- References
- 38
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 7 · 2023: 3 · 2022: 5 · 2021: 3 · 2020: 6 · 2019: 6 · 2018: 9 · 2017: 4 · 2016: 7 · 2015: 9 · 2014: 9 · 2013: 8 · 2012: 9
- FWCI
- 1.92
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- DigitalGeorgetown (Georgetown University Library) (submittedVersion)
- Funders
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
- Medical Research Council
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- early stopping, randomized controlled trials, interim analysis, overestimation of treatment effects, statistical power, p-value threshold, trial ethics
- MeSH terms
- Decision Making, Humans, Motivation, Bias, Randomized Controlled Trials as Topic, Treatment Outcome, Ethics, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
