The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Early stopping of randomized clinical trials for overt efficacy is problematic.
Abstract
Objective To illustrate controversial issues associated with stopping randomized controlled trials (RCTs) early for apparent benefit.Study design and setting The article presents our review of prior relevant work and our research group's reflections on early stopping.Results Compelling evidence suggests that trials stopped early for benefit systematically overestimate treatment effects, sometimes by a large amount. Unresolved controversies in trials stopped early for benefit include ethical and statistical problems in the interpretation of results.Conclusions The best strategy to minimize the problems associated with early stopping of RCTs for benefit is not to stop early. As an alternative, we suggest a threefold approach: a low P-value as the threshold for stopping at the time of interim analyses, not to look before a sufficiently large number of events has accrued and continuation of enrollment and follow-up for a further period.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The authors determined that a sufficient number of clusters had completed implementation of the planned intervention to allow analysis of the primary outcome but were unable to use the final 3 months of the stepped-wedge design (after the policy change) for the planned analysis.”
In the citing paper
“In a multicenter stepped-wedge cluster randomized pragmatic trial, regional government enacted a policy change partway through the rollout that mandated widespread use of a modified version of the experimental intervention as the new standard of care.”
In the citing paper
“The trialists identified this policy change as an extenuating circumstance leading to a premature and unanticipated modification to the implementation schedule at 9 months, before the planned completion at 12 months.”
In the citing paper
“The international survey yielded examples of trials that encountered extenuating circumstances including regulatory changes, new standards of care, and security threats (Table).”
In the citing paper
“The trial by Vaillancourt et al 20 on out-of-hospital cervical spine immobilization was stopped early when regional authorities enacted a new policy mandating a modified immobilization rule and adopting many aspects of the trial's experimental intervention as standard of care (Table).”
In the citing paper
“The investigators' mitigating strategies included omission of the final study period from the randomized clinical trial analysis, an extended observational data collection period for 3 months beyond the anticipated stop date, and a revised design and analysis plan to include 3 conditions: usual care, planned intervention, and an uncontrolled modified intervention condition.”
Citations
Cites
Peto, R., Pike, M. C., Armitage, P., Breslow, N. E., Cox, D. R., Howard, S. V., Mantel, N., McPherson, K., Peto, J., & Smith, P. G. (1977) · British Journal of Cancer · doi:10.1038/bjc.1977.1 · Cited by 8,285
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999) · The Lancet · doi:10.1016/s0140-6736(99)04149-5 · Cited by 4,826
A Multiple Testing Procedure for Clinical Trials
O'Brien, P. C., & Fleming, T. R. (1979) · Biometrics · doi:10.2307/2530245 · Cited by 3,311
Sleight, R. B., Sleight, P., Parish, S., Appleby, P. N., Collins, R., & Baigent, C. (1988) · PubMed · Cited by 2,868
急性心筋梗塞の疑いのある17187症例におけるストレプトキナーゼ静脈内投与,アスピリン経口投与,上記2つの併用または無処置の無作偽試験 ISIS-2
Dienstl, F., Backer, G. D., Cairns, J., Pedersen, F., Kala, R., Boissel, J., Schroeder, R., Horgan, J., & Tognoni, G. (1988) · The Lancet · Cited by 2,754
What Makes Clinical Research Ethical?
Emanuel, E. J. (2000) · JAMA · doi:10.1001/jama.283.20.2701 · Cited by 2,710
Auvert, B., Taljaard, D., Lagarde, E., Sobngwi-Tambekou, J., Sitta, R., & Puren, A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020298 · Cited by 2,451
Equipoise and the Ethics of Clinical Research
Freedman, B. (1987) · New England Journal of Medicine · doi:10.1056/nejm198707163170304 · Cited by 2,339
Discrete sequential boundaries for clinical trials
GORDON LAN, K. K., & DEMETS, D. L. (1983) · Biometrika · doi:10.1093/biomet/70.3.659 · Cited by 2,310
Cited by
Sorted by citations
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
Bassler, D. (2010) · JAMA · doi:10.1001/jama.2010.310 · Cited by 613
Orkin, A. M., Gill, P. J., Ghersi, D., Campbell, L., Sugarman, J., Emsley, R., Steg, P. G., Weijer, C., Simes, J., Rombey, T., et al. (2021) · JAMA · doi:10.1001/jama.2021.9941 · Cited by 299
Imberger, G. L., Thorlund, K., Gluud, C. N., & Wetterslev, J. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-011890 · Cited by 278
Transarterial (chemo)embolisation for unresectable hepatocellular carcinoma
Oliveri, R. S., Wetterslev, J., & Gluud, C. N. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004787.pub2 · Cited by 266
Dewint, P., Hansen, B. E., Verhey, E., Oldenburg, B., Hommes, D. W., Pierik, M. J., Ponsioen, C. Y., van Dullemen, H. M., Rüssel, M. G., Bodegraven, A. A. V., et al. (2013) · Gut · doi:10.1136/gutjnl-2013-304488 · Cited by 263
Evidence at a glance: error matrix approach for overviewing available evidence
Keus, F., Wetterslev, J., Gluud, C. N., & van Laarhoven, C. J. H. M. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-90 · Cited by 218
“Therefore, the p-values of intervention effect estimates certainly are not suitable for comparison of the risk of random error between different studies [32,34-37].”
Assessment of risk of bias in randomized clinical trials in surgery
Gurusamy, K. S., Gluud, C. N., Nikolova, D., & Davidson, B. R. (2009) · British journal of surgery · doi:10.1002/bjs.6558 · Cited by 216
Sydes, M. R., Parmar, M., James, N. D., Clarke, N. W., Dearnaley, D. P., Mason, M. D., Morgan, R. C., Sanders, K., & Royston, P. J. (2009) · Trials · doi:10.1186/1745-6215-10-39 · Cited by 150
“Many instances of early stopping for benefit appear to occur on a random high [27,28] and the credibility and interpretability of a trial can be adversely affected: caution should be applied in this respect.”
BackgroundBayesian statistics for clinical research
Goligher, E. C., Heath, A., & Harhay, M. O. (2024) · The Lancet · doi:10.1016/s0140-6736(24)01295-9 · Cited by 141
Majumdar, S. R., Kjellstrand, C. M., Tymchak, W. J., Hervas‐Malo, M., Taylor, D. A., & Teo, K. K. (2009) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2009.03.024 · Cited by 123
Chamberlain, P., Bradley, A., Arumugam, B., Hammond, D., Mcnally, J. J., Logan, N. S., JONES, D. A., Ngo, C. S., Peixoto-de-Matos, S. C., Hunt, C., et al. (2022) · Optometry and Vision Science · doi:10.1097/opx.0000000000001873 · Cited by 120
for The Closed Or Open after Laparotomy (COOL) after Source Control for Severe Complicated Intra-Abdominal Sepsis Investigators, Kirkpatrick, A. W., Coccolini, F., Ansaloni, L., Roberts, D. J., Tolonen, M., McKee, J. L., Leppäniemi, A. K., Faris, P., Doig, C. J., et al. (2018) · World Journal of Emergency Surgery · doi:10.1186/s13017-018-0183-4 · Cited by 114
Systematic Reviews of Anesthesiologic Interventions Reported as Statistically Significant
Imberger, G. L., Gluud, C. N., Boylan, J. F., & Wetterslev, J. (2015) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000000892 · Cited by 102
Atorvastatin versus placebo in patients with covid-19 in intensive care: randomized controlled trial
INSPIRATION-S Investigators (2022) · BMJ · doi:10.1136/bmj-2021-068407 · Cited by 86
“30 The data and safety monitoring board had independent access to the study data and conducted prespecified interim safety analyses at 25%, 50%, and 75% of recruitment.”
When to keep it simple – adaptive designs are not always useful
Wason, J. M. S., Brocklehurst, P., & Yap, C. (2019) · BMC Medicine · doi:10.1186/s12916-019-1391-9 · Cited by 83
Bikdeli, B., Talasaz, A. H., Rashidi, F., Sharif-Kashani, B., Farrokhpour, M., Bakhshandeh, H., Sezavar, S. H., Dabbagh, A., Beigmohammadi, M. T., Payandemehr, P., et al. (2020) · Thrombosis Research · doi:10.1016/j.thromres.2020.09.027 · Cited by 84
“To provide the most informative results, without risking the possibility of an overinflated estimate of the effect size, the steering committee decided not to set pre-specified criteria for early termination for efficacy [41].”
BackgroundMultimorbidity: a challenge for evidence-based medicine
Campbell-Scherer, D. L. (2010) · BMJ Evidence-Based Medicine · doi:10.1136/ebm1154 · Cited by 86
Why marketplace experimentation is harder than it seems
Blake, T., & Coey, D. (2014) · ACM Conference on Economics and Computation (EC) · doi:10.1145/2600057.2602837 · Cited by 76
Stopping clinical trials early for benefit: impact on estimation
Freidlin, B., & Korn, E. L. (2009) · Clinical Trials · doi:10.1177/1740774509102310 · Cited by 76
Naci, H., & Ioannidis, J. P. A. (2014) · The Annual Review of Pharmacology and Toxicology · doi:10.1146/annurev-pharmtox-010814-124614 · Cited by 70
Expanding the Orbit of Primary Prevention — Moving beyond JUPITER
Hlatky, M. A. (2008) · New England Journal of Medicine · doi:10.1056/nejme0808320 · Cited by 69
Completion and Publication Rates of Randomized Controlled Trials in Surgery
Rosenthal, R., Kasenda, B., Dell‐Kuster, S., von Elm, E. B., You, J. J., Blümle, A., Tomonaga, Y., Saccilotto, R. T., Amstutz, A., Bengough, T., et al. (2014) · Annals of Surgery · doi:10.1097/sla.0000000000000810 · Cited by 67
Duodenal Infusion of Feces for Recurrent Clostridium difficile
Schooneveld, T. C. V., Gross, A., & Kalil, A. C. (2013) · New England Journal of Medicine · doi:10.1056/nejmc1303919 · Cited by 72
Rivaroxaban Prophylaxis in Noncirrhotic Portal Vein Thrombosis
Plessier, A., Goria, O., Cervoni, J. P., Ollivier‐Hourmand, I., Bureau, C., Poujol‐Robert, A., Minello, A., Houssel‐Debry, P., Rautou, P., Payancé, A., et al. (2022) · NEJM Evidence · doi:10.1056/evidoa2200104 · Cited by 60
Stevely, A. K., Dimairo, M., Todd, S., Julious, S. A., Nicholl, J. P., Hind, D., & Cooper, C. L. (2015) · PLoS ONE · doi:10.1371/journal.pone.0141104 · Cited by 60
“Although much attention has been paid to group sequential RCTs that are stopping early for benefit [62,63,66,68], the consequences could be similar when trials are stopped early for futility, since the evidence can be used to withdraw intervention(s) already in the care pathway.”
Photo induced antibacterial activity of CeO2/GO against wound pathogens
Sharma, G., Prema, D., Venkataprasanna, K. S., Prakash, J., Sahabuddin, S., & Venkatasubbu, G. D. (2020) · Arabian Journal of Chemistry · doi:10.1016/j.arabjc.2020.09.004 · Cited by 56
Hartling, L., Wittmeier, K. D. M., Caldwell, P. H. Y., van der Lee, H., Klassen, T. P., Craig, J. C., Offringa, M., & for the StaR Child Health Group (2012) · PEDIATRICS · doi:10.1542/peds.2012-0055c · Cited by 53
Korn, E. L., Freidlin, B., & Mooney, M. (2009) · Journal of Clinical Oncology · doi:10.1200/jco.2008.19.5339 · Cited by 49
Is acute respiratory distress syndrome an iatrogenic disease?
Villar, J., & Slutsky, A. S. (2010) · Critical Care · doi:10.1186/cc8842 · Cited by 46
“Early stopping tends to overestimate treatment eff ects [12,13]; this is particularly true for studies with low event rates, as was the case in this study, in which only 12 patients in total developed ALI.”
ResultHumbert, P. G., Faivre, B., Véran, Y., Debure, C., Truchetet, F., Bécherel, P., Plantin, P., Kerihuel, J., Eming, S. A., Dissemond, J., et al. (2014) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.12400 · Cited by 47
Fernandes, R. M., van der Lee, J. H., & Offringa, M. (2009) · BMC Pediatrics · doi:10.1186/1471-2431-9-77 · Cited by 43
“This implies knowing which, if any, statistical monitoring methods and stopping rules or guidelines were used, for which outcome (whether primary or secondary), and at which stage of the trial[11,13] It may be relevant in some cases to check if the results were corrected for multiplicity[147] The ongoing controversies…”
InfluentialNarayan, V. M., Gandhi, S., Chrouser, K. L., Evaniew, N., & Dahm, P. (2018) · British Journal of Urology · doi:10.1111/bju.14210 · Cited by 42
Stopping guidelines for an effectiveness trial: what should the protocol specify?
Tyson, J. E., Pedroza, C., Wallace, D., D’Angio, C. T., Bell, E. F., & Das, A. (2016) · Trials · doi:10.1186/s13063-016-1367-4 · Cited by 41
“Despite long-standing problems in decisions to stop clinical trials [1–6], stopping guidelines are often vague or unspecified in the trial protocol.”
BackgroundMechanical Thrombectomy vs Bridging Therapy for Anterior Circulation Large Vessel Occlusion Stroke
Trifan, G., Biller, J., & Testai, F. D. (2022) · Neurology · doi:10.1212/wnl.0000000000200029 · Cited by 39
Magnitude of effects in clinical trials published in high-impact general medical journals
Siontis, K., Εvangelou, Ε., & Ioannidis, J. P. A. (2011) · International Journal of Epidemiology · doi:10.1093/ije/dyr095 · Cited by 40
Maitland, K., Kiguli, S., Opoka, R. O., Olupot‐Olupot, P., Engoru, C., Njuguna, P., Bandika, V., Mpoya, A., Bush, A., Williams, T. N., et al. (2018) · Wellcome Open Research · doi:10.12688/wellcomeopenres.12747.2 · Cited by 38
Anand, V., Scales, D. C., Parshuram, C. S., & Kavanagh, B. P. (2014) · Intensive Care Medicine · doi:10.1007/s00134-014-3250-7 · Cited by 36
“the magnitude of the apparent treatment benefit) is frequently overestimated, as has been described in cases of premature trial termination for apparent benefit [17].”
BackgroundYounus, J., Verma, S., Franek, J. A., Coakley, N., & the Sarcoma Disease Site Group of Cancer Care Ontario’s Program in Evidence-Based Care (2010) · Current Oncology · doi:10.3747/co.v17i4.560 · Cited by 36
“Early termination of clinical trials due to benefit often overestimates overall treatment effect as such trials tend to be on a ―random high‖ with subsequent followup data from the same or similar trials showing ―regression to the truth‖ ( 11-14 ).”
BackgroundEarly Stopping of Clinical Trials: Charting the Ethical Terrain
Malmqvist, E., Juth, N., Lynöe, N., & Helgesson, G. (2011) · Kennedy Institute of Ethics journal · doi:10.1353/ken.2011.0002 · Cited by 35
“Many believe that early analyses of small amounts of data are prone to exaggerating the difference between the treatments (Pocock 1992; Montori et al. 2005; Mueller et al. 2007; Bassler et al. 2008).”
BackgroundMaitland, K., Kiguli, S., Opoka, R. O., Olupot‐Olupot, P., Engoru, C., Njuguna, P., Bandika, V., Mpoya, A., Bush, A., Williams, T. N., et al. (2017) · Wellcome Open Research · doi:10.12688/wellcomeopenres.12747.1 · Cited by 33
Dimairo, M., Julious, S. A., Todd, S., Nicholl, J. P., & Boote, J. D. (2015) · Trials · doi:10.1186/s13063-015-1119-x · Cited by 31
“For instance, suboptimal reporting of appropriate statistical methods to obtain unbiased trial results (point estimates, CIs and P-values) following an AD may influence consumers of research findings to view them with suspicion [23 – 25].”
Meta‐analysis of clinical trials with early stopping: an investigation of potential bias
Schou, I. M., & Marschner, I. C. (2013) · Statistics in Medicine · doi:10.1002/sim.5893 · Cited by 32
Pedroza, C., Tyson, J. E., Das, A., Laptook, A. R., Bell, E. F., & Shankaran, S. (2016) · Trials · doi:10.1186/s13063-016-1480-4 · Cited by 32
Stegert, M., Kasenda, B., von Elm, E., You, J. J., Blümle, A., Tomonaga, Y., Saccilotto, R., Amstutz, A., Bengough, T., Briel, M., et al. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.05.023 · Cited by 30
van den Bogert, C. A., Souverein, P. C., Brekelmans, C. T. M., Janssen, S. W. J., Koëter, G. H., Leufkens, H. G. M., & Bouter, L. M. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.05.001 · Cited by 31
The Impact of Clinical Trials Conducted by Research Networks in Pediatric Critical Care
Choong, K. L. M., Duffett, M., Cook, D. J., & Randolph, A. G. (2016) · Pediatric Critical Care Medicine · doi:10.1097/pcc.0000000000000835 · Cited by 28
“Effects may be overestimated when trials are stopped early for benefit, and some recommend against early stopping for this reason (31).”
BackgroundThe unrestricted global effort to complete the COOL trial
Kirkpatrick, A., Coccolini, F., Tolonen, M., Minor, S. F., Catena, F., Gois, E., Doig, C., Hill, M. D., Ansaloni, L., Chiarugi, M., et al. (2023) · World Journal of Emergency Surgery · doi:10.1186/s13017-023-00500-z · Cited by 17
Stopping Randomized Trials Early for Benefit and Estimation of Treatment Effects
Bassler, D., Briel, M., Montori, V. M., Glasziou, P., Zhou, Q. X., Heels‐Ansdell, D., Walter, S. D., & Guyatt, G. H. (2012) · Cited by 24
Standard 3: Data Monitoring Committees
Ellenberg, S., Fernandes, R., Saloojee, H., Bassler, D., Askie, L., Vandermeer, B., Offringa, M., van der Tweel, I., Altman, D., & van der Lee, J. H. (2012) · Pediatrics · doi:10.1542/peds.2012-0055f · Cited by 23
Paul, M., Ram, R., Kugler, E., Farbman, L., Peck, A., Leibovici, L., Lahav, M., Yeshurun, M., Shpilberg, O., Herscovici, C., et al. (2013) · American Journal of Hematology · doi:10.1002/ajh.23622 · Cited by 22
“Trials stopped early for benefit or harm have been shown to systematically exaggerate effects [17].”
BackgroundBalme, K., Zar, H. J., Swift, D. K., & Mann, M. D. (2015) · Clinical Toxicology · doi:10.3109/15563650.2015.1059943 · Cited by 20
Hoff, W. V., & Offringa, M. (2014) · Archives of Disease in Childhood · doi:10.1136/archdischild-2012-303094 · Cited by 17
Data monitoring committees, interim analysis and early termination in paediatric trials
Fernandes, R., van der Lee, J. H., & Offringa, M. (2011) · Acta paediatrica · doi:10.1111/j.1651-2227.2011.02282.x · Cited by 20
“Many trials were terminated for efficacy or futility, which may place them at a higher risk of having biased results (13,15).”
BackgroundInfluentialOptimal Therapy in Hypertensive Subjects with Diabetes Mellitus
Reboldi, G., Gentile, G., Angeli, F., & Verdecchia, P. (2011) · Current Atherosclerosis Reports · doi:10.1007/s11883-011-0160-9 · Cited by 17
“Therefore, the excellent results of the benazepril/amlodipine combination may potentially represent an overestimation of reality, as has been proven to be the case with early truncation of trials [79].”
Kennedy, A. P., Phillips, K. A., Epstein, D., Reamer, D., Schmittner, J., & Preston, K. (2012) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2012.10.025 · Cited by 17
“Trials stopped early can produce false-positive findings (Bassler et al., 2008), and we cannot rule out the possibility that we would have had different results if we had completed our trial.”
ResultEdwards, S. (2010) · Research Ethics · doi:10.1177/174701611000600207 · Cited by 15
“However, this practice has been criticised for leading to false positive results and some trials have even be repeated at a later date to redress the issue of a small sample size [8].”
BackgroundPublishing interim results of randomised clinical trials in peer-reviewed journals
Counsell, N., Biri, D., Fraczek, J., & Hackshaw, A. K. (2016) · Clinical Trials · doi:10.1177/1740774516664689 · Cited by 15
“Disease area Oncology 38 (60) Neurology 9 (14) Cardiology 6 (10) Osteoporosis 3 (5) Gastroenterology 2 (3) Gynaecology 2 (3) Infectious diseases 2 (3) Respiratory 1 (2) Type of sponsor Commercial 17 (27) Non-commercial 46 (73) Commercial funding support received No 33 (52) Yes 30 (48) Subjects blinded No 39 (62) Yes 2…”
BackgroundProtecting Clinical Trial Participants and Protecting Data Integrity: Are We Meeting the Challenges?
Ellenberg, S. S. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001234 · Cited by 15
“A recent variation on this argument is that early termination of trials should be avoided because it leads to artificially elevated estimates of treatment effect; trials that stop early, of course, show higher estimates of effect than trials of the same regimens that do not stop early [33,34].”
BackgroundJiang, Y., Zhao, W., & Durkalski‐Mauldin, V. L. (2019) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543406.2019.1607368 · Cited by 15
“It is known that observed treatment effect sizes are inflated when trials stop early for overwhelming efficacy (Bassler et al. 2008; Freidlin et al. 2009).”
BackgroundResultBubble CPAP for pneumonia: perils of stopping trials early
Shann, F. A., & Lange, T. (2015) · The Lancet · doi:10.1016/s0140-6736(15)60691-2 · Cited by 14
Vaccarino, V., Bremner, J. D., & Kelley, M. E. (2009) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.109.850404 · Cited by 14
Rizzo, M., Berneis, K. K., Spinas, G. A., Rini, G. B., & Kapur, N. K. (2009) · International Journal of Clinical Practice · doi:10.1111/j.1742-1241.2008.01979.x · Cited by 14
A new anti-cancer drug in the market: Good news for investors or for patients?
Apolone, G., Tafuri, G., Trotta, F., & Garattini, S. K. (2008) · European Journal of Cancer · doi:10.1016/j.ejca.2008.06.021 · Cited by 14
Hartling, L., Wittmeier, K. D. M., Caldwell, P. H. Y., van der Lee, J. H., Klassen, T. P., Craig, J. C., & Offringa, M. (2011) · Clinical Pharmacology & Therapeutics · doi:10.1038/clpt.2011.212 · Cited by 13
Voglibose for prevention of type 2 diabetes mellitus
Scheen, A. J. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60575-4 · Cited by 10
The JUPITER Trial: Myth or Reality?
Morrissey, R. P., Diamond, G. A., & Kaul, S. (2011) · Current Atherosclerosis Reports · doi:10.1007/s11883-011-0197-9 · Cited by 11
“Truncated trials are well known to be associated with overestimation of treatment benefits compared with non-truncated trials due to chance (“random variation”), which drift back towards the null (“regress to the mean”) with continued follow-up [12, 20, 32].”
BackgroundSurveying risk subjects: Public health surveys as instruments of biomedicalization
Norton, A. T. (2013) · BioSocieties · doi:10.1057/biosoc.2013.20 · Cited by 9
Interim monitoring of efficacy data is important and appropriate
Sydes, M. R., & Parmar, M. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.08.002 · Cited by 9
Sargent, D. J. (2009) · Journal of Clinical Oncology · doi:10.1200/jco.2008.20.8611 · Cited by 9
“As expected, the trials had a wide variety of primary end points; in 70% of the trials, accrual was complete at the time of closing, which has been recommended by some as an important criterion before trial reporting should be considered.(6) In only one case were data released before at least 25% of the protocol speci…”
ResultBesonderheiten von Onkologischen Studien im Rahmen der frühen Nutzenbewertung von Arzneimitteln
Lange, S. (2015) · Zeitschrift für Evidenz Fortbildung und Qualität im Gesundheitswesen · doi:10.1016/j.zefq.2015.08.003 · Cited by 8
Early stopping rules in oncology: Considerations for clinicians
Mukherjee, S. D., Goffin, J. R., Taylor, V. H., Anderson, K. K., & Pond, G. R. (2011) · European Journal of Cancer · doi:10.1016/j.ejca.2011.05.019 · Cited by 8
Shimura, M., Maruo, K., & Gosho, M. (2018) · Pharmaceutical Statistics · doi:10.1002/pst.1859 · Cited by 7
Das, S., & Xaviar, S. (2019) · Epilepsy Research · doi:10.1016/j.eplepsyres.2019.106258 · Cited by 7
Muir, O. S., Shinozuka, K., Beutler, B. D., Arenas, A., Cherian, K. N., Evans, V. D., Fasano, C., & Tabaac, B. J. (2024) · American Journal of Therapeutics · doi:10.1097/mjt.0000000000001720 · Cited by 7
“27 Psychedelic therapies require significant postmarketing surveillance to understand the complete adverse effect profile and variability between specific population cohorts.”
BackgroundHow to Distinguish between Statistically Significant Results and Clinically Relevant Results
Bennett, D. (2016) · Monographs in clinical neuroscience/Frontiers of neurology and neuroscience/Monographs in neural sciences · doi:10.1159/000445411 · Cited by 6
Culbertson, M. J., McCole, D., & McNamara, P. E. (2014) · Journal of Development Effectiveness · doi:10.1080/19439342.2014.919339 · Cited by 6
“However, this strategy should be used with caution (Montori et al. 2005; Bassler et al. 2008).”
Trial Termination and Drug Misclassification in Sequential Adaptive Clinical Trials
Rojas-Cordova, A. C., & Hosseinichimeh, N. (2018) · Service Science · doi:10.1287/serv.2018.0217 · Cited by 6
Evidence-Based Medicine - A Paradigm Ready To Be Challenged?
Schulte, M.-C. (2020) · J.B. Metzler eBooks · doi:10.1007/978-3-476-05703-7 · Cited by 6
Reducing overestimation of the treatment effect by interim analysis when designing clinical trials
Shimura, M. (2018) · Journal of Clinical Pharmacy and Therapeutics · doi:10.1111/jcpt.12777 · Cited by 5
The new US and European guidelines in hypertension: A multi-dimensional analysis
Fernandes, M., & Rikkert, M. G. M. O. (2019) · Contemporary Clinical Trials · doi:10.1016/j.cct.2019.04.008 · Cited by 5
Published intimate partner violence studies often differ from their trial registration records
Madden, K., Tai, K., Ali, Z., Schneider, P., Singh, M., Ghert, M. A., & Bhandari, M. (2017) · Women & Health · doi:10.1080/03630242.2017.1421287 · Cited by 4
Optimal Patient Enrollment in Sequential Adaptive Clinical Trials with Binary ResponsePreprint
Rojas-Cordova, A. C., & Bish, E. K. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3234590 · Cited by 5
Lawson, D. O., Mellor, K., Eddy, S., Lee, C., Kim, K. H., Kim, K., Mbuagbaw, L. C. E., & Thabane, L. (2022) · PubMed · doi:10.1097/phm.0000000000001797 · Cited by 4
Vasopressors and the search for the optimal trial design
Maharaj, R. (2011) · Contemporary Clinical Trials · doi:10.1016/j.cct.2011.07.010 · Cited by 4
“Recent reviews have shown that RCTs stopped early for benefit may be associated with greater effect size than those RCT not stopped early [38–41].”
BackgroundOxygen therapy in acute resuscitation
Frei, D. R., & Young, P. J. (2018) · Current Opinion in Critical Care · doi:10.1097/mcc.0000000000000549 · Cited by 4
Monitoring in clinical trials: benefit or bias?
Nardini, C. (2013) · Metamedicine · doi:10.1007/s11017-013-9264-2 · Cited by 4
“[9], the effect magnitude is considered ‘‘implausible’’ with respect to normal expectations in the medical community about treatment effectiveness.”
BackgroundResultWhen should an interim analysis be unblinded to the data monitoring committee?
Knottnerus, J. A., & Spigt, M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.07.003 · Cited by 4
Marschner, I. C., Askie, L. M., & Schou, I. M. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1394 · Cited by 4
“Some researchers have argued that there is the potential for overestimation of treatment effects in RCTs that stopped early for benefit, which are often referred to as truncated studies.(5) Of particular concern has been the impact that truncation may have on evidence synthesis when conducting metaanalyses.”
BackgroundTherapeutic hypothermia and the ‘equal air‐time’ solution for controversial randomised trials
Wilkinson, D. J. C. (2010) · Journal of Paediatrics and Child Health · doi:10.1111/j.1440-1754.2010.01864.x · Cited by 4
Schwarzer, R., Schnell‐Inderst, P., Grabein, K., Göhler, A., Stollenwerk, B., Grandi, N. C., Klauß, V., Wasem, J., & Siebert, U. (2009) · Zeitschrift für Evidenz Fortbildung und Qualität im Gesundheitswesen · doi:10.1016/j.zefq.2009.05.020 · Cited by 4
Rojas-Cordova, A. C., Bish, E. K., & Hosseinichimeh, N. (2019) · Women in engineering and science · doi:10.1007/978-3-030-11866-2_14 · Cited by 3
Early termination of clinical trials in paediatrics
Camara, C. (2020) · British Journal of Nursing · doi:10.12968/bjon.2020.29.14.814 · Cited by 3
Promoting Women’s Participation in Maternity Care in Lebanon
Kabakian‐Khasholian, T., Shayboub, R., Kanaan, M. N., Mahfoud, Z. R., & Kak, F. E. (2015) · International Journal of Childbirth · doi:10.1891/2156-5287.5.3.166 · Cited by 3
Rosuvastatin, C-reactive protein, LDL cholesterol, and the JUPITER trial
Bloom, J. M. (2009) · The Lancet · doi:10.1016/s0140-6736(09)61232-0 · Cited by 3
Rosuvastatin, C-reactive protein, LDL cholesterol, and the JUPITER trial
Rosenstein, R. S., & Parra, D. (2009) · The Lancet · doi:10.1016/s0140-6736(09)61230-7 · Cited by 3
Fleming, S., Mwandigha, L., & Fanshawe, T. R. (2024) · Diagnostic and Prognostic Research · doi:10.1186/s41512-024-00174-4 · Cited by 2
“A study may also be terminated early if there is sufficient interim evidence that it is very likely to yield a clinically useful result (termination for efficacy), although this practice has been criticised as likely to overestimate effect sizes [13].”
BackgroundMethodische Anforderungen an klinische Studien und ihre Interpretation
Windeler, J., & Lange, S. (2009) · Bundesgesundheitsblatt - Gesundheitsforschung - Gesundheitsschutz · doi:10.1007/s00103-009-0826-4 · Cited by 2
González‐Padilla, D. A., Subiela, J. D., & Villacampa, F. (2024) · Translational Andrology and Urology · doi:10.21037/tau-23-563 · Cited by 2
“While not directly applicable to this context, it is worth noting that early termination for benefit in randomized controlled trials has historically presented challenges in the interpretation of the data showing implausibly large treatment effects (11).”
The latest therapeutic advances with spontaneous intracerebral hemorrhage
Griffin, M., Kwok, C. S., Qureshi, A. I., & Lip, G. Y. H. (2025) · Expert Review of Neurotherapeutics · doi:10.1080/14737175.2025.2502048 · Cited by 1
“Stopping trials early is known to increase the chance of overestimation of treatment benefit [20,21].”
Metadata
- Authors
- Bassler, D., Montori, V. M., Briel, M., Glasziou, P., & Guyatt, G.
- Affiliations
- University Children's Hospital Tübingen (DE)
- University of Basel (CH)
- University of Oxford (GB)
- McMaster University (CA)
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2008-03
- Volume
- 61
- Issue
- 3
- Pages
- 241-246
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2042803707
- Cited by
- 130
- References
- 38
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 7 · 2023: 3 · 2022: 5 · 2021: 3 · 2020: 6 · 2019: 6 · 2018: 9 · 2017: 4 · 2016: 7 · 2015: 9 · 2014: 9 · 2013: 8 · 2012: 9
- FWCI
- 1.92
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- DigitalGeorgetown (Georgetown University Library) (submittedVersion)
- Funders
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
- Medical Research Council
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- early stopping, randomized controlled trials, interim analysis, overestimation of treatment effects, statistical power, p-value threshold, trial ethics
- MeSH terms
- Decision Making, Humans, Motivation, Bias, Randomized Controlled Trials as Topic, Treatment Outcome, Ethics, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
