Outcomes and Statistical Power in Adult Critical Care Randomized Trials
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
RATIONALE: Intensive care unit (ICU)-based randomized clinical trials (RCTs) among adult critically ill patients commonly fail to detect treatment benefits. OBJECTIVES: Appraise the rates of success, outcomes used, statistical power, and design characteristics of published trials. METHODS: One hundred forty-six ICU-based RCTs of diagnostic, therapeutic, or process/systems interventions published from January 2007 to May 2013 in 16 high-impact general or critical care journals were studied. MEASUREMENT AND MAIN RESULTS: Of 146 RCTs, 54 (37%) were positive (i.e., the a priori hypothesis was found to be statistically significant). The most common primary outcomes were mortality (n = 40 trials), infection-related outcomes (n = 33), and ventilation-related outcomes (n = 30), with positive results found in 10, 58, and 43%, respectively. Statistical power was discussed in 135 RCTs (92%); 92 cited a rationale for their power parameters. Twenty trials failed to achieve at least 95% of their reported target sample size, including 11 that were stopped early due to insufficient accrual/logistical issues. Of 34 superiority RCTs comparing mortality between treatment arms, 13 (38%) accrued a sample size large enough to find an absolute mortality reduction of 10% or less. In 22 of these trials the observed control-arm mortality rate differed from the predicted rate by at least 7.5%. CONCLUSIONS: ICU-based RCTs are commonly negative and powered to identify what appear to be unrealistic treatment effects, particularly when using mortality as the primary outcome. Additional concerns include a lack of standardized methods for assessing common outcomes, unclear justifications for statistical power calculations, insufficient patient accrual, and incorrect predictions of baseline event rates.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2008.08.010 · Cited by 51,384
(2000) · New England Journal of Medicine · doi:10.1056/nejm200005043421801 · Cited by 12,941
Prone positioning in severe acute respiratory distress syndrome.
Guérin, C., Reignier, J., Richard, J.-C., Beuret, P., Gacouin, A., Boulain, T., Mercier, E., Badet, M., Mercat, A., Baudin, O., et al. (2013) · New England Journal of Medicine · doi:10.1056/nejmoa1214103 · Cited by 4,040
Intensive insulin therapy in the medical ICU
Van den Berghe, G., Wilmer, A., Hermans, G., Meersseman, W., Wouters, P. J., Milants, I., Van Wijngaerden, E., Bobbaers, H., & Bouillon, R. (2006) · New England Journal of Medicine · doi:10.1056/nejmoa052521 · Cited by 3,639
Assessing the quality of controlled clinical trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · BMJ · doi:10.1136/bmj.323.7303.42 · Cited by 3,309
Neuromuscular Blockers in Early Acute Respiratory Distress Syndrome
Papazian, L., Forel, J.-M., Gacouin, A., Penot-Ragon, C., Perrin, G., Loundou, A., Jaber, S., Arnal, J.-M., Perez, D., Seghboyan, J.-M., et al. (2010) · New England Journal of Medicine · doi:10.1056/nejmoa1005372 · Cited by 2,432
Jansen, T. C., van Bommel, J., Schoonderbeek, F. J., Sleeswijk Visser, S. J., van der Klooster, J. M., Lima, A. P., Willemsen, S. P., & Bakker, J. (2010) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.200912-1918oc · Cited by 894
Hopewell, S., Clarke, M., Moher, D., Wager, E., Middleton, P., Altman, D. G., Schulz, K. F., & and the CONSORT Group (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050020 · Cited by 674
Hernández, A. V., Steyerberg, E. W., & Habbema, J. D. F. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2003.09.014 · Cited by 258
McHugh, G. S., Butcher, I., Steyerberg, E. W., Marmarou, A., Lu, J., Lingsma, H. F., Weir, J., Maas, A. I. R., & Murray, G. D. (2010) · Clinical Trials · doi:10.1177/1740774509356580 · Cited by 108
Cited by
Reappraisal of Ventilator-Free Days in Critical Care Research
Yehya, N., Harhay, M. O., Curley, M. A. Q., Schoenfeld, D. A., & Reeder, R. W. (2019) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201810-2050cp · Cited by 385
Self-citationRandomized controlled trials – a matter of design
Siepmann, T., Spieth, P. M., Kubasch, A. S., Penzlin, A. I., Illigens, B. M.-W., & Barlinn, K. (2016) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s101938 · Cited by 292
“…the subgroup analysis should be viewed solely as a hypothesis-generating technique warranting further confirmation in a subsequent RCT. Harhay et al 33 reviewed 146 RCTs in the field of intensive care that have been published between 2007 and 2013 in 16 high-impact or critical care journals.”
Iwashyna, T. J., Burke, J. F., Sussman, J. B., Prescott, H. C., Hayward, R. A., & Angus, D. C. (2015) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201411-2125cp · Cited by 276
States Worse Than Death Among Hospitalized Patients With Serious Illnesses
Rubin, E. B., Buehler, A. E., & Halpern, S. D. (2016) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.4362 · Cited by 232
Self-citationUnderstanding and Enhancing Sepsis Survivorship. Priorities for Research and Practice
Prescott, H. C., Iwashyna, T. J., Blackwood, B., Calandra, T., Chlan, L. L., Choong, K., Connolly, B., Dark, P., Ferrucci, L., Finfer, S., et al. (2019) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201812-2383cp · Cited by 166
Metadata
- Authors
- Harhay, M. O., Wagner, J., Ratcliffe, S. J., Bronheim, R. S., Gopal, A., Green, S., Cooney, E., Mikkelsen, M. E., Kerlin, M. P., Small, D. S., & Halpern, S. D.
- Venue
- American Journal of Respiratory and Critical Care Medicine
- Publisher
- Oxford University Press (OUP)
- Published
- 2014-06-15
- Volume
- 189
- Issue
- 12
- Pages
- 1469-1478
- ISSN
- 1073-449X (ISSN-L)
- 1535-4970
- OpenAlex
- W2072051993
- Cited by
- 202
- References
- 10
- FWCI
- 13.06
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.