Implications of Heterogeneity of Treatment Effect for Reporting and Analysis of Randomized Trials in Critical Care
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
Randomized clinical trials (RCTs) are conducted to guide clinicians' selection of therapies for individual patients. Currently, RCTs in critical care often report an overall mean effect and selected individual subgroups. Yet work in other fields suggests that such reporting practices can be improved. Specifically, this Critical Care Perspective reviews recent work on so-called "heterogeneity of treatment effect" (HTE) by baseline risk and extends that work to examine its applicability to trials of acute respiratory failure and severe sepsis. Because patients in RCTs in critical care medicine-and patients in intensive care units-have wide variability in their risk of death, these patients will have wide variability in the absolute benefit that they can derive from a given therapy. If the side effects of the therapy are not perfectly collinear with the treatment benefits, this will result in HTE, where different patients experience quite different expected benefits of a therapy. We use simulations of RCTs to demonstrate that such HTE could result in apparent paradoxes, including: (1) positive trials of therapies that are beneficial overall but consistently harm or have little benefit to low-risk patients who met enrollment criteria, and (2) overall negative trials of therapies that still consistently benefit high-risk patients. We further show that these results persist even in the presence of causes of death unmodified by the treatment under study. These results have implications for reporting and analyzing RCT data, both to better understand how our therapies work and to improve the bedside applicability of RCTs. We suggest a plan for measurement in future RCTs in the critically ill.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Le Gall, J.-R. (1993) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.270.24.2957 · Cited by 7,773
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544 · cited 1×
“11) In the simulations, we assumed that each individual’s odds of 30-day mortality were influenced by four factors: (1) severity of acute respiratory failure; (2) comorbid conditions (such as chronic health problems or other features of the critical illness); (3) the treatment’s reduction in mortality from the primary…”
BackgroundSubgroup analysis in randomised controlled trials: importance, indications, and interpretation
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(05)17709-5 · Cited by 942
Targeting of Low-Dose CT Screening According to the Risk of Lung-Cancer Death
Kovalchik, S. A., Tammemagi, M., Berg, C. D., Caporaso, N. E., Riley, T. L., Korch, M., Silvestri, G. A., Chaturvedi, A. K., & Katki, H. A. (2013) · New England Journal of Medicine · doi:10.1056/nejmoa1301851 · Cited by 575
Kent, D. M., & Hayward, R. A. (2007) · JAMA · doi:10.1001/jama.298.10.1209 · Cited by 505
Self-citationPrediction of benefit from carotid endar terectomy in individual patients: a risk-modelling study
Rothwell, P., & Warlow, C. (1999) · The Lancet · doi:10.1016/s0140-6736(98)11415-0 · Cited by 411
Ely, E. W., Laterre, P.-F., Angus, D. C., Helterbrand, J. D., Levy, H., Dhainaut, J.-F., Vincent, J.-L., Macias, W. L., & Bernard, G. R. (2003) · Critical Care Medicine · doi:10.1097/00003246-200301000-00002 · Cited by 283
Self-citationFrom subgroups to individuals: general principles and the example of carotid endarterectomy
Rothwell, P. M., Mehta, Z., Howard, S. C., Gutnikov, S. A., & Warlow, C. P. (2005) · The Lancet · doi:10.1016/s0140-6736(05)17746-0 · Cited by 281
Comparison of Medical Admissions to Intensive Care Units in the United States and United Kingdom
Wunsch, H., Angus, D. C., Harrison, D. A., Linde-Zwirble, W. T., & Rowan, K. M. (2011) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201012-1961oc · Cited by 256
Self-citationOutcomes and Statistical Power in Adult Critical Care Randomized Trials
Harhay, M. O., Wagner, J., Ratcliffe, S. J., Bronheim, R. S., Gopal, A., Green, S., Cooney, E., Mikkelsen, M. E., Kerlin, M. P., Small, D. S., et al. (2014) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201401-0056cp · Cited by 202
Hayward, R. A., Kent, D. M., Vijan, S., & Hofer, T. P. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-18 · Cited by 196
Self-citationIn silico design of clinical trials: A method coming of age
Clermont, G., Bartels, J., Kumar, R., Constantine, G., Vodovotz, Y., & Chow, C. (2004) · Critical Care Medicine · doi:10.1097/01.ccm.0000142394.28791.c3 · Cited by 189
Optimizing Statin Treatment for Primary Prevention of Coronary Artery Disease
Hayward, R. A., Krumholz, H. M., Zulman, D. M., Timbie, J. W., & Vijan, S. (2010) · Annals of Internal Medicine · doi:10.7326/0003-4819-152-2-201001190-00004 · Cited by 132
Self-citationThe impact of high-risk patients on the results of clinical trials
Ioannidis, J. P. A., & Lau, J. (1997) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(97)00149-2 · Cited by 108
Reporting Clinical Trial Results To Inform Providers, Payers, And Consumers
Hayward, R. A., Kent, D. M., Vijan, S., & Hofer, T. P. (2005) · Health Affairs · doi:10.1377/hlthaff.24.6.1571 · Cited by 91
Self-citationBurke, J. F., Hayward, R. A., Nelson, J. P., & Kent, D. M. (2014) · Circulation: Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.113.000497 · Cited by 86
Self-citationAutomated intensive care unit risk adjustment: Results from a National Veterans Affairs study
Render, M. L., Kim, H. M., Welsh, D. E., Timmons, S., Johnston, J., Hui, S., Connors, A. F., Wagner, D., Daley, J., & Hofer, T. P. (2003) · Critical Care Medicine · doi:10.1097/01.ccm.0000055372.08235.09 · Cited by 78
Patient-Centered Performance Management: Enhancing Value for Patients and Health Care Systems
Kerr, E. A., & Hayward, R. A. (2013) · JAMA · doi:10.1001/jama.2013.6828 · Cited by 78
Self-citationVeterans Affairs intensive care unit risk adjustment model: Validation, updating, recalibration*
Render, M. L., Deddens, J., Freyberg, R., Almenoff, P., Connors, A. F., Wagner, D., & Hofer, T. P. (2008) · Critical Care Medicine · doi:10.1097/ccm.0b013e318169f290 · Cited by 76
Variation in outcomes in Veterans Affairs intensive care units with a computerized severity measure*
Render, M. L., Kim, H. M., Deddens, J., Sivaganesin, S., Welsh, D. E., Bickel, K., Freyberg, R., Timmons, S., Johnston, J., Connors, A. F., et al. (2005) · Critical Care Medicine · doi:10.1097/01.ccm.0000162497.86229.e9 · Cited by 75
Cooke, C. R., Kennedy, E. H., Wiitala, W. L., Almenoff, P. L., Sales, A. E., & Iwashyna, T. J. (2012) · Critical Care Medicine · doi:10.1097/ccm.0b013e3182591eee · Cited by 39
Self-citationRender, M. L., Welsh, D. E., Kollef, M., Lott, J. H., Hui, S., Weinberger, M., Tsevat, J., Hayward, R. A., & Hofer, T. P. (2000) · Critical Care Medicine · doi:10.1097/00003246-200010000-00033 · Cited by 34
Self-citationRender, M. L., Freyberg, R. W., Hasselbeck, R., Hofer, T. P., Sales, A. E., Deddens, J. A., Levesque, O., & Almenoff, P. L. (2011) · BMJ Quality & Safety · doi:10.1136/bmjqs.2009.037218 · Cited by 32
Cited by
Acute Respiratory Distress Syndrome: Advances in Diagnosis and Treatment
Fan, E., Brodie, D., & Slutsky, A. S. (2018) · JAMA · doi:10.1001/jama.2017.21907 · Cited by 1,470
Famous, K. R., Delucchi, K., Ware, L. B., Kangelaris, K. N., Liu, K. D., Thompson, B. T., & Calfee, C. S. (2017) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201603-0645oc · Cited by 820
Data mining in clinical big data: the frequently used databases, steps, and methodological models
Wu, W.-T., Li, Y.-J., Feng, A.-Z., Li, L., Huang, T., Xu, A.-D., & Lyu, J. (2021) · Military Medical Research · doi:10.1186/s40779-021-00338-z · Cited by 505
Lung- and Diaphragm-Protective Ventilation
Goligher, E. C., Dres, M., Patel, B. K., Sahetya, S. K., Beitler, J. R., Telias, I., Yoshida, T., Vaporidi, K., Grieco, D. L., Schepens, T., et al. (2020) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.202003-0655cp · Cited by 411
Prescott, H. C., Calfee, C. S., Thompson, B. T., Angus, D. C., & Liu, V. X. (2016) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201512-2544cp · Cited by 322
Self-citation
Metadata
- Authors
- Iwashyna, T. J., Burke, J. F., Sussman, J. B., Prescott, H. C., Hayward, R. A., & Angus, D. C.
- Venue
- American Journal of Respiratory and Critical Care Medicine
- Publisher
- Oxford University Press (OUP)
- Published
- 2015-11-01
- Volume
- 192
- Issue
- 9
- Pages
- 1045-1051
- ISSN
- 1073-449X (ISSN-L)
- 1535-4970
- OpenAlex
- W1924238075
- Cited by
- 276
- References
- 25
- FWCI
- 18.69
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.