CoChat
Sign Up
PreprintCited by 3Sepsis Diagnosis and TreatmentMachine Learning in HealthcareArtificial Intelligence in Healthcare

The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.

More Generalizable Models For Sepsis Detection Under Covariate Shift

Open access
Gao, Jifan, Mar, Philip L., Chen, Guanhua·2021·arXiv
DOI 10.48550/arxiv.2105.09385arXiv 2105.09385PMC PMC8378628
Open this paper in CoChat — free account

Abstract

Sepsis is a major cause of mortality in the intensive care units (ICUs). Early intervention of sepsis can improve clinical outcomes for sepsis patients. Machine learning models have been developed for clinical recognition of sepsis. A common assumption of supervised machine learning models is that the covariates in the testing data follow the same distributions as those in the training data. When this assumption is violated (e.g., there is covariate shift), models that performed well for training data could perform badly for testing data. Covariate shift happens when the relationships between covariates and the outcome stay the same, but the marginal distributions of the covariates differ among training and testing data. Covariate shift could make clinical risk prediction model nongeneralizable. In this study, we applied covariate shift corrections onto common machine learning models and have observed that these corrections can help the models be more generalizable under the occurrence of covariate shift when detecting the onset of sepsis.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2105.09385

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A common reason for decline in model performance is data drift, which is seen when the population characteristics on which the model was developed is different from the population characteristics on which the model is applied.

    “A common reason for such decline is data drift, which is seen when the population characteristics on which the model was developed is different from the population characteristics on which the model is applied [44].”

    From Bias in artificial intelligence algorithms and recommendations for mitigation (2023) · cites this paper as [44]

  • Data drift can be problematic when predicting the onset of life-threatening diseases such as sepsis.

    “This can be problematic when predicting the onset of life-threatening diseases such as sepsis [44].”

    From Bias in artificial intelligence algorithms and recommendations for mitigation (2023) · cites this paper as [44]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 27

  1. [1]1. Sakr Y, Jaschinski U, Wittebole X, Szakmany T, Lipman J, ˜ Namendys-Silva SA, et al. Sepsis in intensive care unit patients: worldwide data from the intensive care over nations audit. In: Open forum infectious diseases. vol. 5. Oxford University Press US; 2018. p. ofy313.
  2. [2]2. Rhee C, Kadri SS, Danner RL, Suffredini AF, Massaro AF, Kitch BT, et al. Diagnosing sepsis is subjective and highly variable: a survey of intensivists using case vignettes. Critical Care. 2016;20(1):89.
  3. [3]3. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). Jama. 2016;315(8):801-810.
  4. [4]4. Mitra A, Ashraf K. Sepsis prediction and vital signs ranking in intensive care unit patients. arXiv preprint arXiv:181206686. 2018;.
  5. [6]6. Pollard TJ, Johnson AE, Raffa JD, Celi LA, Mark RG, Badawi O. The eICU Collaborative Research Database, a freely available multi-center database for critical care research. Scientific data. 2018;5:180178.
  6. [7]7. Kumar A, Roberts D, Wood KE, Light B, Parrillo JE, Sharma S, et al. Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Critical care medicine. 2006;34(6):1589-1596.
  7. [8]8. Subbe C, Slater A, Menon D, Gemmell L. Validation of physiological scoring systems in the accident and emergency department. Emergency Medicine Journal. 2006;23(11):841-845.
  8. [9]9. Rangel-Frausto MS, Pittet D, Costigan M, Hwang T, Davis CS, Wenzel RP. The natural history of the systemic inflammatory response syndrome (SIRS): a prospective study. Jama. 1995;273(2):117-123.
  9. [10]10. Vincent JL, Moreno R, Takala J, Willatts S, De Mendonc ¸a A, Bruining H, et al.. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. Springer-Verlag; 1996.
  10. [11]11. Lyra S, Leonhardt S, Antink CH. Early Prediction of Sepsis Using Random Forest Classification for Imbalanced Clinical Data. In: 2019 Computing in Cardiology (CinC). IEEE; 2019. p. 1-4.
  11. [12]12. Dexter GP, Grannis SJ, Dixon BE, Kasthurirathne SN. Generalization of Machine Learning Approaches to Identify Notifiable Conditions from a Statewide Health Information Exchange. AMIA Summits on Translational Science Proceedings. 2020;2020:152.
  12. [13]13. Quionero-Candela J, Sugiyama M, Schwaighofer A, Lawrence ND. Dataset shift in machine learning. The MIT Press; 2009.
  13. [14]14. Moreno-Torres JG, Raeder T, Alaiz-Rodr´ ıGuez R, Chawla NV, Herrera F. A unifying view on dataset shift in classification. Pattern recognition. 2012;45(1):521-530.
  14. [15]15. Hwang EJ, Park S, Jin KN, Im Kim J, Choi SY, Lee JH, et al. Development and validation of a deep learningbased automated detection algorithm for major thoracic diseases on chest radiographs. JAMA network open. 2019;2(3):e191095-e191095.
  15. [16]16. Nestor B, McDermott M, Boag W, Berner G, Naumann T, Hughes MC, et al. Feature robustness in non-stationary health records: caveats to deployable model performance in common clinical machine learning tasks. arXiv preprint arXiv:190800690. 2019;.
  16. [17]17. Curth A, Thoral P, van den Wildenberg W, Bijlstra P, de Bruin D, Elbers P, et al. Transferring clinical prediction models across hospitals and electronic health record systems. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer; 2019. p. 605-621.
  17. [18]18. Sugiyama M, Krauledat M, M ˜ Aˇ zller KR. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research. 2007;8(May):985-1005.
  18. [19]19. Huang J, Gretton A, Borgwardt K, Sch¨ olkopf B, Smola AJ. Correcting sample selection bias by unlabeled data. In: Advances in neural information processing systems; 2007. p. 601-608.
  19. [20]20. Yamada M, Suzuki T, Kanamori T, Hachiya H, Sugiyama M. Relative density-ratio estimation for robust distribution comparison. Neural computation. 2013;25(5):1324-1370.
  20. [21]21. Bickel S, Br¨ uckner M, Scheffer T. Discriminative learning for differing training and test distributions. In: Proceedings of the 24th international conference on Machine learning; 2007. p. 81-88.
  21. [22]22. Breiman L. Random forests. Machine learning. 2001;45(1):5-32.
  22. [23]23. Zou H, Hastie T. Regularization and variable selection via the elastic net. Journal of the royal statistical society: series B (statistical methodology). 2005;67(2):301-320.
  23. [24]24. Alba AC, Agoritsas T, Walsh M, Hanna S, Iorio A, Devereaux P, et al. Discrimination and calibration of clinical prediction models: users' guides to the medical literature. Jama. 2017;318(14):1377-1384.
  24. [25]25. Davis SE, Lasko TA, Chen G, Matheny ME. Calibration drift among regression and machine learning models for hospital mortality. In: AMIA Annual Symposium Proceedings. vol. 2017. American Medical Informatics Association; 2017. p. 625.
  25. [27]27. Kalai E, Samet D. On weighted Shapley values. International journal of game theory. 1987;16(3):205-222.
  26. [28]28. Lundberg SM, Lee SI. A unified approach to interpreting model predictions. In: Advances in neural information processing systems; 2017. p. 4765-4774.
  27. [29]29. Kaukonen KM, Bailey M, Pilcher D, Cooper DJ, Bellomo R. Systemic inflammatory response syndrome criteria in defining severe sepsis. New England Journal of Medicine. 2015;372(17):1629-1638.

Cited by

Sorted by citations

Metadata

Authors
Gao, J., Mar, P. L., & Chen, G.
Affiliations
Saint Louis University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2021
Volume
2021
Pages
220-228
DOI
10.48550/arxiv.2105.09385
arXiv
2105.09385
PMC
PMC8378628
OpenAlex
W3162911549
Cited by
3
References
1
Citations by year
2025: 2 · 2023: 1
FWCI
0.18
Type
journal article
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Sepsis Diagnosis and Treatment
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
sepsis detection, covariate shift, model generalization, clinical risk prediction
MeSH terms
Machine Learning, Humans, Intensive Care Units, Sepsis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review