Prediction of Sepsis in the Intensive Care Unit With Minimal Electronic Health Record Data: A Machine Learning Approach
Open accessAbstract
BACKGROUND: Sepsis is one of the leading causes of mortality in hospitalized patients. Despite this fact, a reliable means of predicting sepsis onset remains elusive. Early and accurate sepsis onset predictions could allow more aggressive and targeted therapy while maintaining antimicrobial stewardship. Existing detection methods suffer from low performance and often require time-consuming laboratory test results. OBJECTIVE: To study and validate a sepsis prediction method, InSight, for the new Sepsis-3 definitions in retrospective data, make predictions using a minimal set of variables from within the electronic health record data, compare the performance of this approach with existing scoring systems, and investigate the effects of data sparsity on InSight performance. METHODS: We apply InSight, a machine learning classification system that uses multivariable combinations of easily obtained patient data (vitals, peripheral capillary oxygen saturation, Glasgow Coma Score, and age), to predict sepsis using the retrospective Multiparameter Intelligent Monitoring in Intensive Care (MIMIC)-III dataset, restricted to intensive care unit (ICU) patients aged 15 years or more. Following the Sepsis-3 definitions of the sepsis syndrome, we compare the classification performance of InSight versus quick sequential organ failure assessment (qSOFA), modified early warning score (MEWS), systemic inflammatory response syndrome (SIRS), simplified acute physiology score (SAPS) II, and sequential organ failure assessment (SOFA) to determine whether or not patients will become septic at a fixed period of time before onset. We also test the robustness of the InSight system to random deletion of individual input observations. RESULTS: In a test dataset with 11.3% sepsis prevalence, InSight produced superior classification performance compared with the alternative scores as measured by area under the receiver operating characteristic curves (AUROC) and area under precision-recall curves (APR). In detection of sepsis onset, InSight attains AUROC = 0.880 (SD 0.006) at onset time and APR = 0.595 (SD 0.016), both of which are superior to the performance attained by SIRS (AUROC: 0.609; APR: 0.160), qSOFA (AUROC: 0.772; APR: 0.277), and MEWS (AUROC: 0.803; APR: 0.327) computed concurrently, as well as SAPS II (AUROC: 0.700; APR: 0.225) and SOFA (AUROC: 0.725; APR: 0.284) computed at admission (P<.001 for all comparisons). Similar results are observed for 1-4 hours preceding sepsis onset. In experiments where approximately 60% of input data are deleted at random, InSight attains an AUROC of 0.781 (SD 0.013) and APR of 0.401 (SD 0.015) at sepsis onset time. Even with 60% of data missing, InSight remains superior to the corresponding SIRS scores (AUROC and APR, P<.001), qSOFA scores (P=.0095; P<.001) and superior to SOFA and SAPS II computed at admission (AUROC and APR, P<.001), where all of these comparison scores (except InSight) are computed without data deletion. CONCLUSIONS: Despite using little more than vitals, InSight is an effective tool for predicting sepsis onset and performs well even with randomly missing data.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Existing studies have largely focused on binary outcome prediction and less on dynamic prediction of patient recovery trajectories.
“However, existing studies have largely focused on binary outcome prediction (e.g., mortality) and less on dynamic prediction of patient recovery trajectories12,13.”
From Machine learning predicts sepsis deterioration trajectories. (2026) · cites this paper as [13]
The Methods (Machine Learning Methods; page unavailable) describe a binary classifier predicting whether an ICU patient will become septic, so this paper is an example of binary-outcome prediction. It does not establish that existing studies have largely focused on binary outcomes or rarely predict recovery trajectories. Its predictions concern sepsis onset at several time horizons, not mortality or recovery. (LLM, conf=0.86)
Experimental — accuracy not yet measuredNumerous models were tested on the freely accessible MIMIC database.
“numerous models were tested on the freely accessible MIMIC database [27, 33, 52–59]”
From Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy (2020) · cites this paper as [54]
The Methods (Dataset; Data Extraction and Imputation) say the study used MIMIC-III to calculate InSight and comparison scores, and the Results compare their performance on that dataset. Our copy does not state that MIMIC-III is freely accessible, so that qualifier is not verified by this source. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vincent, J. L., Moreno, R., Takala, J., Willatts, S., De Mendonça, A., Bruining, H., Reinhart, C. K., Suter, P. M., & Thijs, L. G. (1996) · Intensive Care Medicine · doi:10.1007/bf01709751 · Cited by 11,674 · see citing works
A computational approach to early sepsis detection
Calvert, J. S., Price, D. A., Chettipally, U. K., Barton, C. W., Feldman, M. D., Hoffman, J. L., Jay, M., & Das, R. (2016) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2016.05.003 · Cited by 244 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 18
- [1]Fleischmann C Scherag A Adhikari NK Hartog CS Tsaganos T Schlattmann P Angus DC Reinhart K International Forum of Acute Care Trialists Assessment of Global Incidence and Mortality of Hospital-treated Sepsis. Current Estimates and Limitations Am J Respir Crit Care Med 2016 02 1 193 3 259 72 10.1164/rccm.201504-0781OC 26414292 26414292 · DOI · Paper page
- [2]Angus DC Linde-Zwirble WT Lidicker J Clermont G Carcillo J Pinsky MR Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care Crit Care Med 2001 07 29 7 1303 10 11445675 11445675
- [3]Singer M Deutschman CS Seymour CW Shankar-Hari M Annane D Bauer M Bellomo R Bernard GR Chiche J Coopersmith CM Hotchkiss RS Levy MM Marshall JC Martin GS Opal SM Rubenfeld GD van der Poll T Vincent J Angus DC The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) JAMA 2016 02 23 315 8 801 10 10.1001/jama.2016.0287 26903338 2492881 26903338 PMC4968574 · DOI · Paper page
- [6]Calvert J Desautels T Chettipally U Barton C Hoffman J Jay M Mao Q Mohamadlou H Das R High-performance detection and early prediction of septic shock for alcohol-use disorder patients Ann Med Surg (Lond) 2016 06 8 50 5 10.1016/j.amsu.2016.04.023 27489621 S2049-0801(16)30041-3 27489621 PMC4960347 · DOI · Paper page
- [7]Subbe CP Slater A Menon D Gemmell L Validation of physiological scoring systems in the accident and emergency department Emerg Med J 2006 11 23 11 841 5 10.1136/emj.2006.035816 17057134 23/11/841 17057134 PMC2464409 · DOI
- [8]Le Gall J Lemeshow S Saulnier F A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study JAMA 1993 270 24 2957 63 8254858 8254858
- [9]Balk RA Severe sepsis and septic shock. Definitions, epidemiology, and clinical manifestations Crit Care Clin 2000 04 16 2 179 92 10768078 10768078
- [10]Johnson AE Pollard TJ Shen L Lehman LH Feng M Ghassemi M Moody B Szolovits P Celi LA Mark RG MIMIC-III, a freely accessible critical care database Sci Data 2016 3 160035 10.1038/sdata.2016.35 sdata201635 27219127 PMC4878278 · DOI · Paper page
- [11]Seymour CW Liu VX Iwashyna TJ Brunkhorst FM Rea TD Scherag A Rubenfeld G Kahn JM Shankar-Hari M Singer M Deutschman CS Escobar GJ Angus DC Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) JAMA 2016 02 23 315 8 762 74 10.1001/jama.2016.0288 26903335 2492875 26903335 PMC5433435 · DOI · Paper page
- [12]Jones AE Trzeciak S Kline JA The Sequential Organ Failure Assessment score for predicting outcome in patients with severe sepsis and evidence of hypoperfusion at the time of emergency department presentation Crit Care Med 2009 05 37 5 1649 54 10.1097/CCM.0b013e31819def97 19325482 19325482 PMC2703722 · DOI
- [13]Holcomb JB Salinas J McManus JM Miller CC Cooke WH Convertino VA Manual vital signs reliably predict need for life-saving interventions in trauma patients J Trauma 2005 10 59 4 821 8; discussion 828 16374268 00005373-200510000-00003 16374268
- [15]Zou H Hastie T Regularization and variable selection via the elastic net J Royal Statistical Soc B 2005 04 67 2 301 320 10.1111/j.1467-9868.2005.00503.x · DOI · Paper page
- [16]Calvert JS Price DA Barton CW Chettipally UK Das R Discharge recommendation based on a novel technique of homeostatic analysis J Am Med Inform Assoc 2016 03 28 Epub ahead of print(forthcoming) 10.1093/jamia/ocw014 27026611 ocw014 PMC7654071 27026611 · DOI
- [17]Ruskin KJ Hueske-Kraus D Alarm fatigue: impacts on patient safety Curr Opin Anaesthesiol 2015 12 28 6 685 90 10.1097/ACO.0000000000000260 26539788 00001503-201512000-00013 26539788 · DOI · Paper page
- [18]Nachimuthu SK Haug PJ Early detection of sepsis in the emergency department using Dynamic Bayesian Networks AMIA Annu Symp Proc 2012 2012 653 62 23304338 23304338 PMC3540576
- [19]Stanculescu I Williams C Freer Y Autoregressive hidden Markov models for the early detection of neonatal sepsis IEEE J Biomed Health Inform 2014 09 18 5 1560 70 10.1109/JBHI.2013.2294692 25192568 25192568 · DOI
- [20]Stanculescu I Williams CK Freer Y A Hierarchical Switching Linear Dynamical System Applied to the Detection of Sepsis in Neonatal Condition Monitoring. Presented at the Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence (UAI) 2014 07 23 2014 Quebec City, Quebec, Canada 752 761
- [21]Henry KE Hager DN Pronovost PJ Saria S A targeted real-time early warning score (TREWScore) for septic shock Sci Transl Med 2015 08 5 7 299 299ra122 10.1126/scitranslmed.aab3719 26246167 7/299/299ra122 26246167 · DOI · Paper page
Cited by
Sorted by citations
Artificial intelligence, bias and clinical safety
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008370 · Cited by 956 · see citing works
An interpretable machine learning model for accurate prediction of sepsis in the ICU
Nemati, S., Holder, A., Razmi, F., Stanley, M. D., Clifford, G. D., & Buchman, T. G. (2018) · Critical Care Medicine · doi:10.1097/ccm.0000000000002936 · Cited by 832 · see citing works
Fleuren, L. M., Klausch, T. L. T., Zwager, C. L., Schoonmade, L. J., Guo, T., Roggeveen, L. F., Swart, E. L., Girbes, A. R. J., Thoral, P., Ercole, A., et al. (2020) · Intensive Care Medicine · doi:10.1007/s00134-019-05872-y · Cited by 755 · see citing works
Shimabukuro, D. W., Barton, C. W., Feldman, M. D., Mataraso, S. J., & Das, R. (2017) · BMJ Open Respiratory Research · doi:10.1136/bmjresp-2017-000234 · Cited by 382 · see citing works
Self-citationAdams, R., Henry, K. E., Sridharan, A., Soleimani, H., Zhan, A., Rawat, N., Johnson, L., Hager, D. N., Cosgrove, S. E., Markowski, A., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01894-0 · Cited by 365 · see citing works
Predicting hospital admission at emergency department triage using machine learning
Hong, W. S., Haimovich, A. D., & Taylor, R. A. (2018) · PLOS ONE · doi:10.1371/journal.pone.0201016 · Cited by 326 · see citing works
Sendak, M. P., Ratliff, W., Sarro, D., Alderton, E., Futoma, J., Gao, M., Nichols, M., Revoir, M., Yashar, F., Miller, C., et al. (2020) · JMIR Medical Informatics · doi:10.2196/15182 · Cited by 287 · see citing works
Prediction of sepsis patients using machine learning approach: a meta-analysis
Islam, M. M., Nasrin, T., Walther, B. A., Wu, C.-C., Yang, H.-C., & Li, Y.-C. (2019) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2018.12.027 · Cited by 259 · see citing works
Delahanty, R. J., Alvarez, J., Flynn, L. M., Sherwin, R. L., & Jones, S. S. (2019) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2018.11.036 · Cited by 206 · see citing works
Barton, C., Chettipally, U., Zhou, Y., Jiang, Z., Lynn-Palevsky, A., Le, S., Calvert, J., & Das, R. (2019) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2019.04.027 · Cited by 175 · see citing works
Self-citationShashikumar, S. P., Stanley, M. D., Sadiq, I., Li, Q., Holder, A., Clifford, G. D., & Nemati, S. (2017) · Journal of Electrocardiology · doi:10.1016/j.jelectrocard.2017.08.013 · Cited by 167 · see citing works
Ginestra, J. C., Giannini, H. M., Schweickert, W. D., Meadows, L., Lynch, M. J., Pavan, K., Chivers, C. J., Draugelis, M., Donnelly, P. J., Fuchs, B. D., et al. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003803 · Cited by 159 · see citing works
McCoy, A., & Das, R. (2017) · BMJ Open Quality · doi:10.1136/bmjoq-2017-000158 · Cited by 148 · see citing works
Self-citationClinical applications of artificial intelligence in sepsis: A narrative review
Schinkel, M., Paranjape, K., Nannan Panday, R. S., Skyttberg, N., & Nanayakkara, P. W. B. (2019) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2019.103488 · Cited by 130 · see citing works
Machine learning for early detection of sepsis: an internal and temporal validation study
Bedoya, A. D., Futoma, J., Clement, M. E., Corey, K., Brajer, N., Lin, A., Simons, M. G., Gao, M., Nichols, M., Balu, S., et al. (2020) · JAMIA Open · doi:10.1093/jamiaopen/ooaa006 · Cited by 109 · see citing works
Luz, C. F., Vollmer, M., Decruyenaere, J., Nijsten, M. W., Glasner, C., & Sinha, B. (2020) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2020.02.003 · Cited by 106 · see citing works
The initial resuscitation of septic shock
Cinel, I., Kasapoglu, U. S., Gul, F., & Dellinger, R. P. (2020) · Journal of Critical Care · doi:10.1016/j.jcrc.2020.02.004 · Cited by 69 · see citing works
van der Vegt, A. H., Scott, I. A., Dermawan, K., Schnetler, R. J., Kalke, V. R., & Lane, P. J. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad075 · Cited by 72 · see citing works
“Deciding which features to input and how simple (eg, vital signs only) or complex (eg, waveform data and laboratory results) they are was seen to influence model generalizability (D6) to different care locations.(38,57,58,62) Group (B) supported different variables for different sites, claiming flexibility,(62) althou…”
BackgroundKijpaisalratana, N., Sanglertsinlapachai, D., Techaratsami, S., Musikatavorn, K., & Saoraya, J. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104689 · Cited by 65 · see citing works
Persson, I., Östling, A., Arlbrandt, M., Söderberg, J., & Becedas, D. (2021) · JMIR Formative Research · doi:10.2196/28000 · Cited by 58 · see citing works
Joshi, M., Mecklai, K., Rozenblum, R., & Samal, L. (2022) · JAMIA Open · doi:10.1093/jamiaopen/ooac022 · Cited by 46 · see citing works
Hassan, N., Slight, R., Weiand, D., Vellinga, A., Morgan, G., Aboushareb, F., & Slight, S. P. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104457 · Cited by 42 · see citing works
Transferability and interpretability of the sepsis prediction models in the intensive care unit
Chen, Q., Li, R., Lin, C., Lai, C., Chen, D., Qu, H., Huang, Y., Lu, W., Tang, Y., & Li, L. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-02090-3 · Cited by 28 · see citing works
Boussi Rahmouni, H., Hassine, N. B. E. H., Chouchen, M., Ceylan, H. İ., Muntean, R. I., Bragazzi, N. L., & Dergaa, I. (2025) · Healthcare · doi:10.3390/healthcare13202553 · Cited by 25 · see citing works
Enhancing sepsis management through machine learning techniques: A review
Ocampo-Quintero, N., Vidal-Cortés, P., del Río Carbajo, L., Fdez-Riverola, F., Reboiro-Jato, M., & Glez-Peña, D. (2022) · Medicina Intensiva · doi:10.1016/j.medin.2020.04.003 · Cited by 20 · see citing works
Lipatov, K., Daniels, C. E., Park, J. G., Elmer, J., Hanson, A. C., Madsen, B. E., Clements, C. M., Gajic, O., Pickering, B. W., & Herasevich, V. (2022) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2021.09.086 · Cited by 11 · see citing works
An Explainable Machine Learning Model for Early Prediction of Sepsis Using ICU Data
Nesaragi, N., & Patidar, S. (2021) · Infections and Sepsis Development · doi:10.5772/intechopen.98957 · Cited by 11 · see citing works
Machine learning predicts sepsis deterioration trajectories.
Zhang, R., Long, F., Zhao, Z., Wu, J., Tan, R., Xu, W., Li, L., Long, Y., & Qu, H. (2026) · NPJ Digital Medicine · doi:10.1038/s41746-026-02565-x · Cited by 6 · see citing works
“However, existing studies have largely focused on binary outcome prediction (e.g., mortality) and less on dynamic prediction of patient recoverytrajectories 12,13 .”
- Authors
- Desautels, T., Calvert, J., Hoffman, J., Jay, M., Kerem, Y., Shieh, L., Shimabukuro, D., Chettipally, U., Feldman, M. D., Barton, C., Wales, D. J., & Das, R.
- Affiliations
- Dascena (United States) (US)
- Kaiser Permanente Redwood City Medical Center (US)
- Stanford Medicine (US)
- University of California, San Francisco (US)
- University of Cambridge (GB)
- Corresponding author
- Hoffman, Jana
- ORCID
- Venue
- JMIR Medical Informatics
- Publisher
- JMIR Publications Inc.
- Published
- 2016-09-30
- Volume
- 4
- Issue
- 3
- Pages
- e28
- ISSN
- 2291-9694 (ISSN-L)
- PMC
- PMC5065680
- OpenAlex
- W2523834880
- Cited by
- 509
- References
- 2
- Citations by year
- 2026: 43 · 2025: 56 · 2024: 54 · 2023: 41 · 2022: 57 · 2021: 80 · 2020: 77 · 2019: 55 · 2018: 27 · 2017: 19
- FWCI
- 31.58
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Medical Informatics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- National Science Foundation (1549867)
- Topics
- Health Sciences › Medicine › Epidemiology › Sepsis Diagnosis and Treatment
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- sepsis prediction, insight, Sepsis-3 definitions, intensive care unit patients, electronic health record data, qSOFA, Modified Early Warning Score, SIRS, SAPS II, SOFA, sepsis severity scoring, data sparsity, MIMIC-III, machine learning classification, early warning scores
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
