CoChat
Sign Up
Cited by 43Sepsis Diagnosis and TreatmentMachine Learning in HealthcareArtificial Intelligence in Healthcare

The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.

Interpretable machine learning for predicting sepsis risk in emergency triage patients

Open access
Liu, Zheng, Shu, Wenqi, Li, Teng, Zhang, Xuan, Chong, Wei·2025·Scientific Reports
DOI 10.1038/s41598-025-85121-zPMC PMC11704257
Open this paper in CoChat — free accountOpen PDF

Abstract

The study aimed to develop and validate a sepsis prediction model using structured electronic medical records (sEMR) and machine learning (ML) methods in emergency triage. The goal was to enhance early sepsis screening by integrating comprehensive triage information beyond vital signs. This retrospective cohort study utilized data from the MIMIC-IV database. Two models were developed: Model 1 based on vital signs alone, and Model 2 incorporating vital signs, demographic characteristics, medical history, and chief complaints. Eight ML algorithms were employed, and model performance was evaluated using metrics such as AUC, F1 Score, and calibration curves. SHapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME) methods were used to enhance model interpretability. The study included 189,617 patients, with 5.95% diagnosed with sepsis. Model 2 consistently outperformed Model 1 across most algorithms. In Model 2, Gradient Boosting achieved the highest AUC of 0.83, followed by Extra Tree, Random Forest, and Support Vector Machine (all 0.82). The SHAP method provided more comprehensible explanations for the Gradient Boosting algorithm. Modeling with comprehensive triage information using sEMR and ML methods was more effective in predicting sepsis at triage compared to using vital signs alone. Interpretable ML enhanced model transparency and provided sepsis prediction probabilities, offering a feasible approach for early sepsis screening and aiding healthcare professionals in making informed decisions during the triage process.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41598-025-85121-z

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Flow Chart. MIMIC Medical Information Mart for Intensive Care, SBP systolic blood pressure, DBP diastolic blood pressure, o2sat oxygen saturation, AUC area under the receiver operating characteristic curve, AUC-PR area under the precision-recall curve, PPV positive predictive value, NPV negative predictive value, SHAP SHapley Additive exPlanations, LIME Local Interpretable Model-agnostic Explanations.
Fig. 1 Flow Chart. MIMIC Medical Information Mart for Intensive Care, SBP systolic blood pressure, DBP diastolic blood pressure, o2sat oxygen saturation, AUC area under the receiver operating characteristic curve, AUC-PR area under the precision-recall curve, PPV positive predictive value, NPV negative predictive value, SHAP SHapley Additive exPlanations, LIME Local Interpretable Model-agnostic Explanations.CC BY-NC-ND · PubMed Central
Fig. 2 Comparison of ROC Curves of Different Algorithms on Two Models. (a) Logistic Regression; (b) Decision Tree; (c) Extra Tree; (d) Gradient Boosting; (e) k-Nearest Neighbor: (f) Naive Bayes; (g) Random Forest; (h) Support Vector Machine. ROC receiver operating characteristic curve, AUC area under the receiver operating characteristic curve.
Fig. 2 Comparison of ROC Curves of Different Algorithms on Two Models. (a) Logistic Regression; (b) Decision Tree; (c) Extra Tree; (d) Gradient Boosting; (e) k-Nearest Neighbor: (f) Naive Bayes; (g) Random Forest; (h) Support Vector Machine. ROC receiver operating characteristic curve, AUC area under the receiver operating characteristic curve.CC BY-NC-ND · PubMed Central
Fig. 3 Calibration Curves and Decision Curve Analysis Curves for the Four Best-Performing Algorithms in Model 2. (a) Calibration Curves; (b) Decision Curve Analysis Curves; SVM Support Vector Machine.
Fig. 3 Calibration Curves and Decision Curve Analysis Curves for the Four Best-Performing Algorithms in Model 2. (a) Calibration Curves; (b) Decision Curve Analysis Curves; SVM Support Vector Machine.CC BY-NC-ND · PubMed Central
Fig. 4 Feature Importance of Four Algorithms in Model 2.
Fig. 4 Feature Importance of Four Algorithms in Model 2.CC BY-NC-ND · PubMed Central
Fig. 5 Interpretation of Four Algorithms in Model 2. SHAP SHapley Additive exPlanations, LIME Local Interpretable Model-agnostic Explanations. In the SHAP method, f (X) represented the final prediction result, which equaled the baseline value E [f (X)] plus the sum of all variable SHAP values. The SHAP values quantified the quantity and direction of each variable’s influence on predicting the outcome. Blue and red respectively represented decreases or increases in risk, with longer arrows indicating greater effects. The baseline value E [f (X)] was equivalent to the average risk in the dataset. The LIME method provided the overall prediction probability of the model and the prediction weight for each variable. Orange indicated an increase in risk, while blue indicated a decrease in risk.
Fig. 5 Interpretation of Four Algorithms in Model 2. SHAP SHapley Additive exPlanations, LIME Local Interpretable Model-agnostic Explanations. In the SHAP method, f (X) represented the final prediction result, which equaled the baseline value E [f (X)] plus the sum of all variable SHAP values. The SHAP values quantified the quantity and direction of each variable’s influence on predicting the outcome. Blue and red respectively represented decreases or increases in risk, with longer arrows indicating greater effects. The baseline value E [f (X)] was equivalent to the average risk in the dataset. The LIME method provided the overall prediction probability of the model and the prediction weight for each variable. Orange indicated an increase in risk, while blue indicated a decrease in risk.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 40

  1. Yealy, D. M. et al. Early care of adults with suspected sepsis in the emergency department and out-of-hospital environment: A consensus-based task force report. Ann. Emerg. Med. 78, 1–19. https://doi.org/10.1016/j.annemergmed.2021.02.006 (2021). · DOI
  2. Kalich, B. A. et al. Impact of an antibiotic-specific sepsis bundle on appropriate and timely antibiotic administration for severe sepsis in the emergency department. J. Emerg. Med. 50, 79-88.e71. https://doi.org/10.1016/j.jemermed.2015.09.007 (2016). · DOI
  3. Levy, M. M., Evans, L. E. & Rhodes, A. The Surviving Sepsis Campaign Bundle: 2018 update. Intensive Care Med. 44, 925–928. https://doi.org/10.1007/s00134-018-5085-0 (2018). · DOI · Paper page
  4. Wang, H. E., Jones, A. R. & Donnelly, J. P. Revised national estimates of emergency department visits for sepsis in the United States. Crit. Care Med. 45, 1443–1449. https://doi.org/10.1097/ccm.0000000000002538 (2017). · DOI
  5. Rhee, C. et al. Incidence and trends of sepsis in US hospitals using clinical vs claims data, 2009–2014. JAMA 318, 1241–1249. https://doi.org/10.1001/jama.2017.13836 (2017). · DOI · Paper page
  6. Wang, H. E., Weaver, M. D., Shapiro, N. I. & Yealy, D. M. Opportunities for Emergency Medical Services care of sepsis. Resuscitation 81, 193–197. https://doi.org/10.1016/j.resuscitation.2009.11.008 (2010). · DOI
  7. Femling, J., Weiss, S., Hauswald, E. & Tarby, D. EMS patients and walk-in patients presenting with severe sepsis: Differences in management and outcome. South Med. J. 107, 751–756. https://doi.org/10.14423/smj.0000000000000206 (2014). · DOI
  8. Rahmatinejad, Z. et al. Comparing in-hospital mortality prediction by senior emergency resident’s judgment and prognostic models in the emergency department. Biomed. Res. Int. 2023, 6042762. https://doi.org/10.1155/2023/6042762 (2023). · DOI
  9. Wattanasit, P. & Khwannimit, B. Comparison the accuracy of early warning scores with qSOFA and SIRS for predicting sepsis in the emergency department. Am. J. Emerg. Med. 46, 284–288. https://doi.org/10.1016/j.ajem.2020.07.077 (2021). · DOI
  10. Oduncu, A. F., Kıyan, G. S. & Yalçınlı, S. Comparison of qSOFA, SIRS, and NEWS scoring systems for diagnosis, mortality, and morbidity of sepsis in emergency department. Am. J. Emerg. Med. 48, 54–59. https://doi.org/10.1016/j.ajem.2021.04.006 (2021). · DOI
  11. Sabir, L., Ramlakhan, S. & Goodacre, S. Comparison of qSOFA and Hospital Early Warning Scores for prognosis in suspected sepsis in emergency department patients: A systematic review. Emerg. Med. J. EMJ 39, 284–294. https://doi.org/10.1136/emermed-2020-210416 (2022). · DOI
  12. Usman, O. A., Usman, A. A. & Ward, M. A. Comparison of SIRS, qSOFA, and NEWS for the early identification of sepsis in the Emergency Department. Am. J. Emerg. Med. 37, 1490–1497. https://doi.org/10.1016/j.ajem.2018.10.058 (2019). · DOI
  13. van der Woude, S. W., van Doormaal, F. F., Hutten, B. A., Nellen, F. J. & Holleman, F. Classifying sepsis patients in the emergency department using SIRS, qSOFA or MEWS. Netherlands J. Med. 76, 158–166 (2018).
  14. Brink, A. et al. Predicting mortality in patients with suspected sepsis at the Emergency Department; A retrospective cohort study comparing qSOFA, SIRS and National Early Warning Score. PLoS One 14, e0211133. https://doi.org/10.1371/journal.pone.0211133 (2019). · DOI
  15. Loritz, M., Busch, H. J., Helbing, T. & Fink, K. Prospective evaluation of the quickSOFA score as a screening for sepsis in the emergency department. Intern. Emerg. Med. 15, 685–693. https://doi.org/10.1007/s11739-019-02258-2 (2020). · DOI
  16. Rodriguez, R. M. et al. Comparison of qSOFA with current emergency department tools for screening of patients with sepsis for critical illness. Emerg. Med. J. EMJ 35, 350–356. https://doi.org/10.1136/emermed-2017-207383 (2018). · DOI
  17. Monclús Cols, E. et al. Comparison of the Quick Sepsis-related Organ Dysfunction score and severity levels assigned with the Andorran Triage Model in an urban tertiary care hospital emergency department. Emergencias revista de la Sociedad Espanola de Medicina de Emergencias 30, 400–404 (2018).
  18. Baig, M. A. et al. Comparison of qSOFA and SOFA score for predicting mortality in severe sepsis and septic shock patients in the emergency department of a low middle income country. Turk. J. Emerg. Med. 18, 148–151. https://doi.org/10.1016/j.tjem.2018.08.002 (2018). · DOI
  19. Evans, L. et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021. Intens. Care Med. 47, 1181–1247. https://doi.org/10.1007/s00134-021-06506-y (2021). · DOI · Paper page
  20. Fernandes, M. et al. Clinical decision support systems for triage in the emergency department using intelligent systems: A review. Artif. Intell. Med. 102, 101762. https://doi.org/10.1016/j.artmed.2019.101762 (2020). · DOI
  21. Mueller, B., Kinoshita, T., Peebles, A., Graber, M. A. & Lee, S. Artificial intelligence and machine learning in emergency medicine: A narrative review. Acute Med. Surg. 9, e740. https://doi.org/10.1002/ams2.740 (2022). · DOI
  22. Yun, H., Choi, J. & Park, J. H. Prediction of critical care outcome for adult patients presenting to emergency department using initial triage information: An XGBoost algorithm analysis. JMIR Med. Inform. 9, e30770. https://doi.org/10.2196/30770 (2021). · DOI
  23. De Hond, A. et al. Machine learning for developing a prediction model of hospital admission of emergency department patients: Hype or hope?. Int. J. Med. Inform. 152, 104496. https://doi.org/10.1016/j.ijmedinf.2021.104496 (2021). · DOI · Paper page
  24. Jin, Y. & Kattan, M. W. Methodologic issues specific to prediction model development and evaluation. Chest 164, 1281–1289. https://doi.org/10.1016/j.chest.2023.06.038 (2023). · DOI
  25. Freitag, A., Constanti, M., O’Flynn, N. & Faust, S. N. Suspected sepsis: Summary of NICE guidance. BMJ (Clin. Res. Ed.) 354, i4030. https://doi.org/10.1136/bmj.i4030 (2016). · DOI
  26. Filbin, M. R. et al. Challenges and opportunities for emergency department sepsis screening at triage. Sci. Rep. 8, 11059. https://doi.org/10.1038/s41598-018-29427-1 (2018). · DOI
  27. Rahmatinejad, Z. et al. A comparative study of explainable ensemble learning and logistic regression for predicting in-hospital mortality in the emergency department. Sci. Rep. 14, 3406. https://doi.org/10.1038/s41598-024-54038-4 (2024). · DOI
  28. Zaboli, A. et al. Triage of patients with fever: The Manchester triage system’s predictive validity for sepsis or septic shock and seven-day mortality. J. Crit. Care 59, 63–69. https://doi.org/10.1016/j.jcrc.2020.05.019 (2020). · DOI
  29. Heffner, A. C., Horton, J. M., Marchick, M. R. & Jones, A. E. Etiology of illness in patients with severe sepsis admitted to the hospital from the emergency department. Clin. Infect. Dis. 50, 814–820. https://doi.org/10.1086/650580 (2010). · DOI · Paper page
  30. Klein Klouwenberg, P. M. et al. Likelihood of infection in patients with presumed sepsis at the time of intensive care unit admission: A cohort study. Crit. Care 19, 319. https://doi.org/10.1186/s13054-015-1035-1 (2015). · DOI · Paper page
  31. Molnár, G. et al. Differentiating sepsis from similar groups of symptoms at triage level in emergency care. Physiol. Int. https://doi.org/10.1556/2060.2021.00005 (2021). · DOI
  32. Smyth, M. A. et al. Derivation and internal validation of the screening to enhance prehospital identification of sepsis (SEPSIS) score in adults on arrival at the emergency department. Scand. J. Trauma Resuscitat. Emerg. Med. 27, 67. https://doi.org/10.1186/s13049-019-0642-2 (2019). · DOI
  33. Liu, B. et al. Development and internal validation of a simple prognostic score for early sepsis risk stratification in the emergency department. BMJ Open 11, e046009. https://doi.org/10.1136/bmjopen-2020-046009 (2021). · DOI
  34. Petruniak, L., El-Masri, M. & Fox-Wasylyshyn, S. Exploring the Predictors of Emergency Department Triage Acuity Assignment in Patients With Sepsis. Can. J. Nurs. Res. Revue canadienne de recherche en sciences infirmieres 50, 81–88. https://doi.org/10.1177/0844562118766178 (2018). · DOI
  35. Shibata, J. et al. Risk factors of sepsis among patients with qSOFA<2 in the emergency department. Am. J. Emerg. Med. 50, 699–706. https://doi.org/10.1016/j.ajem.2021.09.035 (2021). · DOI
  36. Luo, Y., Wang, Z. & Wang, C. Improvement of APACHE II score system for disease severity based on XGBoost algorithm. BMC Med. Inform. Decis. Mak. 21, 237. https://doi.org/10.1186/s12911-021-01591-x (2021). · DOI
  37. Olza, A., Millán, E. & Rodríguez-Álvarez, M. X. Development and validation of predictive models for unplanned hospitalization in the Basque Country: Analyzing the variability of non-deterministic algorithms. BMC Med. Inform. Decis. Mak. 23, 152. https://doi.org/10.1186/s12911-023-02226-z (2023). · DOI
  38. Klug, M. et al. A gradient boosting machine learning model for predicting early mortality in the emergency department triage: Devising a nine-point triage score. J. General Intern. Med. 35, 220–227. https://doi.org/10.1007/s11606-019-05512-7 (2020). · DOI
  39. Ali, S. et al. The enlightening role of explainable artificial intelligence in medical & healthcare domains: A systematic literature review. Comput. Biol. Med. 166, 107555. https://doi.org/10.1016/j.compbiomed.2023.107555 (2023). · DOI
  40. Rahmatinejad, Z. et al. Comparison of six scoring systems for predicting in-hospital mortality among patients with SARS-COV2 presenting to the emergency department. Indian J. Crit. Care Med. 27, 416–425. https://doi.org/10.5005/jp-journals-10071-24463 (2023). · DOI

Cited by

Sorted by citations

See all 43

Metadata

Authors
Liu, Z., Shu, W., Li, T., Zhang, X., & Chong, W.
Affiliations
First Hospital of China Medical University (CN)
Corresponding author
Chong, Wei
ORCID
Venue
Scientific Reports
Publisher
Springer Science and Business Media LLC
Published
2025-01-06
Volume
15
Issue
1
Pages
887-887
ISSN
2045-2322 (ISSN-L)
DOI
10.1038/s41598-025-85121-z
PMC
PMC11704257
OpenAlex
W4406081467
Cited by
43
References
4
Citations by year
2026: 31 · 2025: 12
FWCI
29.80
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Education Department of Liaoning Province, China (LJ232410159024)
Topics
  • Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Sepsis Diagnosis and Treatment
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
sepsis prediction, emergency department triage, model interpretability, SHAP, lime, vital signs, electronic health records, gradient boosting, MIMIC-IV, clinical decision support
MeSH terms
Machine Learning, Adult, Aged, Algorithms, Emergency Service, Hospital, Female, Humans, Male, Middle Aged, Retrospective Studies, Triage, Risk Assessment, Sepsis, Vital Signs, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review