Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index; 40 of the 55 sources its claims cite were checked for support.

Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy

Open access

Abstract

PURPOSE: Early clinical recognition of sepsis can be challenging. With the advancement of machine learning, promising real-time models to predict sepsis have emerged. We assessed their performance by carrying out a systematic review and meta-analysis. METHODS: A systematic search was performed in PubMed, Embase.com and Scopus. Studies targeting sepsis, severe sepsis or septic shock in any hospital setting were eligible for inclusion. The index test was any supervised machine learning model for real-time prediction of these conditions. Quality of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology, with a tailored Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) checklist to evaluate risk of bias. Models with a reported area under the curve of the receiver operating characteristic (AUROC) metric were meta-analyzed to identify strongest contributors to model performance. RESULTS: After screening, a total of 28 papers were eligible for synthesis, from which 130 models were extracted. The majority of papers were developed in the intensive care unit (ICU, n = 15; 54%), followed by hospital wards (n = 7; 25%), the emergency department (ED, n = 4; 14%) and all of these settings (n = 2; 7%). For the prediction of sepsis, diagnostic test accuracy assessed by the AUROC ranged from 0.68-0.99 in the ICU, to 0.96-0.98 in-hospital and 0.87 to 0.97 in the ED. Varying sepsis definitions limit pooling of the performance across studies. Only three papers clinically implemented models with mixed results. In the multivariate analysis, temperature, lab values, and model type contributed most to model performance. CONCLUSION: This systematic review and meta-analysis show that on retrospective data, individual machine learning models can accurately predict sepsis onset ahead of time. Although they present alternatives to traditional scoring systems, between-study heterogeneity limits the assessment of pooled results. Systematic reporting and clinical implementation studies are needed to bridge the gap between bytes and bedside.

Overview

CoChat Summary

TL;DR

This systematic review and meta-analysis of 28 papers and 130 machine learning models found that models retrospectively predict sepsis with AUROC ranges of 0.68–0.99 in the ICU, 0.96–0.98 in-hospital, and 0.87–0.97 in the ED. Between-study heterogeneity, varying sepsis definitions, and scarce prospective/clinical implementation studies limit pooled assessment and evidence on patient outcomes.

Key findings

  • From 28 eligible papers, 130 models were extracted; most studies were in the ICU (n = 15; 54%), followed by hospital wards (n = 7; 25%), ED (n = 4; 14%), and all settings (n = 2; 7%). Reported AUROCs for sepsis ranged from 0.68–0.99 in the ICU, 0.96–0.98 in-hospital, and 0.87–0.97 in the ED; septic shock predictions ranged from 0.86–0.94 in-hospital and 0.83–0.96 in the ICU.
  • All studies reported retrospective diagnostic test accuracy; models were prospectively validated in 2 papers (7%) and clinically implemented in 3 papers (11%), with mixed results.
  • In the meta-analysis of 111 models, temperature (multivariate coefficient 0.812, p = 0.000), lab values (0.842, p = 0.003), and model type (p = 0.020) contributed most to model performance. Univariate positive contributors included heart rate, respiratory rate, temperature, lab values, arterial blood gas values, and neural networks; negative contributors included the Seymour et al. Sepsis-3 definition, SOFA in the target condition definition, and models other than ensemble methods or neural networks.
  • Risk of bias was unclear in 9/28 papers (32%) and high in all others; all papers had unclear risk of bias for the reference standard, and GRADE quality of evidence was low for each hospital setting.

Methods

  • Systematic review and meta-analysis conducted per PRISMA-DTA, registered on PROSPERO (CRD42019118716). Searches were run in PubMed, Embase.com, and Scopus up to September 13, 2019, for adult patients in any hospital setting, targeting right-aligned supervised machine learning models predicting sepsis, severe sepsis, or septic shock.
  • After screening 2,684 papers, 130 full texts were read and 28 papers met inclusion criteria; 130 models were retrieved. Quality of evidence used GRADE, and risk of bias used a tailored QUADAS-2 checklist combined with an adapted Joanna Briggs Institute checklist.
  • Meta-analysis used AUROC as the main metric, logit-transformed; a linear random effect model with a paper-specific random effect accounted for multiple models per paper. After grouping features and removing low-variance or collinear covariates, 39 covariates were analyzed, with backward AIC selection for the multivariate model; analyses were carried out in R.

Limitations

  • Paper-stated: AUROC was chosen pragmatically and may underperform with imbalanced datasets; no contingency tables were feasible because necessary data were infrequently reported; few papers reported confidence intervals or standard deviations; heterogeneity in sepsis definitions, settings, and models prevented computing an overall pooled point estimate; there is no gold standard for diagnosing sepsis; high risk of bias and low GRADE evidence.
  • Paper-stated: all models were developed in the United States, many used the MIMIC database, and no models were developed or validated in middle- or low-income countries; code was not released by any study, and only one paper reported adhering to a reporting standard.
  • Evident from the text: only 3 interventional/implementation studies and 2 prospective validation studies were identified, so clinical impact on patient-relevant outcomes remains uncertain; the review is dominated by retrospective diagnostic accuracy studies.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s00134-019-05872-y

Cites

68 of 68 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Left versus right alignment. Left alignment (top) versus right alignment (bottom). Cases are aligned at the alignment point, in the feature window data are collected, the prediction window is the time of the prediction ahead of sepsis onset. Red sepsis cases, green non-septic cases
Fig. 1 Left versus right alignment. Left alignment (top) versus right alignment (bottom). Cases are aligned at the alignment point, in the feature window data are collected, the prediction window is the time of the prediction ahead of sepsis onset. Red sepsis cases, green non-septic casesCC BY-NC · PubMed Central
Fig. 2 Flow diagram. Papers identified in databases, title/abstract screened, read full text, and included in the synthesis. Reasons for exclusion are listed
Fig. 2 Flow diagram. Papers identified in databases, title/abstract screened, read full text, and included in the synthesis. Reasons for exclusion are listedCC BY-NC · PubMed Central
Fig. 3 Prospective versus retrospective models. Percentages specified per paper and for all models
Fig. 3 Prospective versus retrospective models. Percentages specified per paper and for all modelsCC BY-NC · PubMed Central
Fig. 4 Overview of retrospective diagnostic test accuracy studies. Papers are binned per hospital setting, data are sorted in ascending order of AUROC values. AUROC ranges are displayed per paper. AUROC area under the curve of the receiver operating characteristic, SVM support vector machines, GLM generalized linear model, NB Naive Bayes, EM ensemble methods, NNM neural network model, DT decision trees, PHM proportional hazards model, LSTM long short term memory, Hrs bef. onset hours before onset * DT, EM, GLM, LSTM, NB, NNM, SVM
Fig. 4 Overview of retrospective diagnostic test accuracy studies. Papers are binned per hospital setting, data are sorted in ascending order of AUROC values. AUROC ranges are displayed per paper. AUROC area under the curve of the receiver operating characteristic, SVM support vector machines, GLM generalized linear model, NB Naive Bayes, EM ensemble methods, NNM neural network model, DT decision trees, PHM proportional hazards model, LSTM long short term memory, Hrs bef. onset hours before onset * DT, EM, GLM, LSTM, NB, NNM, SVMCC BY-NC · PubMed Central
Fig. 5 Features used in the papers. Features are grouped by type. ESR erythrocyte sedimentation rate, HR heart rate, MAP mean arterial pressure
Fig. 5 Features used in the papers. Features are grouped by type. ESR erythrocyte sedimentation rate, HR heart rate, MAP mean arterial pressureCC BY-NC · PubMed Central
Fig. 6 Relative effect of hours before sepsis onset on AUROC for different models. Expected change in AUROC for three models at different prediction windows (hours before sepsis onset)
Fig. 6 Relative effect of hours before sepsis onset on AUROC for different models. Expected change in AUROC for three models at different prediction windows (hours before sepsis onset)CC BY-NC · PubMed Central

Verification

Claims the paper makes

18 Valid · 6 Not valid · 7 Not checked · 73 Own results and interpretations

45 more claims · 1 with their cited sources checked

Create a free account to see all 104 claims and to verify the 2 references still open.

See all claims — free account
  • Probably not a substantial claim. 31
  • It may state two separate claims as one. 31

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Show all 68 references

Research works printed but not linked: 3

  1. Brown SM, Jones J, Kuttler KG et al (2016) Prospective evaluation of an automated method to identify patients with severe sepsis or septic shock in the emergency department. BMC Emerg Med 16:31. https://doi.org/10.1186/s12873-016-0095-0 · DOI · Paper page
  2. Giannini HM, Ginestra JC, Chivers C et al (2019) A machine learning algorithm to predict severe sepsis and septic shock: development, implementation, and impact on clinical practice. Crit Care Med. https://doi.org/10.1097/CCM.0000000000003891 · DOI · Paper page
  3. Mao Q, Jay M, Hoffman JL et al (2018) Multicentre validation of a sepsis prediction algorithm using only vital sign data in the emergency department, general ward and ICU. BMJ Open 8:e017833. https://doi.org/10.1136/bmjopen-2017-017833 · DOI · Paper page

Cited by

See all 755

Metadata

Authors
Fleuren, L. M., Klausch, T. L. T., Zwager, C. L., Schoonmade, L. J., Guo, T., Roggeveen, L. F., Swart, E. L., Girbes, A. R. J., Thoral, P., Ercole, A., Hoogendoorn, M., & Elbers, P. W. G.
Affiliations
  • Amsterdam Cardiovascular Sciences (NL)
  • Vrije Universiteit Amsterdam (NL)
  • University of Cambridge (GB)
  • European Society of Intensive Care Medicine (BE)
Corresponding author
Fleuren, Lucas M.
Venue
Intensive Care Medicine
Published
2020-01-21
Volume
46
Issue
3
Pages
383-400
ISSN
  • 0342-4642 (ISSN-L)
  • 1432-1238
OpenAlex
W2992764683
Cited by
755
References
68
Citations by year
2026: 142 · 2025: 142 · 2024: 130 · 2023: 124 · 2022: 118 · 2021: 75 · 2020: 24
FWCI
12.43
Type
journal article
Language
en
Open access
Licence
Locations
  • Intensive Care Medicine (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • Pure Amsterdam UMC (publishedVersion, cc-by)
  • Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion, other-oa)
  • VU Research Portal (publishedVersion, other-oa)
  • Apollo (University of Cambridge) (submittedVersion)
  • Apollo (University of Cambridge) (submittedVersion, cc-by-nc)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
  • Apollo
Topics
  • Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Sepsis Diagnosis and Treatment
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
sepsis prediction, machine learning, diagnostic accuracy, septic shock, severe sepsis, area under the curve, intensive care unit, emergency department, sepsis definitions, clinical implementation
MeSH terms
Machine Learning, Diagnostic Tests, Routine, Humans, Retrospective Studies, Shock, Septic, Sepsis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-23, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-23 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy. https://app.cochat.ai/papers/10.1007/s00134-019-05872-y
The paper
Fleuren, L. M., Klausch, T. L. T., Zwager, C. L., Schoonmade, L. J., Guo, T., Roggeveen, L. F., Swart, E. L., Girbes, A. R. J., Thoral, P., Ercole, A., Hoogendoorn, M., & Elbers, P. W. G. (2020). Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy. Intensive Care Medicine. https://doi.org/10.1007/s00134-019-05872-y

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.