Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 91 of the 91 references it cites were found in a registry or an index; 18 of the 28 sources its claims cite were checked for support.

Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing

Open access

Published version of a preprint

Abstract

Machine learning-based clinical decision support tools for sepsis create opportunities to identify at-risk patients and initiate treatments at early time points, which is critical for improving sepsis outcomes. In view of the increasing use of such systems, better understanding of how they are adopted and used by healthcare providers is needed. Here, we analyzed provider interactions with a sepsis early detection tool (Targeted Real-time Early Warning System), which was deployed at five hospitals over a 2-year period. Among 9,805 retrospectively identified sepsis cases, the early detection tool achieved high sensitivity (82% of sepsis cases were identified) and a high rate of adoption: 89% of all alerts by the system were evaluated by a physician or advanced practice provider and 38% of evaluated alerts were confirmed by a provider. Adjusting for patient presentation and severity, patients with sepsis whose alert was confirmed by a provider within 3 h had a 1.85-h (95% CI 1.66-2.00) reduction in median time to first antibiotic order compared to patients with sepsis whose alert was either dismissed, confirmed more than 3 h after the alert or never addressed in the system. Finally, we found that emergency department providers and providers who had previous interactions with an alert were more likely to interact with alerts, as well as to confirm alerts on retrospectively identified patients with sepsis. Beyond efforts to improve the performance of early warning systems, efforts to improve adoption are essential to their clinical impact and should focus on understanding providers' knowledge of, experience with and attitudes toward such systems.

Overview

CoChat Summary

TL;DR

TREWS, a machine-learning sepsis early warning system deployed across five hospitals, achieved high provider adoption and was associated with faster antibiotic ordering for sepsis patients. Provider-related factors, especially emergency department work and prior tool experience, were most strongly associated with alert evaluation.

Key findings

  • TREWS screened 469,419 patient encounters; 9,805 (2.1%) were retrospectively identified as sepsis. The system flagged 31,591 (6.7%) encounters and 8,033 of 9,805 (82%) sepsis cases. Among all alerts, 28,243 (89%) were ever evaluated, 22,982 (73%) within 3 hours, and 10,644 (38%) of evaluated alerts were confirmed; among evaluated sepsis alerts, 5,388 (71%) were confirmed.
  • Timely evaluation entered by a physician was associated with a 1.12-hour reduction in adjusted median time from alert to first antibiotic order (95% CI 0.87–1.30). Timely alert confirmation was associated with a 1.85-hour reduction (95% CI 1.66–2.00).
  • Provider factors had the strongest association with timely evaluation: ED provider adjusted RR 1.22 (95% CI 1.14–1.32) and prior alert interaction adjusted RR 1.22 (95% CI 1.19–1.26). High alert volume in the prior 24 hours was associated with decreased evaluation (adjusted RR 0.94, 95% CI 0.91–0.96), while advanced age >70 was associated with increased evaluation (adjusted RR 1.06, 95% CI 1.03–1.10).
  • Among evaluated sepsis patients with 4+ qualifying antibiotic days, dismissal was more likely with absence of key sepsis symptoms (adjusted RR 1.28, 95% CI 1.06–1.45) and high acute general severity (adjusted RR 1.46, 95% CI 1.28–1.66); ED provider (adjusted RR 0.47, 95% CI 0.40–0.54) and prior provider experience with alerts (adjusted RR 0.66, 95% CI 0.56–0.73) were associated with decreased dismissal.

Methods

  • Observational analysis of real-time provider interactions with TREWS at five Johns Hopkins Health System hospitals (two academic, three community) in Maryland/Washington, D.C.; adult ED or medical/surgical admissions were included, with staggered deployment dates from April 2018 to March 2020.
  • Sepsis was retrospectively identified using EHR-based sepsis phenotyping. Adoption was defined as a provider entering an evaluation in TREWS, considered timely if within 3 hours. Impact and factor analyses were restricted to sepsis patients with an alert, no antibiotic order before the alert, and antibiotics within 24 hours after the alert; dismissal analysis further required 4+ qualifying antibiotic days.
  • Quantile regression estimated adjusted median differences in hours from alert to first antibiotic order. Logistic regression estimated adjusted risk ratios for timely evaluation and alert dismissal, with bootstrap-based 95% CIs; analyses used Python 3.7.6.

Limitations

  • Stated: there is no consensus retrospective sepsis definition; the EHR-based phenotype may still include non-infectious mimics. ICD-10 codes for chronic conditions and alternative diagnoses may introduce coding bias. All hospitals were in one health system. The study captures only interactions recorded within the tool, not sepsis-related discussions or actions outside it. Findings are tied to TREWS’s specific performance, interface, and workflow, and provider background/attitudes and over-prescription/over-reliance were not measured.
  • Evident: the design is observational, so reported associations between adoption and antibiotic timing should not be read as causal effects.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41591-022-01895-z

Cites

91 of 91 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

8 Valid · 1 Not valid · 2 Not checked · 70 Own results and interpretations

53 more claims

Create a free account to see all 81 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 19
  • It may state two separate claims as one. 8

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Show all 85 references

Research works printed but not linked: 3

  1. Khairat, S., Marc, D., Crosby, W. & Al Sanousi, A. Reasons for physicians not adopting clinical decision support systems: critical analysis. JMIR Med. Informatics https://doi.org/10.2196/medinform.8912 (2018). · DOI · Paper page
  2. Subbaswamy, A., Schulam, P. & Saria, S. Preventing failures due to dataset shift: learning predictive models that transport. In Proc. of the International Conference on Artificial Intelligence and Statistics, PMLR 89:3118-3127 (2019).
  3. Henry, K., Wongvibulsin, S., Zhan, A., Saria, S. & Hager, D. Can septic shock be identified early? Evaluating performance of a targeted real-time early warning score (TREW score) for septic shock in a community hospital: global and subpopulation performance. Am. J. Respir. Crit. Care Med. 195, 7016 (2017).

Cited by

See all 144

Metadata

Authors
Henry, K. E., Adams, R., Parent, C., Soleimani, H., Sridharan, A., Johnson, L., Hager, D. N., Cosgrove, S. E., Markowski, A., Klein, E. Y., Chen, E. S., Saheed, M. O., Henley, M., Miranda, S., Houston, K., Linton, R. C., Ahluwalia, A. R., Wu, A. W., & Saria, S.
Affiliations
  • Johns Hopkins University (US)
  • University of California, San Francisco (US)
  • Howard County General Hospital (US)
  • Suburban Hospital (US)
Corresponding author
  • Wu, Albert W.
  • Saria, Suchi
Venue
Nature Medicine
Published
2022-07
Volume
28
Issue
7
Pages
1447-1454
ISSN
  • 1078-8956 (ISSN-L)
  • 1546-170X
OpenAlex
W4286491015
Cited by
144
References
85
Citations by year
2026: 46 · 2025: 35 · 2024: 24 · 2023: 24 · 2022: 15
FWCI
17.63
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Medicine (publishedVersion)
  • medRxiv (submittedVersion, cc-by-nd)
  • PubMed (publishedVersion)
Funders
  • Gordon and Betty Moore Foundation (3186.01)
  • National Science Foundation (1840088)
  • Alfred P. Sloan Foundation
Topics
  • Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Sepsis Diagnosis and Treatment
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
sepsis, early warning systems, clinical decision support, provider adoption, early sepsis detection, timing of antibiotic administration, emergency department staff, alert response, machine learning
MeSH terms
Machine Learning, Humans, Retrospective Studies, Sepsis, Early Diagnosis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-11-03, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-03 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing. https://app.cochat.ai/papers/10.1038/s41591-022-01895-z
The paper
Henry, K. E., Adams, R., Parent, C., Soleimani, H., Sridharan, A., Johnson, L., Hager, D. N., Cosgrove, S. E., Markowski, A., Klein, E. Y., Chen, E. S., Saheed, M. O., Henley, M., Miranda, S., Houston, K., Linton, R. C., Ahluwalia, A. R., Wu, A. W., & Saria, S. (2022). Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing. Nature Medicine. https://doi.org/10.1038/s41591-022-01895-z

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.