The DOI matches its Crossref record; 91 of the 91 references it cites were found in a registry or an index; 18 of the 28 sources its claims cite were checked for support.
Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing
Open accessPublished version of a preprint
Abstract
Machine learning-based clinical decision support tools for sepsis create opportunities to identify at-risk patients and initiate treatments at early time points, which is critical for improving sepsis outcomes. In view of the increasing use of such systems, better understanding of how they are adopted and used by healthcare providers is needed. Here, we analyzed provider interactions with a sepsis early detection tool (Targeted Real-time Early Warning System), which was deployed at five hospitals over a 2-year period. Among 9,805 retrospectively identified sepsis cases, the early detection tool achieved high sensitivity (82% of sepsis cases were identified) and a high rate of adoption: 89% of all alerts by the system were evaluated by a physician or advanced practice provider and 38% of evaluated alerts were confirmed by a provider. Adjusting for patient presentation and severity, patients with sepsis whose alert was confirmed by a provider within 3 h had a 1.85-h (95% CI 1.66-2.00) reduction in median time to first antibiotic order compared to patients with sepsis whose alert was either dismissed, confirmed more than 3 h after the alert or never addressed in the system. Finally, we found that emergency department providers and providers who had previous interactions with an alert were more likely to interact with alerts, as well as to confirm alerts on retrospectively identified patients with sepsis. Beyond efforts to improve the performance of early warning systems, efforts to improve adoption are essential to their clinical impact and should focus on understanding providers' knowledge of, experience with and attitudes toward such systems.
Overview
CoChat Summary
TL;DR
TREWS, a machine-learning sepsis early warning system deployed across five hospitals, achieved high provider adoption and was associated with faster antibiotic ordering for sepsis patients. Provider-related factors, especially emergency department work and prior tool experience, were most strongly associated with alert evaluation.
Key findings
- TREWS screened 469,419 patient encounters; 9,805 (2.1%) were retrospectively identified as sepsis. The system flagged 31,591 (6.7%) encounters and 8,033 of 9,805 (82%) sepsis cases. Among all alerts, 28,243 (89%) were ever evaluated, 22,982 (73%) within 3 hours, and 10,644 (38%) of evaluated alerts were confirmed; among evaluated sepsis alerts, 5,388 (71%) were confirmed.
- Timely evaluation entered by a physician was associated with a 1.12-hour reduction in adjusted median time from alert to first antibiotic order (95% CI 0.87–1.30). Timely alert confirmation was associated with a 1.85-hour reduction (95% CI 1.66–2.00).
- Provider factors had the strongest association with timely evaluation: ED provider adjusted RR 1.22 (95% CI 1.14–1.32) and prior alert interaction adjusted RR 1.22 (95% CI 1.19–1.26). High alert volume in the prior 24 hours was associated with decreased evaluation (adjusted RR 0.94, 95% CI 0.91–0.96), while advanced age >70 was associated with increased evaluation (adjusted RR 1.06, 95% CI 1.03–1.10).
- Among evaluated sepsis patients with 4+ qualifying antibiotic days, dismissal was more likely with absence of key sepsis symptoms (adjusted RR 1.28, 95% CI 1.06–1.45) and high acute general severity (adjusted RR 1.46, 95% CI 1.28–1.66); ED provider (adjusted RR 0.47, 95% CI 0.40–0.54) and prior provider experience with alerts (adjusted RR 0.66, 95% CI 0.56–0.73) were associated with decreased dismissal.
Methods
- Observational analysis of real-time provider interactions with TREWS at five Johns Hopkins Health System hospitals (two academic, three community) in Maryland/Washington, D.C.; adult ED or medical/surgical admissions were included, with staggered deployment dates from April 2018 to March 2020.
- Sepsis was retrospectively identified using EHR-based sepsis phenotyping. Adoption was defined as a provider entering an evaluation in TREWS, considered timely if within 3 hours. Impact and factor analyses were restricted to sepsis patients with an alert, no antibiotic order before the alert, and antibiotics within 24 hours after the alert; dismissal analysis further required 4+ qualifying antibiotic days.
- Quantile regression estimated adjusted median differences in hours from alert to first antibiotic order. Logistic regression estimated adjusted risk ratios for timely evaluation and alert dismissal, with bootstrap-based 95% CIs; analyses used Python 3.7.6.
Limitations
- Stated: there is no consensus retrospective sepsis definition; the EHR-based phenotype may still include non-infectious mimics. ICD-10 codes for chronic conditions and alternative diagnoses may introduce coding bias. All hospitals were in one health system. The study captures only interactions recorded within the tool, not sepsis-related discussions or actions outside it. Findings are tied to TREWS’s specific performance, interface, and workflow, and provider background/attitudes and over-prescription/over-reliance were not measured.
- Evident: the design is observational, so reported associations between adoption and antibiotic timing should not be read as causal effects.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
8 Valid · 1 Not valid · 2 Not checked · 70 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, the simulated environment is unable to capture the full complexity of high-risk decision-making in a live-care delivery setting like the hospital environment, where there are many competing time and attention demands 4,15,16,29 .”
§ Evaluating Adoption, Impact, and Factors Driving Adoption for TREWS, a Machine Learning-Based Sepsis Alerting System > DISCUSSION
What the sources say
- Not checked — no full text of the sourcecites [4] Bauer, M. et al. Automation to optimise physician treatment of individual patients: examples in psychiatry. Lancet Psychiat. 6, 338–349 (2019). source
- Partly supportedcites [15] Celi, L. A., Fine, B. & Stone, D. J. An awakening in medicine: the partnership of humanity and intelligent machines. Lancet Digit. Health 1, e255–e257 (2019). sourceExperimental — accuracy not yet measured
The paper’s main text (page not provided) describes complex clinical decisions and warns that poorly designed AI could worsen information overload and cognitive fatigue in clinical workflows. It does not discuss simulated environments or establish that they fail to capture the competing time and attention demands of live hospital care. (LLM, conf=0.76)
- Not checked — no full text of the sourcecites [16] Mertz, L. From Annoying to appreciated: turning clinical decision support systems into a medical professional’s best friend. IEEE Pulse 6, 4–9 (2015). source
- Partly supportedcites [29] Gaube, S. et al. Do as AI say: susceptibility in deployment of clinical decision-aids. NPJ Digit. Med. https://doi.org/10.1038/s41746-021-00385-9 (2021). sourceExperimental — accuracy not yet measured
The Discussion says the web-based study could not fully capture decision-making risk and might affect behavior differently from a real deployment. It does not describe competing time and attention demands in hospitals, so that added detail is not supported by this source. (LLM, conf=0.87)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Additionally, ease of use and integration into the workflow have been noted as important factors influencing adoption 26,28,47,48 .”
§ Evaluating Adoption, Impact, and Factors Driving Adoption for TREWS, a Machine Learning-Based Sepsis Alerting System > DISCUSSION
What the sources say
- Partly supportedcites [26] Escobar, G. J. et al. Automated identification of adults at risk for in-hospital clinical deterioration. N. Engl. J. Med. 383, 1951–1960 (2020). sourceExperimental — accuracy not yet measured
The Discussion (page unavailable) says that automation and the lack of a required immediate staff response facilitated the system’s incorporation into a rapid-response program built around standardized workflows. This supports the workflow-integration point, but the paper does not explicitly assess ease of use or establish it as a factor influencing adoption. (LLM, conf=0.82)
- Partly supportedcites [28] Cai, C. J. et al. Human-centered tools for coping with imperfect algorithms during medical decision-making. Proc of the CHI Conference on Human Factors in Computing Systems, pp. 1–14 (2019). sourceExperimental — accuracy not yet measured
The User Study Results report that SMILY required less effort than a conventional interface and that participants were more likely to use it in clinical practice. This supports a connection between usability and prospective use, but the paper does not identify integration into clinical workflow as an adoption factor. (LLM, conf=0.83)
- Partly supportedcites [47] Shimabukuro, D. W., Barton, C. W., Feldman, M. D., Mataraso, S. J. & Das, R. Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial. BMJ Open Respir. Res. https://doi.org/10.1136/bmjresp-2017-000234 (2017). sourceExperimental — accuracy not yet measured
The Discussion (page not available in our copy) says false alarms could affect clinicians’ willingness to use a sepsis tool and suggests that an algorithm requiring only vital signs and age may have contributed to improved outcomes. This supports the general point that practical usability can matter for use, but the paper does not identify ease of use or workflow integration as factors influencing adoption. (LLM, conf=0.77)
- Partly supportedcites [48] Wong, A. et al. External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Intern. Med. 48109, 1–6 (2021). sourceExperimental — accuracy not yet measured
The Conclusions (page not specified) identify ease of EHR integration as one reason hundreds of US hospitals began using these algorithms. The Introduction (page not specified) says EHR integration makes alerts accessible to clinicians and linkable to interventions. The paper does not identify ease of use as a separate factor influencing adoption. (LLM, conf=0.86)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Past deployed systems have reported significantly lower predictive performance on similar hospital populations 20,42,45 .”
§ Evaluating Adoption, Impact, and Factors Driving Adoption for TREWS, a Machine Learning-Based Sepsis Alerting System > DISCUSSION
What the sources say
- Partly supportedcites [20] Downing, N. L. et al. Electronic health record-based clinical decision support alert for severe sepsis: a randomised evaluation. BMJ Qual. Saf. 28, 762–768 (2019). sourceExperimental — accuracy not yet measured
The Discussion reports a 61% positive predictive value (PPV) for this deployed sepsis alert and cites a review finding PPVs of 20.5%–53.8% for earlier alerts. It does not establish that those systems served similar hospital populations or that the performance differences were statistically significant; it describes its PPV as equal or superior to most prior studies. (LLM, conf=0.85)
- Not checked — no full text of the sourcecites [42] Adams, R. et al. 1405: assessing clinical use and performance of a machine learning sepsis alert for sex and racial bias. Crit. Care Med. 50, 705 (2022). source
- Not checked — no full text of the sourcecites [45] Perlin, J. B. et al. 2019 John M. Eisenberg patient safety and quality awards: SPOTting sepsis to save lives: a nationwide computer algorithm for early detection of sepsis: innovation in patient safety and quality at the national level (Eisenberg award). Jt. Comm. J. Qual. Patient Saf. 46, 381–391 (2020). source
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“In order to better understand factors impacting the adoption of CDS, previous studies of deployed systems have used post-hoc surveys or interviews to gauge provider impressions 28,44,53 ; however, provider impressions can differ from their actual use 4,18,27 .”
§ Evaluating Adoption, Impact, and Factors Driving Adoption for TREWS, a Machine Learning-Based Sepsis Alerting System > DISCUSSION
What the sources say
- Not checked — no full text of the sourcecites [4] Bauer, M. et al. Automation to optimise physician treatment of individual patients: examples in psychiatry. Lancet Psychiat. 6, 338–349 (2019). source
- Could not verifycites [18] Wright, A. et al. Analysis of clinical decision support system malfunctions: a case series and survey. J. Am. Med. Inform. Assoc. 23, 1068–1076 (2016). sourceExperimental — accuracy not yet measured
The Discussion (Key Findings; page unavailable) says user reports can miss CDS alert malfunctions, but it does not compare providers’ impressions of CDS with their actual use of it. A search for perceived versus measured CDS use found no passage addressing that comparison. (LLM, conf=0.75)
- Not checked — no full text of the sourcecites [27] Schaefer, K. E., Chen, J. Y. C., Szalma, J. L. & Hancock, P. A. A meta-analysis of factors influencing the development of trust in automation: implications for understanding autonomy in future systems. Hum. Factors 58, 377–400 (2016). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In order to better understand factors impacting the adoption of CDS, previous studies of deployed systems have used post-hoc surveys or interviews to gauge provider impressions 28,44,53 ; however, provider impressions can differ from their actual use 4,18,27 .”
§ Evaluating Adoption, Impact, and Factors Driving Adoption for TREWS, a Machine Learning-Based Sepsis Alerting System > DISCUSSION
What the sources say
- Partly supportedcites [28] Cai, C. J. et al. Human-centered tools for coping with imperfect algorithms during medical decision-making. Proc of the CHI Conference on Human Factors in Computing Systems, pp. 1–14 (2019). sourceExperimental — accuracy not yet measured
The Method section reports that pathologists completed a post-study questionnaire and semi-structured interview, and User Study Results reports their impressions of SMILY. But the Method describes a controlled study using six images, not a study of a deployed clinical system; that qualifier is unsupported. (LLM, conf=0.86)
- Not checked — no full text of the sourcecites [44] Guy, J. S., Jackson, E. & Perlin, J. B. Accelerating the clinical workflow using the sepsis prediction and optimization of therapy (SPOT) tool for real-time clinical monitoring. NEJM Catal. Innov. Care Deliv. https://doi.org/10.1056/CAT.19.1036 (2020). source
- Supportedcites [53] Ginestra, J. C. et al. Clinician perception of a machine learning–based early warning system designed to predict severe sepsis and septic shock. Crit. Care Med. https://doi.org/10.1097/ccm.0000000000003803 (2019). source
The Survey Deployment and Administration section describes questionnaires assessing clinicians’ perceptions of an early warning system deployed five months earlier. Clinicians completed surveys within six hours of an alert and, for some, again after 48 hours, making this a study of a deployed system that used surveys to gauge provider impressions. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
53 more claims
Create a free account to see all 81 claims and every source verdict.
- Probably not a substantial claim. 19
- It may state two separate claims as one. 8
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While the overall performance (eg, AUROC) of the new model was generally lower than that reported in other frontier sepsis prediction models, such as COMPOSER,18 Targeted Real-time Early Warning System,19 and SepsisWatch,20”
From Multicenter Prospective Validation of an Updated Proprietary Sepsis Prediction Model (2026) · cites this paper as [19]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“In sepsis detection, AI agents like those deployed at Johns Hopkins Hospital continuously monitor patient vital signs and laboratory values, alerting clinicians to early sepsis indicators with promising sensitivity rates and potential for reducing mortality rates.71”
From A foundational architecture for AI agents in healthcare (2025) · cites this paper as [71]
The cited source says what the claim attributes to it.
In the citing paper
“Successful implementation requires an understanding of intended users, their workflows and resources, and a vision of how these should change based on CDSS output [47, 48].”
From Use of artificial intelligence in critical care: opportunities and obstacles (2024) · cites this paper as [47]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Evaluating Adoption, Impact, and Factors Driving Adoption for TREWS, a Machine Learning-Based Sepsis Alerting System > Study question 3.1: Factors associated with alert adoption
“Alert occurrences between 3pm -11pm and 11pm -3am and high admission volume were not significantly associated with adoption.”
§ Evaluating Adoption, Impact, and Factors Driving Adoption for TREWS, a Machine Learning-Based Sepsis Alerting System > Alert Occurred 11pm-7am
“True if alert occurs between 11pm and 7am”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“a precision of 2.4% (1 in 46 alerts within 24 hours of sepsis onset)”
§ Evaluating Adoption, Impact, and Factors Driving Adoption for TREWS, a Machine Learning-Based Sepsis Alerting System > DISCUSSION
Reported 2.4 · recomputed 2.173913
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Evaluating Adoption, Impact, and Factors Driving Adoption for TREWS, a Machine Learning-Based Sepsis Alerting System > DISCUSSION
“The lack of an association between patient factors and the likelihood of a provider entering a timely evaluation could be viewed as promising.”
§ Evaluating Adoption, Impact, and Factors Driving Adoption for TREWS, a Machine Learning-Based Sepsis Alerting System > Study question 3.1: Factors associated with alert adoption
“Among patient factors, advanced age (>70 years) was significantly associated with an increased likelihood of entering a timely evaluation; the remaining patient factors were not significantly associated (Table 4).”
Citations
Cites
The third international consensus definitions for sepsis and septic shock (Sepsis-3).
Singer, M., Deutschman, C. S., Seymour, C. W., Shankar-Hari, M., Annane, D., Bauer, M., Bellomo, R., Bernard, G. R., Chiche, J.-D., Coopersmith, C. M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0287 · Cited by 28,250 · cited 3×
“The 3-h window was chosen to match the treatment window recommended by the Centers for Medicare and Medicaid Services (CMS) sepsis core measure (SEP-1) and the Surviving Sepsis Campaign guidelines 38–40 .”
BackgroundMethodAPACHE II: A severity of disease classification system
KNAUS, W. A., DRAPER, E. A., WAGNER, D. P., & ZIMMERMAN, J. E. (1985) · Critical Care Medicine · doi:10.1097/00003246-198510000-00009 · Cited by 13,664
Vincent, J. L., Moreno, R., Takala, J., Willatts, S., De Mendonça, A., Bruining, H., Reinhart, C. K., Suter, P. M., & Thijs, L. G. (1996) · Intensive Care Medicine · doi:10.1007/bf01709751 · Cited by 11,674 · cited 1×
“We also adjusted for acute severity based on individual sepsis-related organ failure assessment score components and APACHE II score 76,77 , as well as several sepsis-relevant laboratory measurements, vital signs and treatments, including systolic blood pressure, altered mental status indicated by…”
MethodHigh-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450 · cited 1×
“…Henley 7 , Sheila Miranda 11 , Katrina Houston 7 , Robert C. Linton II 6 , Anushree R. Ahluwalia 7 , Albert W. Wu 8,12,13,14 ✉ and Suchi Saria 1,2,8,12,15 ✉ Machine learning-based clinical decision support tools for sepsis create opportunities to identify at-risk patients and initiate…”
BackgroundTrust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 1×
“Some providers may be more willing to adopt new CDS tools than others; this tendency is sometimes referred to as ‘dispositional trust’ 58,59 .”
BackgroundStatsmodels: Econometric and Statistical Modeling with Python
Seabold, S., & Perktold, J. (2010) · Proceedings of the Python in Science Conference · doi:10.25080/majora-92bf1922-011 · Cited by 6,465 · cited 1×
“Adjusted median time to first antibiotic order (study question 2) was estimated using quantile regression as implemented in the statsmodels Python package (v.0.12.2) 79 .”
MethodInfluentialKumar, A., Roberts, D., Wood, K. E., Light, B., Parrillo, J. E., Sharma, S., Suppes, R., Feinstein, D., Zanotti, S., Taiberg, L., et al. (2006) · Critical Care Medicine · doi:10.1097/01.ccm.0000217961.75225.e9 · Cited by 6,109 · cited 1×
“Early recognition of sepsis is critical for successful treatment and, in particular, early administration of antibiotics is associated with decreased mortality 32–34 .”
BackgroundAPACHE II: a severity of disease classification system.
Knaus, W. A., Draper, E. S., Wagner, D. P., & Zimmerman, J. E. (1985) · PubMed · Cited by 6,034
Seymour, C. W., Liu, V. X., Iwashyna, T. J., Brunkhorst, F. M., Rea, T. D., Scherag, A., Rubenfeld, G., Kahn, J. M., Shankar-Hari, M., Singer, M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0288 · Cited by 3,939
Key challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136
Research works printed but not linked: 3
- Khairat, S., Marc, D., Crosby, W. & Al Sanousi, A. Reasons for physicians not adopting clinical decision support systems: critical analysis. JMIR Med. Informatics https://doi.org/10.2196/medinform.8912 (2018). · DOI · Paper page
- Subbaswamy, A., Schulam, P. & Saria, S. Preventing failures due to dataset shift: learning predictive models that transport. In Proc. of the International Conference on Artificial Intelligence and Statistics, PMLR 89:3118-3127 (2019).
- Henry, K., Wongvibulsin, S., Zhan, A., Saria, S. & Hager, D. Can septic shock be identified early? Evaluating performance of a targeted real-time early warning score (TREW score) for septic shock in a community hospital: global and subpopulation performance. Am. J. Respir. Crit. Care Med. 195, 7016 (2017).
Cited by
Adams, R., Henry, K. E., Sridharan, A., Soleimani, H., Zhan, A., Rawat, N., Johnson, L., Hager, D. N., Cosgrove, S. E., Markowski, A., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01894-0 · Cited by 365
“Notably, we found that sepsis patients who had their alert evaluated and confirmed within 3 h had a 1.85-h lower median time from alert to first antibiotic order 22 .”
BackgroundMethodResultInfluentialSelf-citationArmoundas, A. A., Narayan, S. M., Arnett, D. K., Spector-Bagdady, K., Bennett, D. A., Celi, L. A., Friedman, P. A., Gollob, M. H., Hall, J. L., Kwitek, A. E., et al. (2024) · Circulation · doi:10.1161/cir.0000000000001201 · Cited by 347
“Moreover, few studies have been shown to affect clinical end points such as mortality, 88–90 or make predictions that could directly inform clinical decision-making.”
BackgroundHenry, K. E., Kornfield, R., Sridharan, A., Linton, R. C., Groh, C., Wang, T., Wu, A., Mutlu, B., & Saria, S. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00597-7 · Cited by 207
Self-citationUse of artificial intelligence in critical care: opportunities and obstacles
Pinsky, M. R., Bedoya, A., Bihorac, A., Celi, L., Churpek, M., Economou-Zavlanos, N. J., Elbers, P., Saria, S., Liu, V., Lyons, P. G., et al. (2024) · Critical Care · doi:10.1186/s13054-024-04860-z · Cited by 160
“Successful implementation requires an understanding of intended users, their workflows and resources, and a vision of how these should change based on CDSS output [47, 48].”
BackgroundSelf-citationA Nationwide Network of Health AI Assurance Laboratories
Shah, N. H., Halamka, J. D., Saria, S., Pencina, M., Tazbaz, T., Tripathi, M., Callahan, A., Hildahl, H., & Anderson, B. (2024) · JAMA · doi:10.1001/jama.2023.26930 · Cited by 114
Self-citation
Metadata
- Authors
- Henry, K. E., Adams, R., Parent, C., Soleimani, H., Sridharan, A., Johnson, L., Hager, D. N., Cosgrove, S. E., Markowski, A., Klein, E. Y., Chen, E. S., Saheed, M. O., Henley, M., Miranda, S., Houston, K., Linton, R. C., Ahluwalia, A. R., Wu, A. W., & Saria, S.
- Affiliations
- Johns Hopkins University (US)
- University of California, San Francisco (US)
- Howard County General Hospital (US)
- Suburban Hospital (US)
- Corresponding author
- Wu, Albert W.
- Saria, Suchi
- ORCID
- Venue
- Nature Medicine
- Published
- 2022-07
- Volume
- 28
- Issue
- 7
- Pages
- 1447-1454
- ISSN
- 1078-8956 (ISSN-L)
- 1546-170X
- OpenAlex
- W4286491015
- Semantic Scholar
- 4cf1afc1e27d26a77aca58d7a5ec7fe3d6b7ffad
- Cited by
- 144
- References
- 85
- Citations by year
- 2026: 46 · 2025: 35 · 2024: 24 · 2023: 24 · 2022: 15
- FWCI
- 17.63
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- cc-by-nd
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Nature Medicine (publishedVersion)
- medRxiv (submittedVersion, cc-by-nd)
- PubMed (publishedVersion)
- Funders
- Gordon and Betty Moore Foundation (3186.01)
- National Science Foundation (1840088)
- Alfred P. Sloan Foundation
- Topics
- Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Sepsis Diagnosis and Treatment
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- sepsis, early warning systems, clinical decision support, provider adoption, early sepsis detection, timing of antibiotic administration, emergency department staff, alert response, machine learning
- MeSH terms
- Machine Learning, Humans, Retrospective Studies, Sepsis, Early Diagnosis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-11-03, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-03 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing. https://app.cochat.ai/papers/10.1038/s41591-022-01895-z
- The paper
- Henry, K. E., Adams, R., Parent, C., Soleimani, H., Sridharan, A., Johnson, L., Hager, D. N., Cosgrove, S. E., Markowski, A., Klein, E. Y., Chen, E. S., Saheed, M. O., Henley, M., Miranda, S., Houston, K., Linton, R. C., Ahluwalia, A. R., Wu, A. W., & Saria, S. (2022). Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing. Nature Medicine. https://doi.org/10.1038/s41591-022-01895-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.