The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Early prediction of circulatory failure in the intensive care unit using machine learning
Abstract
Intensive-care clinicians are presented with large quantities of measurements from multiple monitoring systems. The limited ability of humans to process complex information hinders early recognition of patient deterioration, and high numbers of monitoring alarms lead to alarm fatigue. We used machine learning to develop an early-warning system that integrates measurements from multiple organ systems using a high-resolution database with 240 patient-years of data. It predicts 90% of circulatory-failure events in the test set, with 82% identified more than 2 h in advance, resulting in an area under the receiver operating characteristic curve of 0.94 and an area under the precision-recall curve of 0.63. On average, the system raises 0.05 alarms per patient and hour. The model was externally validated in an independent patient cohort. Our model provides early identification of patients at risk for circulatory failure with a much lower false-alarm rate than conventional threshold-based systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Prognostic and risk stratification systems estimate outcomes such as mortality, deterioration, readmission, or complications, supporting early intervention and clinical prioritisation [15,33].”
From Artificial Intelligence in Clinical Decision-Making: A Comprehensive Review of Diagnostic, Prognostic, and Therapeutic Applications, Validation Gaps, Bias, and Deployment Challenges (2026) · cites this paper as [15]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The HiRID database contains high-resolution data for almost 34,000 admissions between 2008–2016 at Bern University Hospital in Switzerland6,7.”
From MIMIC-IV, a freely accessible electronic health record dataset (2023) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“HiRID contains 712 routinely collected physiological variables with one data entry every two minutes.”
From MIMIC-IV, a freely accessible electronic health record dataset (2023) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In clinical workflows, AI supports triage prioritisation, second-opinion review, and reduction of diagnostic variability in high-volume settings [15,27].”
From Artificial Intelligence in Clinical Decision-Making: A Comprehensive Review of Diagnostic, Prognostic, and Therapeutic Applications, Validation Gaps, Bias, and Deployment Challenges (2026) · cites this paper as [15]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These systems may improve triage prioritisation, reduce waiting times, and support resource allocation [9,15].”
From Artificial Intelligence in Clinical Decision-Making: A Comprehensive Review of Diagnostic, Prognostic, and Therapeutic Applications, Validation Gaps, Bias, and Deployment Challenges (2026) · cites this paper as [15]
Experimental — accuracy not yet measured
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Early goal-directed therapy in the treatment of severe sepsis and septic shock
Rivers, E., Nguyen, B., Havstad, S., Ressler, J., Muzzin, A., Knoblich, B., Peterson, E., & Tomlanovich, M. (2001) · New England Journal of Medicine · doi:10.1056/nejmoa010307 · Cited by 10,743
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141 · cited 1×
“The standard deviation of the Gaussian noise model for each variable is the median of the s.d. of the corresponding measurement values during each patient stay for all patients.”
BackgroundHannun, A. Y., Rajpurkar, P., Haghpanahi, M., Tison, G. H., Bourn, C., Turakhia, M. P., & Ng, A. Y. (2019) · Nature Medicine · doi:10.1038/s41591-018-0268-3 · Cited by 3,148
Scalable and accurate deep learning with electronic health records
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0029-1 · Cited by 2,648 · cited 1×
“This finding reflects recent observations 30 that careful feature design, combined with state-of-the-art ML approaches, can outperform deep learning.”
BackgroundAnalysis of Financial Time Series
(2006) · Technometrics · doi:10.1198/tech.2006.s405 · Cited by 2,348
Scalable and accurate deep learning with electronic health records.
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · eScholarship (California Digital Library) · Cited by 2,242
Time to treatment and mortality during mandated emergency care for sepsis.
Seymour, C. W., Gesten, F., Prescott, H. C., Friedrich, M. E., Iwashyna, T. J., Phillips, G. S., Lemeshow, S., Osborn, T., Terry, K. M., & Levy, M. M. (2017) · New England Journal of Medicine · doi:10.1056/nejmoa1703058 · Cited by 2,225 · cited 1×
“This finding is consistent with other clinical trials that indicate that repeated or prolonged episodes of hypotension and high dose vasopressors 11,53 , as well as delays in shock treatment 9,54–58 , are associated with higher mortality.”
ResultPollard, T. J., Johnson, A. E. W., Raffa, J. D., Celi, L. A., Mark, R. G., & Badawi, O. (2018) · Scientific Data · doi:10.1038/sdata.2018.178 · Cited by 1,993
Research works printed but not linked: 11
- Wright, M. C. et al. Toward designing information display to support critical care. Appl. Clin. Inform. 07, 912–929 (2017).
- Dietterich, T. G. in Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition (SPR) and Structural and Syntactic Pattern Recognition (SSPR) vol. 2396 15–30 (Springer, 2002).
- Xu, Y., Biswal, S., Deshpande, S. R., Maher, K. O. & Sun, J. in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining 2565–2573 (ACM, 2018).
- Yoon, J., Alaa, A., Hu, S. & Schaar, M. in Proceedings of The 33rd International Conference on Machine Learning (eds. Balcan, M. F. & Weinberger, K. Q.) 1680–1689 (Proceedings of Machine Learning Research, 2016).
- Ren, O. et al. in 2018 IEEE International Conference on Healthcare Informatics (ICHI) 144–151 (IEEE, 2018).
- Lundberg, S. M. & Lee, S.-I. in Advances in Neural Information Processing Systems 30 (eds. Guyon, I. et al.) 4765–4774 (Curran Associates, Inc., 2017).
- Ke, G. et al. in Advances in Neural Information Processing Systems 30 (eds. Guyon, I. et al.) 3146–3154 (Curran Associates, Inc., 2017).
- Davis, J. & Goadrich, M. in Proceedings of the 23rd International Conference on Machine Learning 233–240 (ACM, 2006).
- Vairavan, S., Eshelman, L., Haider, S., Flower, A. & Seiver, A. Prediction of mortality in an intensive care unit using logistic regression and a hidden Markov model. Comput. Cardiol. 39, 393–396 (2012).
- Choi, E. et al. in Advances in Neural Information Processing Systems 29 (eds. Lee, D. D., Sugiyama, M., Luxburg, U. V., Guyon, I. & Garnett, R.) 3504–3512 (Curran Associates, Inc., 2016).
- Ye, L. & Keogh, E. in Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 947–956 (ACM, 2009).
Cited by
MIMIC-IV, a freely accessible electronic health record dataset
Johnson, A. E. W., Bulgarelli, L., Shen, L., Gayles, A., Shammout, A., Horng, S., Pollard, T. J., Hao, S., Moody, B., Gow, B., et al. (2023) · Scientific Data · doi:10.1038/s41597-022-01899-x · Cited by 3,401
“The HiRID database contains high-resolution data for almost 34,000 admissions between 2008–2016 at Bern University Hospital in Switzerland 6,7 .”
BackgroundA deep learning algorithm to predict risk of pancreatic cancer from disease trajectories
Placido, D., Yuan, B., Hjaltelin, J. X., Zheng, C., Haue, A. D., Chmura, P. J., Yuan, C., Kim, J., Umeton, R., Antell, G., et al. (2023) · Nature Medicine · doi:10.1038/s41591-023-02332-5 · Cited by 349
Machine learning model to predict mental health crises from electronic health records
Garriga, R., Mas, J., Abraha, S., Nolan, J., Harrison, O., Tadros, G., & Matic, A. (2022) · Nature Medicine · doi:10.1038/s41591-022-01811-5 · Cited by 229
“Running predictions continuously 13,14 provides an updated risk score based on the latest available data, which typically contains the most predictive information, which is, in the case of mental health, crucial to improving healthcare management and outcomes.”
BackgroundHolzinger, A., Dehmer, M., Emmert-Streib, F., Cucchiara, R., Augenstein, I., Ser, J. D., Samek, W., Jurisica, I., & Díaz-Rodríguez, N. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.10.007 · Cited by 213
“Properly using such systems would reduce false negatives and false positives, increase quality of treatments and quality of life of patients, decrease the overall cost and free-up clinicians’ time, which becomes more critical as decision-making situations become more patient-centered [144].”
BackgroundSafat, W., Asghar, S., & Gillani, S. A. (2021) · IEEE Access · doi:10.1109/access.2021.3078117 · Cited by 195
“From electronic health records to cybersecurity, almost all real-world applications require time-series data for classification [47].”
Background
Metadata
- Authors
- Hyland, S. L., Faltys, M., Hüser, M., Lyu, X., Gumbsch, T., Esteban, C., Bock, C., Horn, M., Moor, M., Rieck, B., Zimmermann, M., Bodenham, D., Borgwardt, K., Rätsch, G., & Merz, T. M.
- Affiliations
- Memorial Sloan Kettering Cancer Center (US)
- University of Bern (CH)
- ETH Zurich (CH)
- SIB Swiss Institute of Bioinformatics (CH)
- Corresponding author
- Borgwardt, Karsten
- Rätsch, Gunnar
- Merz, Tobias M.
- ORCID
- Venue
- Nature Medicine
- Published
- 2020-03
- Volume
- 26
- Issue
- 3
- Pages
- 364-373
- ISSN
- 1078-8956 (ISSN-L)
- 1546-170X
- OpenAlex
- W3009583110
- Semantic Scholar
- 5e34d633a076f9df0227f8e2e05cfd76e33f2960
- Cited by
- 477
- References
- 55
- Citations by year
- 2026: 100 · 2025: 78 · 2024: 84 · 2023: 69 · 2022: 66 · 2021: 70 · 2020: 10
- FWCI
- 21.13
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Nature Medicine (publishedVersion)
- PubMed (publishedVersion)
- Bern Open Repository and Information System (University of Bern) (submittedVersion)
- Funders
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (155913, 176005)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Sepsis Diagnosis and Treatment
- Keywords
- circulatory failure, early warning systems, clinical deterioration, intensive care unit, machine learning prediction, alarm fatigue, false alarm rate
- MeSH terms
- Machine Learning, Humans, Intensive Care Units, Models, Theoretical, Prognosis, Risk Factors, ROC Curve, Shock, Time Factors, Reproducibility of Results, Cohort Studies, Databases as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Early prediction of circulatory failure in the intensive care unit using machine learning. https://app.cochat.ai/papers/10.1038/s41591-020-0789-4
- The paper
- Hyland, S. L., Faltys, M., Hüser, M., Lyu, X., Gumbsch, T., Esteban, C., Bock, C., Horn, M., Moor, M., Rieck, B., Zimmermann, M., Bodenham, D., Borgwardt, K., Rätsch, G., & Merz, T. M. (2020). Early prediction of circulatory failure in the intensive care unit using machine learning. Nature Medicine. https://doi.org/10.1038/s41591-020-0789-4
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.