Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 23

The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.

Multicenter validation of a machine learning model to predict intensive care unit readmission within 48 hours after discharge

Lim, Leerang, Kim, Mincheol, Cho, Kyungjae, Yoo, Dongjoon, Sim, Dayeon, Ryu, Ho Geol, Lee, Hyung-Chul · 2025 · eClinicalMedicine

Abstract

Background: Intensive care unit (ICU) readmission is a crucial indicator of patient safety. However, discharge decisions often rely on subjective assessment due to a lack of standardized guidelines. We aimed to develop a machine-learning model to predict ICU readmission within 48 h and compare its performance to traditional scoring systems. Methods: We developed an ensemble model, iREAD, that generates a probability score at ICU discharge, representing the likelihood of the patient being readmitted to the ICU within 48 h, using data from Seoul National University Hospital (SNUH) and validated it using the MIMIC-III and eICU-CRD datasets. From September 2007 to August 2021, a total of 70,842 patients were included from SNUH. The MIMIC-III datasets comprised 43,237 patients admitted to ICUs between 2001 and 2012 at Beth Israel Deaconess Medical Center, and the eICU-CRD datasets included 90,271 ICU admissions across 208 hospitals between 2014 and 2015. Patients younger than 18, those who died in ICUs, or who refused life-sustaining treatment were excluded from the final analysis. The model's performance was evaluated using the area under the receiver operating characteristic curve (AUROC) and compared to the traditional scores and conventional machine learning models. Kaplan-Meier analysis was performed to compare the outcome between the high-risk and low-risk groups. Findings: We developed the iREAD, that utilized 30 input features, encompassing demographics, length of stay, vital signs, GCS, and laboratory values. iREAD demonstrated superior performance compared with other models across all cohorts (all P < 0.001). In the internal validation, iREAD achieved AUROCs of 0.771 (95% CI 0.743-0.798), 0.834 (0.821-0.846), and 0.820 (0.808-0.832) for early (≤48 h), late (>48 h), and overall ICU readmissions, respectively. External validations with MIMIC-III and eICU-CRD also showed modest performance with AUROCs of 0.768 (0.748-0.787) and 0.725 (0.712-0.739) for overall readmission in MIMIC-III and eICU-CRD respectively, demonstrating superior performance compared to other models (All P < 0.001; higher than other models). Kaplan-Meier analysis revealed that over 40% of high-risk patients predicted by iREAD were readmitted within 48 h, representing a more than four-fold increase in predictive performance compared to the traditional scores. Interpretation: iREAD demonstrates superior performance in predicting ICU readmission within 48 h after discharge compared to traditional scoring systems or conventional machine learning models in both internal and external validations. While the performance degradation observed in the external validations suggests the need for further prospective validation on diverse patient populations, the robust performance and ability to identify high-risk patients have the potential to guide clinical decision-making. Funding: This work was supported by the Korea Health Technology Research & Development Project through the Korea Health Industry Development Institute, funded by the Ministry of Health and Welfare, Republic of Korea (grant number RS-2021-KH114109).

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.eclinm.2025.103112

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 10 references

Research works printed but not linked: 41

  1. Woldhek, 2017, Readmission of ICU patients: a quality indicator?, J Crit Care, 38, 328, 10.1016/j.jcrc.2016.12.001 · DOI
  2. Kramer, 2013, The association between ICU readmission rate and patient outcomes, Crit Care Med, 41, 24, 10.1097/CCM.0b013e3182657b8a · DOI · Paper page
  3. Berenholtz, 2002, Qualitative review of intensive care unit quality indicators, J Crit Care, 17, 1, 10.1053/jcrc.2002.33035 · DOI · Paper page
  4. Kramer, 2016, Can this patient be safely discharged from the ICU?, Intensive Care Med, 42, 580, 10.1007/s00134-015-4148-8 · DOI
  5. Stelfox, 2015, A scoping review of patient discharge from intensive care: opportunities and tools to improve care, Chest, 147, 317, 10.1378/chest.13-2965 · DOI
  6. Bai, 2021, Managing admission and discharge processes in intensive care units, Health Care Manag Sci, 24, 666, 10.1007/s10729-021-09560-6 · DOI
  7. Badawi, 2012, Readmissions and death after ICU discharge: development and validation of two predictive models, PLoS One, 7, 10.1371/journal.pone.0048758 · DOI
  8. Frost, 2009, Severity of illness and risk of readmission to intensive care: a meta-analysis, Resuscitation, 80, 505, 10.1016/j.resuscitation.2009.02.015 · DOI
  9. Timmers, 2012, Patients' characteristics associated with readmission to a surgical intensive care unit, Am J Crit Care, 21, e120, 10.4037/ajcc2012773 · DOI
  10. Campbell, 2008, Predicting death and readmission after intensive care discharge, Br J Anaesth, 100, 656, 10.1093/bja/aen069 · DOI · Paper page
  11. Gajic, 2008, The Stability and Workload Index for Transfer score predicts unplanned intensive care unit patient readmission: initial development and validation, Crit Care Med, 36, 676, 10.1097/CCM.0B013E318164E3B0 · DOI · Paper page
  12. Reini, 2012, The prognostic value of the Modified Early Warning Score in critically ill patients: a prospective, observational study, Eur J Anaesthesiol, 29, 152, 10.1097/EJA.0b013e32835032d8 · DOI
  13. Jahn, 2019, The predictive performance of SAPS 2 and SAPS 3 in an intermediate care unit for internal medicine at a German university transplant center; a retrospective analysis, PLoS One, 14, 10.1371/journal.pone.0222164 · DOI
  14. Lim, 2024, Real-time machine learning model to predict short-term mortality in critically ill patients: development and international validation, Crit Care, 28, 76, 10.1186/s13054-024-04866-7 · DOI
  15. Thoral, 2021, Explainable machine learning on AmsterdamUMCdb for ICU discharge decision support: uniting intensivists and data scientists, Crit Care Explor, 3
  16. Johnson, 2016, MIMIC-III, a freely accessible critical care database, Sci Data, 3, 10.1038/sdata.2016.35 · DOI · Paper page
  17. Pollard, 2018, The eICU Collaborative Research Database, a freely available multi-center database for critical care research, Sci Data, 5, 10.1038/sdata.2018.178 · DOI · Paper page
  18. Moor, 2023, Predicting sepsis using deep learning across international sites: a retrospective development and validation study, eClinicalMedicine, 62, 10.1016/j.eclinm.2023.102124 · DOI · Paper page
  19. Liu, 2023, Illness severity assessment of older adults in critical illness using machine learning (ELDER-ICU): an international multicentre study with subgroup bias evaluation, Lancet Digit Health, 5, e657, 10.1016/S2589-7500(23)00128-0 · DOI
  20. Rojas, 2018, Predicting intensive care unit readmission with machine learning using electronic health record data, Ann Am Thorac Soc, 15, 846, 10.1513/AnnalsATS.201710-787OC · DOI · Paper page
  21. Lee, 2022, Multi-center validation of machine learning model for preoperative prediction of postoperative mortality, NPJ Digit Med, 5, 91, 10.1038/s41746-022-00625-6 · DOI
  22. Ke, 2017, LightGBM: a highly efficient gradient boosting decision tree, Adv Neural Inf Process Syst, 30, 3149
  23. Ba, 2016, Layer normalization, arXiv
  24. Srivastava, 2014, Dropout: a simple way to prevent neural networks from overfitting, J Mach Learn Res, 15, 1929
  25. Kim, 2023, Early prediction of need for invasive mechanical ventilation in the neonatal intensive care unit using artificial intelligence and electronic health records: a clinical study, BMC Pediatr, 23, 525, 10.1186/s12887-023-04350-1 · DOI
  26. Cho, 2023, Prospective, multicenter validation of the deep learning-based cardiac arrest risk management system for predicting in-hospital cardiac arrest or unplanned intensive care unit transfer in patients admitted to general wards, Crit Care, 27, 346, 10.1186/s13054-023-04609-0 · DOI
  27. Cho, 2024, External validation of deep learning-based cardiac arrest risk management system for predicting in-hospital cardiac arrest in patients admitted to general wards based on rapid response system operating and nonoperating periods: a single-center study, Crit Care Med, 52, e110, 10.1097/CCM.0000000000006137 · DOI
  28. Kim, 2023, Development and validation of deep-learning-based sepsis and septic shock early prediction system (DeepSEPS) using real-world ICU data, J Clin Med, 12, 7156
  29. Lee, 2021, A multicentre validation study of the deep learning-based early warning score for predicting in-hospital cardiac arrest in patients admitted to general wards, Resuscitation, 163, 78, 10.1016/j.resuscitation.2021.04.013 · DOI
  30. Subbe, 2001, Validation of a modified early warning score in medical admissions, QJM, 94, 521, 10.1093/qjmed/94.10.521 · DOI · Paper page
  31. Jones, 2012, NEWSDIG: the national early warning score development and implementation group, Clin Med, 12, 501, 10.7861/clinmedicine.12-6-501 · DOI · Paper page
  32. Knaus, 1985, APACHE II: a severity of disease classification system, Crit Care Med, 13, 818, 10.1097/00003246-198510000-00009 · DOI · Paper page
  33. Smith, 2008, A review, and performance evaluation, of single-parameter "track and trigger" systems, Resuscitation, 79, 11, 10.1016/j.resuscitation.2008.05.004 · DOI · Paper page
  34. Lundberg, 2017, A unified approach to interpreting model predictions, Adv Neural Inf Process Syst, 30, 4768
  35. Wang, 2019, Machine learning for survival analysis: a survey, ACM Comput Surv, 51, 1, 10.1145/3214306 · DOI · Paper page
  36. Kastrup, 2013, Predictive ability of the stability and workload index for transfer score to predict unplanned readmissions after ICU discharge, Crit Care Med, 41, 1608, 10.1097/CCM.0b013e31828a217b · DOI
  37. Lin, 2019, Analysis and prediction of unplanned intensive care unit readmission using recurrent neural networks with long short-term memory, PLoS One, 14
  38. de Hond, 2023, Predicting readmission or death after discharge from the ICU: external validation and retraining of a machine learning model, Crit Care Med, 51, 291, 10.1097/CCM.0000000000005758 · DOI
  39. Frost, 2010, Readmission to intensive care: development of a nomogram for individualising risk, Crit Care Resusc, 12, 83
  40. Jo, 2015, Readmission to medical intensive care units: risk factors and prediction, Yonsei Med J, 56, 543, 10.3349/ymj.2015.56.2.543 · DOI
  41. Hammer, 2021, A tool to predict readmission to the intensive care unit in surgical critical care patients-the RISC score, J Intensive Care Med, 36, 1296, 10.1177/0885066620949164 · DOI

Metadata

Authors
Lim, L., Kim, M., Cho, K., Yoo, D., Sim, D., Ryu, H. G., & Lee, H.-C.
Venue
eClinicalMedicine
Publisher
Elsevier BV
Published
2025-03
Volume
81
Pages
103112
ISSN
2589-5370
Cited by
23
References
10
Language
en
Licence
Funders
  • Korea Health Industry Development Institute
  • Korea Ministry of Health and Welfare (RS-2021-KH114109)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Multicenter validation of a machine learning model to predict intensive care unit readmission within 48 hours after discharge. https://app.cochat.ai/papers/10.1016/j.eclinm.2025.103112
The paper
Lim, L., Kim, M., Cho, K., Yoo, D., Sim, D., Ryu, H. G., & Lee, H.-C. (2025). Multicenter validation of a machine learning model to predict intensive care unit readmission within 48 hours after discharge. eClinicalMedicine. https://doi.org/10.1016/j.eclinm.2025.103112

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.