Early hospital mortality prediction of intensive care unit patients using an ensemble learning approach
Abstract
Background Mortality prediction of hospitalized patients is an important problem. Over the past few decades, several severity scoring systems and machine learning mortality prediction models have been developed for predicting hospital mortality. By contrast, early mortality prediction for intensive care unit patients remains an open challenge. Most research has focused on severity of illness scoring systems or data mining (DM) models designed for risk estimation at least 24 or 48h after ICU admission. Objectives This study highlights the main data challenges in early mortality prediction in ICU patients and introduces a new machine learning based framework for Early Mortality Prediction for Intensive Care Unit patients (EMPICU). Materials and methods The proposed method is evaluated on the Multiparameter Intelligent Monitoring in Intensive Care II (MIMIC-II) database. Mortality prediction models are developed for patients at the age of 16 or above in Medical ICU (MICU), Surgical ICU (SICU) or Cardiac Surgery Recovery Unit (CSRU). We employ the ensemble learning Random Forest (RF), the predictive Decision Trees (DT), the probabilistic Naive Bayes (NB) and the rule-based Projective Adaptive Resonance Theory (PART) models. The primary outcome was hospital mortality. The explanatory variables included demographic, physiological, vital signs and laboratory test variables. Performance measures were calculated using cross-validated area under the receiver operating characteristic curve (AUROC) to minimize bias. 11,722 patients with single ICU stays are considered. Only patients at the age of 16 years old and above in Medical ICU (MICU), Surgical ICU (SICU) or Cardiac Surgery Recovery Unit (CSRU) are considered in this study. Results The proposed EMPICU framework outperformed standard scoring systems (SOFA, SAPS-I, APACHE-II, NEWS and qSOFA) in terms of AUROC and time (i.e. at 6h compared to 48h or more after admission). Discussion and conclusion The results show that although there are many values missing in the first few hour of ICU admission, there is enough signal to effectively predict mortality during the first 6h of admission. The proposed framework, in particular the one that uses the ensemble learning approach - EMPICU Random Forest (EMPICU-RF) offers a base to construct an effective and novel mortality prediction model in the early hours of an ICU patient admission, with an improved performance profile.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Accurate mortality prediction can give clinicians advanced warnings of possible deteriorations and share the burdens of making information-heavy decisions.
“Accurate mortality prediction can give clinicians advanced warnings of possible deteriorations and share the burdens of making information-heavy decisions44.”
From Integrated multimodal artificial intelligence framework for healthcare applications (2022) · cites this paper as [44]
Clinicians in ICU units often need to make rapid evaluations of patient conditions to inform treatment plans.
“Due to its time and outcome-critical environments, clinicians in ICU units often need to make rapid evaluations of patient conditions to inform treatment plans44.”
From Integrated multimodal artificial intelligence framework for healthcare applications (2022) · cites this paper as [44]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865 · see citing works
The third international consensus definitions for sepsis and septic shock (Sepsis-3).
Singer, M., Deutschman, C. S., Seymour, C. W., Shankar-Hari, M., Annane, D., Bauer, M., Bellomo, R., Bernard, G. R., Chiche, J.-D., Coopersmith, C. M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0287 · Cited by 28,250 · see citing works
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750 · see citing works
Knaus, W. A., Draper, E. A., Wagner, D. P., & Zimmerman, J. E. (1985) · Critical Care Medicine · doi:10.1097/00003246-198510000-00009 · Cited by 15,333 · see citing works
Goldberger, A. L., Amaral, L. A. N., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., Mietus, J. E., Moody, G. B., Peng, C.-K., & Stanley, H. E. (2000) · Circulation · doi:10.1161/01.cir.101.23.e215 · Cited by 14,809 · see citing works
APACHE II-A Severity of Disease Classification System
Knaus, W. A., Draper, E. A., Wagner, D. P., & Zimmerman, J. E. (1986) · Critical Care Medicine · doi:10.1097/00003246-198608000-00028 · Cited by 13,379 · see citing works
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141 · see citing works
Le Gall, J.-R. (1993) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.270.24.2957 · Cited by 7,773 · see citing works
Top 10 algorithms in data mining
Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G. J., Ng, A., Liu, B., Yu, P. S., et al. (2008) · Knowledge and Information Systems · doi:10.1007/s10115-007-0114-2 · Cited by 5,667 · see citing works
Seymour, C. W., Liu, V. X., Iwashyna, T. J., Brunkhorst, F. M., Rea, T. D., Scherag, A., Rubenfeld, G., Kahn, J. M., Shankar-Hari, M., Singer, M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0288 · Cited by 3,939 · see citing works
The APACHE III Prognostic System
Knaus, W. A., Wagner, D. P., Draper, E. A., Zimmerman, J. E., Bergner, M., BASTOS, P. G., Sirio, C., Murphy, D. J., Lotring, T., Damiano, A., et al. (1991) · CHEST Journal · doi:10.1378/chest.100.6.1619 · Cited by 3,524 · see citing works
Vincent, J.-L., de Mendonca, A., Cantraine, F., Moreno, R., Takala, J., Suter, P. M., Sprung, C. L., Colardyn, F., & Blecher, S. (1998) · Critical Care Medicine · doi:10.1097/00003246-199811000-00016 · Cited by 3,463 · see citing works
The APACHE III Prognostic System
Teres, D., & Lemeshow, S. A. (1992) · CHEST Journal · doi:10.1378/chest.102.6.1919b · Cited by 2,880 · see citing works
Zimmerman, J. E., Kramer, A. A., McNair, D. S., & Malila, F. M. (2006) · Critical Care Medicine · doi:10.1097/01.ccm.0000215112.84523.f0 · Cited by 1,764 · see citing works
Data Mining Techniques: For Marketing, Sales, and Customer Support
Berry, M. J., & Linoff, G. S. (1997) · University of Maribor digital library (University of Maribor) · Cited by 1,746 · see citing works
Moreno, R. P., Metnitz, P., Almeida, E., Jordan, B., Bauer, P., Campos, R. A., Iapichino, G., Edbrooke, D. L., Capuzzo, M., Gall, J. L., et al. (2005) · Intensive Care Medicine · doi:10.1007/s00134-005-2763-5 · Cited by 1,414 · see citing works
A simplified acute physiology score for ICU patients
Gall, J. L., Loirat, P., Alpérovitch, A., GLASER, P., Granthil, C., Mathieu, D., MERCIER, P., THOMAS, R., & Villers, D. (1984) · Critical Care Medicine · doi:10.1097/00003246-198411000-00012 · Cited by 1,247 · see citing works
Predicting breast cancer survivability: a comparison of three data mining methods
Delen, D., Walker, G., & Kadam, A. (2004) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2004.07.002 · Cited by 1,215 · see citing works
Smith, G. B., Prytherch, D. R., Meredith, P., Schmidt, P. E., & Featherstone, P. I. (2013) · Resuscitation · doi:10.1016/j.resuscitation.2012.12.016 · Cited by 1,077 · see citing works
Lemeshow, S. A. (1993) · JAMA · doi:10.1001/jama.1993.03510200084037 · Cited by 1,030 · see citing works
2015 IEEE Congress on Evolutionary Computation (CEC2015)
(2015) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2014.2369861 · Cited by 805 · see citing works
Engineering in Medicine and Biology Society (EMBC), 2012 Annual International Conference of the IEEE
Fadlallah, B. H., Brockmeier, A. J., Seth, S., Li, L., Keil, A., & Príncipe, J. C. (2012) · CiteSeer X (The Pennsylvania State University) · Cited by 520 · see citing works
2015 International Joint Conference on Neural Networks (IJCNN)
(2015) · doi:10.1109/ijcnn34689.2015 · Cited by 400 · see citing works
Pirracchio, R., Petersen, M. L., Carone, M., Rigon, M. R., Chevret, S., & van der Laan, M. J. (2015) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(14)70239-5 · Cited by 374 · see citing works
Higgins, T. L., Teres, D., Copes, W. S., Nathanson, B. H., Stark, M., & Kramer, A. A. (2007) · Critical Care Medicine · doi:10.1097/01.ccm.0000257337.63529.9f · Cited by 333 · see citing works
A method for predicting survival and mortality of ICU patients using objectively derived weights
Lemeshow, S. A., Teres, D., Pastides, H., Avrunin, J. S., & Steingrub, J. S. (1985) · Critical Care Medicine · doi:10.1097/00003246-198507000-00001 · Cited by 314 · see citing works
2021 Computing in Cardiology (CinC)
(2021) · doi:10.23919/cinc53138.2021 · Cited by 241 · see citing works
Dybowski, R., Gant, V, Weller, P., & Chang, R. (1996) · The Lancet · doi:10.1016/s0140-6736(96)90609-1 · Cited by 240 · see citing works
Simpson, S. Q. (2016) · CHEST Journal · doi:10.1016/j.chest.2016.02.653 · Cited by 215 · see citing works
Mortality prediction using SAPS II: an update for French intensive care units
Gall, J. R. L., Neumann, A., Hémery, F., et al. (2005) · Critical Care · doi:10.1186/cc3821 · Cited by 174 · see citing works
Clermont, G., Angus, D. C., DiRusso, S. M., Griffin, M. F., & Linde‐Zwirble, W. T. (2001) · Critical Care Medicine · doi:10.1097/00003246-200102000-00012 · Cited by 165 · see citing works
Kim, S., Kim, W., & Park, R. W. (2011) · Healthcare Informatics Research · doi:10.4258/hir.2011.17.4.232 · Cited by 132 · see citing works
Patient length of stay and mortality prediction: A survey
Awad, A., Bader–El–Den, M., & McNicholas, J. (2017) · Health Services Management Research · doi:10.1177/0951484817696212 · Cited by 128 · see citing works
Self-citationMachine learning techniques to examine large patient databases
Meyfroidt, G., Güiza, F., Ramon, J., & Bruynooghe, M. (2009) · Best practice & research. Clinical anaesthesiology · doi:10.1016/j.bpa.2008.09.003 · Cited by 104 · see citing works
Recent innovations in intensive care unit risk-prediction models
Rosenberg, A. L. (2002) · Current Opinion in Critical Care · doi:10.1097/00075198-200208000-00009 · Cited by 103 · see citing works
Critical care: advances and future perspectives
Vincent, J., & Singer, M. (2010) · The Lancet · doi:10.1016/s0140-6736(10)60575-2 · Cited by 94 · see citing works
Mining data from intensive care patients
Ramon, J., Daan Fierens, A., Güiza, F., Meyfroidt, G., Blockeel, H., Bruynooghe, M., & den Berghe, G. V. (2007) · Advanced Engineering Informatics · doi:10.1016/j.aei.2006.12.002 · Cited by 85 · see citing works
Calvert, J. S., Mao, Q., Hoffman, J. L., Jay, M., Desautels, T., Mohamadlou, H., Chettipally, U. K., & Das, R. (2016) · Annals of Medicine and Surgery · doi:10.1016/j.amsu.2016.09.002 · Cited by 77 · see citing works
Mortality assessment in intensive care units via adverse events using artificial neural networks
Silva, Á., Cortez, P. C., Santos, M. F., et al. (2005) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2005.07.006 · Cited by 66 · see citing works
Wong, L. S. S., & Young, D. (1999) · Anaesthesia · doi:10.1046/j.1365-2044.1999.01104.x · Cited by 65 · see citing works
Austrian validation and customization of the SAPS 3 Admission Score
on behalf of the ASDI Study Group, Metnitz, B., Schaden, E., Moreno, R. P., Gall, J. L., Bauer, P., & Metnitz, P. (2008) · Intensive Care Medicine · doi:10.1007/s00134-008-1286-2 · Cited by 64 · see citing works
GiViTI, Poole, D., Rossi, C., Latronico, N., Rossi, G., Finazzi, S., & Bertolini, G. (2012) · Intensive Care Medicine · doi:10.1007/s00134-012-2578-0 · Cited by 63 · see citing works
A Database-driven Decision Support System: Customized Mortality Prediction
Celi, L. A., Galvin, S., Davidzon, G., Lee, J., Scott, D., & Mark, R. G. (2012) · Journal of Personalized Medicine · doi:10.3390/jpm2040138 · Cited by 62 · see citing works
Luo, Y., Xin, Y., Joshi, R., Celi, L. A., & Peter Szolovits, B. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.9998 · Cited by 61 · see citing works
Prediction of mortality in an Indian intensive care unit
Nimgaonkar, A., Karnad, D. R., Sudarshan, S., Lucila Ohno-machado, A., & Kohane, I. S. (2004) · Intensive Care Medicine · doi:10.1007/s00134-003-2105-4 · Cited by 51 · see citing works
Gilani, M., Razavi, M., & Azad, A. (2014) · Nigerian Medical Journal · doi:10.4103/0300-1652.129651 · Cited by 47 · see citing works
Doig, G. S., Inman, K. J., Sibbald, W. J., et al. (1993) · PubMed · Cited by 46 · see citing works
Kramer, A. A., Higgins, T. L., & Zimmerman, J. E. (2013) · Critical Care Medicine · doi:10.1097/ccm.0b013e3182a66a49 · Cited by 41 · see citing works
Severe sepsis mortality prediction with relevance vector machines
Ribas, V. J., Lopez, J. C., Ruiz‐Sanmartín, A., et al. (2011) · doi:10.1109/iembs.2011.6089906 · Cited by 40 · see citing works
PhysioNet 2012 Challenge: Predicting mortality of ICU patients using a cascaded SVM-GLM paradigm
Citi, L., & Barbieri, R. (2012) · Open Access at Essex (University of Essex) · Cited by 31 · see citing works
Imbalanced classification using genetically optimized cost sensitive classifiers
Perry, T., Bader-El-Den, M. B., & Cooper, S. (2015) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2015.7256956 · Cited by 30 · see citing works
Self-citationPrediction using patient comparison vs. modeling: A case study for mortality prediction
Hoogendoorn, M., el Hassouni, A., Mok, K., Ghassemi, M., & Peter Szolovits, B. (2016) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2016.7591229 · Cited by 29 · see citing works
Customization of a Severity of Illness Score Using Local Electronic Medical Record Data
Lee, J., & Maslove, D. M. (2015) · Journal of Intensive Care Medicine · doi:10.1177/0885066615585951 · Cited by 27 · see citing works
Hierarchical classification for dealing with the Class imbalance problem
Bader-El-Den, M. B., Teitei, E., & Adda, M. (2016) · doi:10.1109/ijcnn.2016.7727660 · Cited by 20 · see citing works
Self-citationSelf-adaptive heterogeneous random forest
Bader-El-Den, M. B. (2014) · IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa.2014.7073259 · Cited by 17 · see citing works
Self-citationGlance, L. G., Dick, A. W., & Osler, T. M. (2014) · Critical Care Medicine · doi:10.1097/01.ccm.0000435695.23586.04 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Integrated multimodal artificial intelligence framework for healthcare applications
Soenksen, L. R., Ma, Y., Zeng, C., Boussioux, L., Villalobos Carballo, K., Na, L., Wiberg, H. M., Li, M. L., Fuentes, I., & Bertsimas, D. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00689-4 · Cited by 380 · see citing works
“Due to its time and outcome-critical environments, clinicians in ICU units often need to make rapid evaluations of patient conditions to inform treatment plans 44 .”
BackgroundBardak, B., & Tan, M. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102112 · Cited by 48 · see citing works
Sleep quality prediction in caregivers using physiological signals
Sadeghi, R., Banerjee, T., Hughes, J. C., & Lawhorne, L. W. (2019) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2019.05.010 · Cited by 50 · see citing works
Wardi, G., Owens, R., Josef, C., Malhotra, A., Longhurst, C., & Nemati, S. (2023) · Critical Care Medicine · doi:10.1097/ccm.0000000000005894 · Cited by 35 · see citing works
Boussi Rahmouni, H., Hassine, N. B. E. H., Chouchen, M., Ceylan, H. İ., Muntean, R. I., Bragazzi, N. L., & Dergaa, I. (2025) · Healthcare · doi:10.3390/healthcare13202553 · Cited by 25 · see citing works
- Authors
- Awad, A., Bader-El-Den, M., McNicholas, J., & Briggs, J.
- Affiliations
- University of Portsmouth (GB)
- Queen Alexandra Hospital (GB)
- Corresponding author
- Bader-El-Den, Mohamed
- Venue
- International Journal of Medical Informatics
- Publisher
- Elsevier BV
- Published
- 2017-12
- Volume
- 108
- Pages
- 185-195
- ISSN
- 1386-5056 (ISSN-L)
- OpenAlex
- W2764165920
- Cited by
- 208
- References
- 57
- Citations by year
- 2026: 15 · 2025: 23 · 2024: 17 · 2023: 34 · 2022: 41 · 2021: 25 · 2020: 30 · 2019: 16 · 2018: 7
- FWCI
- 8.65
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Medical Informatics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Epidemiology › Sepsis Diagnosis and Treatment
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- early mortality prediction, intensive care unit patients, in-hospital mortality, ensemble learning, random forest, MIMIC-II, disease severity assessment, naive Bayes
- MeSH terms
- Machine Learning, Adolescent, Adult, Aged, Aged, 80 and over, Bayes Theorem, Female, Heart Diseases, Humans, Intensive Care Units, Male, Middle Aged, ROC Curve, Severity of Illness Index, Databases, Factual, Hospital Mortality, Outcome Assessment, Health Care, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
