Machine learning for real-time prediction of complications in critical care: a retrospective study
Abstract
Background The large amount of clinical signals in intensive care units can easily overwhelm health-care personnel and can lead to treatment delays, suboptimal care, or clinical errors. The aim of this study was to apply deep machine learning methods to predict severe complications during critical care in real time after cardiothoracic surgery.Methods We used deep learning methods (recurrent neural networks) to predict several severe complications (mortality, renal failure with a need for renal replacement therapy, and postoperative bleeding leading to operative revision) in post cardiosurgical care in real time. Adult patients who underwent major open heart surgery from Jan 1, 2000, to Dec 31, 2016, in a German tertiary care centre for cardiovascular diseases formed the main derivation dataset. We measured the accuracy and timeliness of the deep learning model's forecasts and compared predictive quality to that of established standard-of-care clinical reference tools (clinical rule for postoperative bleeding, Simplified Acute Physiology Score II for mortality, and the Kidney Disease: Improving Global Outcomes staging criteria for acute renal failure) using positive predictive value (PPV), negative predictive value, sensitivity, specificity, area under the curve (AUC), and the F 1 measure (which computes a harmonic mean of sensitivity and PPV). Results were externally retrospectively validated with 5898 cases from the published MIMIC-III dataset.Findings Of 47 559 intensive care admissions (corresponding to 42 007 patients), we included 11 492 (corresponding to 9269 patients). The deep learning models yielded accurate predictions with the following PPV and sensitivity scores: PPV 0·90 and sensitivity 0·85 for mortality, 0·87 and 0·94 for renal failure, and 0·84 and 0·74 for bleeding. The predictions significantly outperformed the standard clinical reference tools, improving the absolute complication prediction AUC by 0·29 (95% CI 0·23-0·35) for bleeding, by 0·24 (0·19-0·29) for mortality, and by 0·24 (0·13-0·35) for renal failure (p<0·0001 for all three analyses). The deep learning methods showed accurate predictions immediately after patient admission to the intensive care unit. We also observed an increase in performance in our validation cohort when the machine learning approach was tested against clinical reference tools, with absolute improvements in AUC of 0·09 (95% CI 0·03-0·15; p=0·0026) for bleeding, of 0·18 (0·07-0·29; p=0·0013) for mortality, and of 0·25 (0·18-0·32; p<0·0001) for renal failure.Interpretation The observed improvements in prediction for all three investigated clinical outcomes have the potential to improve critical care. These findings are noteworthy in that they use routinely collected clinical data exclusively, without the need for any manual processing. The deep machine learning method showed AUC scores that significantly surpass those of clinical reference tools, especially soon after admission. Taken together, these properties are encouraging for prospective deployment in critical care settings to direct the staff's attention towards patients who are most at risk.Funding No specific funding.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Although various research projects are focusing on CDSS in the ICU, translation into clinical routine is lagging far behind.
“Although various research projects are focusing on CDSS in the ICU, translation into the clinical routine is lagging far behind [15,46-49].”
From Clinical requirements of future patient monitoring in the intensive care unit: qualitative study (2019) · cites this paper as [15]
Models for real-time prediction have been trained in clinical contexts such as circulatory failure in critical care and post-operative complications.
“Models for real-time prediction have been trained in clinical contexts such as circulatory failure in critical care24 and post-operative complications25,26.”
From Machine learning for real-time aggregated prediction of hospital admission for emergency patients (2022) · cites this paper as [26]
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
Knaus, W. A., Draper, E. A., Wagner, D. P., & Zimmerman, J. E. (1985) · Critical Care Medicine · doi:10.1097/00003246-198510000-00009 · Cited by 15,333 · see citing works
APACHE II-A Severity of Disease Classification System
Knaus, W. A., Draper, E. A., Wagner, D. P., & Zimmerman, J. E. (1986) · Critical Care Medicine · doi:10.1097/00003246-198608000-00028 · Cited by 13,379 · see citing works
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · see citing works
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141 · see citing works
KDIGO Clinical Practice Guidelines for Acute Kidney Injury
Khwaja, A. (2012) · Nephron Clinical Practice · doi:10.1159/000339789 · Cited by 8,039 · see citing works
Le Gall, J.-R. (1993) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.270.24.2957 · Cited by 7,773 · see citing works
Kumar, A., Roberts, D., Wood, K. E., Light, B., Parrillo, J. E., Sharma, S., Suppes, R., Feinstein, D., Zanotti, S., Taiberg, L., et al. (2006) · Critical Care Medicine · doi:10.1097/01.ccm.0000217961.75225.e9 · Cited by 6,109 · see citing works
Predicting the future - big data, machine learning, and clinical medicine
Obermeyer, Z., & Emanuel, E. J. (2016) · New England Journal of Medicine · doi:10.1056/nejmp1606181 · Cited by 3,652 · see citing works
Recurrent Neural Networks for Multivariate Time Series with Missing Values
Che, Z., Purushotham, S., Cho, K., Sontag, D., & Liu, Y. (2018) · Scientific Reports · doi:10.1038/s41598-018-24271-9 · Cited by 2,370 · see citing works
A Probabilistic Interpretation of Precision, Recall and F-Score, with Implication for Evaluation
Goutte, C., & Gaussier, E. (2005) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-31865-1_25 · Cited by 2,285 · see citing works
European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · see citing works
Machine Learning and Prediction in Medicine — Beyond the Peak of Inflated Expectations
Chen, J. H., & Asch, S. M. (2017) · New England Journal of Medicine · doi:10.1056/nejmp1702071 · Cited by 1,171 · see citing works
Artificial Intelligence in Precision Cardiovascular Medicine
Krittanawong, C., Zhang, H., Wang, Z., Aydar, M., & Kitai, T. (2017) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2017.03.571 · Cited by 1,126 · see citing works
Gaieski, D. F., Mikkelsen, M. E., Band, R. A., Pines, J. M., Massone, R., Furia, F. F., Shofer, F. S., & Goyal, M. (2010) · Critical Care Medicine · doi:10.1097/ccm.0b013e3181cc4824 · Cited by 970 · see citing works
Halpern, N. A., & Pastores, S. M. (2009) · Critical Care Medicine · doi:10.1097/ccm.0b013e3181b090d0 · Cited by 818 · see citing works
Bell, G., Hey, T., & Szalay, A. (2009) · Science · doi:10.1126/science.1170411 · Cited by 537 · see citing works
What This Computer Needs Is a Physician: Humanism and Artificial Intelligence
Verghese, A., Shah, N. H., & Harrington, R. A. (2018) · JAMA · doi:10.1001/jama.2017.19198 · Cited by 468 · see citing works
How Many Variables Can Humans Process?
Halford, G. S., Baker, R., McCredden, J., & Bain, J. (2005) · Psychological Science · doi:10.1111/j.0956-7976.2005.00782.x · Cited by 452 · see citing works
Machine Learning and Decision Support in Critical Care
Johnson, A. E. W., Ghassemi, M. M., Nemati, S., Niehaus, K. E., Clifton, D. A., & Clifford, G. D. (2016) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2501978 · Cited by 438 · see citing works
Learning from Big Health Care Data
Schneeweiß, S. (2014) · New England Journal of Medicine · doi:10.1056/nejmp1401111 · Cited by 321 · see citing works
The hostile environment of the intensive care unit
Donchin, Y., & Seagull, F. J. (2002) · Current Opinion in Critical Care · doi:10.1097/00075198-200208000-00008 · Cited by 297 · see citing works
Lost in thought—the limits of the human mind and the future of medicine
Obermeyer, Z., & Lee, T. H. (2017) · New England Journal of Medicine · doi:10.1056/nejmp1705348 · Cited by 252 · see citing works
Interpretable Deep Models for ICU Outcome Prediction.
Che, Z., Purushotham, S., Khemani, R. G., & Liu, Y. (2016) · PubMed Central · Cited by 240 · see citing works
Recurrent Neural Networks for Multivariate Time Series with Missing ValuesPreprint
Che, Z., Purushotham, S., Cho, K., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.01865 · Cited by 230 · see citing works
“Big Data” in the Intensive Care Unit. Closing the Data Loop
Celi, L. A., Mark, R. G., Stone, D. J., & Montgomery, R. A. (2013) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201212-2311ed · Cited by 209 · see citing works
Modeling Missing Data in Clinical Time Series with RNNsPreprint
Lipton, Z. C., Kale, D. C., & Wetzel, R. C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04130 · Cited by 179 · see citing works
Multicenter, randomized, controlled trials evaluating mortality in intensive care: Doomed to fail?
Ospina-Tascón, G. A., Büchele, G. L., & Vincent, J. (2008) · Critical Care Medicine · doi:10.1097/ccm.0b013e318168ea3e · Cited by 172 · see citing works
Sakr, Y. L., Lobo, S. M., Moreno, R. P., Gerlach, H., Ranieri, V. M., Michalopoulos, A. S., Vincent, J., & the SOAP Investigators (2012) · Critical Care · doi:10.1186/cc11868 · Cited by 149 · see citing works
Artificial intelligence in health care: within touching distance
Lancet, T. (2017) · The Lancet · doi:10.1016/s0140-6736(17)31540-4 · Cited by 117 · see citing works
A path to precision in the ICU
Maslove, D. M., Lamontagne, F., Marshall, J. C., & Heyland, D. K. (2017) · Critical Care · doi:10.1186/s13054-017-1653-x · Cited by 109 · see citing works
Manual of Perioperative Care in Adult Cardiac Surgery
Bojar, R. M. (2020) · Wiley eBooks · doi:10.1002/9781119582540 · Cited by 89 · see citing works
Advances in Information Retrieval
Ounis, I., Amati, G., Plachouras, V., He, B., Macdonald, C., & Johnson, D. (2005) · Lecture notes in computer science · doi:10.1007/b107096 · Cited by 82 · see citing works
Clinical Intervention Prediction and Understanding using Deep NetworksPreprint
Suresh, H., Hunt, N. J., Johnson, A. E. W., Celi, L. A., Peter Szolovits, B., & Ghassemi, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08498 · Cited by 81 · see citing works
Acute abdomen in the medical intensive care unit
Gajic, O., Urrutia, L., Sewani, H. H., Schroeder, D. R., Cullinane, D. C., & Peters, S. G. (2002) · Critical Care Medicine · doi:10.1097/00003246-200206000-00001 · Cited by 73 · see citing works
Manual of Perioperative Care in Adult Cardiac Surgery
Rosenbloom, M. (2011) · Critical Care Medicine · doi:10.1097/ccm.0b013e31822d4495 · Cited by 54 · see citing works
Costs of Critical Care Medicine
Pastores, S. M., Dakwar, J., & Halpern, N. A. (2011) · Critical Care Clinics · doi:10.1016/j.ccc.2011.10.003 · Cited by 53 · see citing works
Digital medicine: empowering both patients and clinicians
Topol, E. J. (2016) · The Lancet · doi:10.1016/s0140-6736(16)31355-1 · Cited by 49 · see citing works
Neural Document Embeddings for Intensive Care Patient Mortality PredictionPreprint
Grnarova, P., Schmidt, F., Hyland, S. L., & Eickhoff, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00467 · Cited by 46 · see citing works
Self-citationMcKenna, J. (2017) · European Heart Journal · doi:10.1093/eurheartj/ehx021 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Early prediction of circulatory failure in the intensive care unit using machine learning
Hyland, S. L., Faltys, M., Hüser, M., Lyu, X., Gumbsch, T., Esteban, C., Bock, C., Horn, M., Moor, M., Rieck, B., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0789-4 · Cited by 477 · see citing works
Xue, B., Li, D., Lu, C., King, C. R., Wildes, T. S., Avidan, M. S., Kannampallil, T., & Abraham, J. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.2240 · Cited by 373 · see citing works
Thorsen-Meyer, H.-C., Nielsen, A. B., Nielsen, A. P., Kaas-Hansen, B. S., Toft, P., Schierbeck, J., Strøm, T., Chmura, P. J., Heimann, M., Dybdahl, L., et al. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30018-2 · Cited by 336 · see citing works
Sendak, M. P., Ratliff, W., Sarro, D., Alderton, E., Futoma, J., Gao, M., Nichols, M., Revoir, M., Yashar, F., Miller, C., et al. (2020) · JMIR Medical Informatics · doi:10.2196/15182 · Cited by 287 · see citing works
Mobile Health in Remote Patient Monitoring for Chronic Diseases: Principles, Trends, and Challenges
El-Rashidy, N. M., El–Sappagh, S., Islam, S. M. R., El-Bakry, H. M., & Abdelrazek, S. M. (2021) · Diagnostics · doi:10.3390/diagnostics11040607 · Cited by 274 · see citing works
Application of artificial intelligence in surgery
Zhou, X.-Y., Guo, Y., Shen, M., & Yang, G.-Z. (2020) · Frontiers of Medicine · doi:10.1007/s11684-020-0770-0 · Cited by 181 · see citing works
Rank, N., Pfahringer, B., Kempfert, J., Stamm, C., Kühne, T., Schoenrath, F., Falk, V., Eickhoff, C., & Meyer, A. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00346-8 · Cited by 153 · see citing works
Self-citationBenedetto, U., Dimagli, A., Sinha, S., Cocomello, L., Gibbison, B., Caputo, M., Gaunt, T. R., Lyon, M., Holmes, C., & Angelini, G. D. (2020) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2020.07.105 · Cited by 142 · see citing works
Hu, M.-Y., Shu, X.-H., Yu, G., Wu, X.-Y., Välimäki, M., & Feng, H. (2021) · Journal of Medical Internet Research · doi:10.2196/20298 · Cited by 132 · see citing works
Artificial Intelligence in Critical Care Medicine
Yoon, J. H., Pinsky, M. R., & Clermont, G. (2022) · Critical Care · doi:10.1186/s13054-022-03915-3 · Cited by 143 · see citing works
Clinical requirements of future patient monitoring in the intensive care unit: qualitative study
Poncette, A.-S., Spies, C., Mosch, L., Schieler, M., Weber-Carstens, S., Krampe, H., & Balzer, F. (2019) · JMIR Medical Informatics · doi:10.2196/13064 · Cited by 136 · see citing works
Machine and deep learning for longitudinal biomedical data: a review of methods and applications
Cascarano, A., Mur-Petit, J., Hernández-González, J., Camacho, M., Eadie, N. D. T., Gkontra, P., Chadeau‐Hyam, M., Vitria, J. M., & Lekadir, K. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10561-w · Cited by 133 · see citing works
Explainable artificial intelligence to detect atrial fibrillation using electrocardiogram
Jo, Y., Cho, Y., Lee, S. Y., Kwon, J., Kim, K., Jeon, K., Cho, S., Park, J., & Oh, B. (2020) · International Journal of Cardiology · doi:10.1016/j.ijcard.2020.11.053 · Cited by 121 · see citing works
Towards risk-aware artificial intelligence and machine learning systems: An overview
Zhang, X., Chan, F. T. S., Yan, C., & Bose, I. (2022) · Decision Support Systems · doi:10.1016/j.dss.2022.113800 · Cited by 113 · see citing works
Machine learning in perioperative medicine: a systematic review
Bellini, V., Valente, M., Bertorelli, G., Pifferi, B., Craca, M., Mordonini, M., Lombardo, G., Bottani, E., RIO, P. D., & Bignami, E. G. (2022) · Journal of Anesthesia Analgesia and Critical Care · doi:10.1186/s44158-022-00033-y · Cited by 115 · see citing works
Machine learning for early detection of sepsis: an internal and temporal validation study
Bedoya, A. D., Futoma, J., Clement, M. E., Corey, K., Brajer, N., Lin, A., Simons, M. G., Gao, M., Nichols, M., Balu, S., et al. (2020) · JAMIA Open · doi:10.1093/jamiaopen/ooaa006 · Cited by 109 · see citing works
Deep learning method for prediction of patient-specific dose distribution in breast cancer
Ahn, S. H., Kim, E., Kim, C., Cheon, W., Kim, M., Lee, S. B., Lim, Y. K., Kim, H., Shin, D., Kim, D. Y., et al. (2021) · Radiation Oncology · doi:10.1186/s13014-021-01864-9 · Cited by 97 · see citing works
Moazemi, S., Vahdati, S., Li, J., Kalkhoff, S., Castano, L. J. V., Dewitz, B., Bibo, R., Sabouniaghdam, P., Tootooni, M. S., Bundschuh, R. A., et al. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1109411 · Cited by 97 · see citing works
Artificial intelligence in spine care: current applications and future utility
Hornung, A. L., Hornung, C. M., Mallow, G. M., Barajas, J. N., Rush, A. J., Sayari, A. J., Galbusera, F., Wilke, H., Colman, M. W., Phillips, F. M., et al. (2022) · European Spine Journal · doi:10.1007/s00586-022-07176-0 · Cited by 89 · see citing works
Machine learning for real-time aggregated prediction of hospital admission for emergency patients
King, Z., Farrington, J., Utley, M., Kung, E., Elkhodair, S., Harris, S., Sekula, R., Gillham, J., Li, K., & Crowe, S. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00649-y · Cited by 87 · see citing works
Garvey, C. S., & Maskal, C. (2020) · Omics · doi:10.1089/omi.2019.0078 · Cited by 80 · see citing works
Jauk, S., Kramer, D., Großauer, B., Rienmüller, S., Avian, A., Berghold, A., Leodolter, W., & Schulz, S. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa113 · Cited by 84 · see citing works
Machine learning prediction models for prognosis of critically ill patients after open-heart surgery
Zhong, Z., Yuan, X., Liu, S., Yang, Y., & Liu, F. (2021) · Scientific Reports · doi:10.1038/s41598-021-83020-7 · Cited by 83 · see citing works
Oosterhoff, J. H. F., & Doornberg, J. N. (2020) · EFORT Open Reviews · doi:10.1302/2058-5241.5.190092 · Cited by 81 · see citing works
Rockenschaub, P., Hilbert, A., Kossen, T., Elbers, P., von Dincklage, F., Madai, V. I., & Frey, D. (2024) · Critical Care Medicine · doi:10.1097/ccm.0000000000006359 · Cited by 91 · see citing works
Hu, H., Wang, W., Chen, L., Ruan, S., Chen, S., Li, X., Lu, M., Xie, X., & Kuang, M. (2021) · Journal of Gastroenterology and Hepatology · doi:10.1111/jgh.15522 · Cited by 76 · see citing works
Natural language processing with machine learning to predict outcomes after ovarian cancer surgery
Barber, E. L., Garg, R., Persenaire, C., & Simon, M. A. (2020) · Gynecologic Oncology · doi:10.1016/j.ygyno.2020.10.004 · Cited by 70 · see citing works
Kumar, S. A., Kumar, T. D. A., Beeraka, N. M., Pujar, G. V., Singh, M., Akshatha, H. S., & Bhagyalalitha, M. (2021) · Future Medicinal Chemistry · doi:10.4155/fmc-2021-0243 · Cited by 71 · see citing works
Pagallo, U., O’Sullivan, S., Nevejans, N., Holzinger, A., Friebe, M., Jeanquartier, F., Jean-Quartier, C., & Miernik, A. J. (2023) · Health and Technology · doi:10.1007/s12553-023-00806-7 · Cited by 77 · see citing works
Amugongo, L. M., Kriebitz, A., Boch, A., & Lütge, C. (2025) · AI and Ethics · doi:10.1007/s43681-023-00331-3 · Cited by 68 · see citing works
Syed, M., Syed, S., Sexton, K. W., Syeda, H. B., Garza, M. Y., Zozus, M. N., Syed, F., Begum, S., Syed, A. U., Sanford, J. A., et al. (2021) · Informatics · doi:10.3390/informatics8010016 · Cited by 68 · see citing works
Lim, L., Gim, U., Cho, K., Yoo, D., Ryu, H. G., & Lee, H. (2024) · Critical Care · doi:10.1186/s13054-024-04866-7 · Cited by 68 · see citing works
Madakkatel, I., Zhou, A., McDonnell, M. D., & Hyppönen, E. (2021) · Scientific Reports · doi:10.1038/s41598-021-02476-9 · Cited by 64 · see citing works
Kindle, R. D., Badawi, O., Celi, L. A., & Sturland, S. (2019) · Critical Care Clinics · doi:10.1016/j.ccc.2019.02.005 · Cited by 67 · see citing works
Surgical data science – from concepts toward clinical translation
Maier‐Hein, L., Eisenmann, M., Sarıkaya, D., März, K., Collins, T., Malpani, A., Fallert, J., Feußner, H., Giannarou, S., Mascagni, P., et al. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102306 · Cited by 88 · see citing works
The use of machine learning and artificial intelligence within pediatric critical care
Shah, N. A., Arshad, A., Mazer, M. B., Carroll, C. L., Shein, S. L., & Remy, K. E. (2022) · Pediatric Research · doi:10.1038/s41390-022-02380-6 · Cited by 64 · see citing works
Vagliano, I., Chesnaye, N. C., Leopold, J. H., Jager, K. J., Abu–Hanna, A., & Schut, M. C. (2022) · Clinical Kidney Journal · doi:10.1093/ckj/sfac181 · Cited by 65 · see citing works
Nardone, V., Marmorino, F., Germani, M. M., Cichowska-Cwalińska, N., Menditti, V. S., Gallo, P., Studiale, V., Taravella, A., Landi, M., Reginelli, A., et al. (2024) · Current Oncology · doi:10.3390/curroncol31090369 · Cited by 64 · see citing works
Characterization of Risk Prediction Models for Acute Kidney Injury
Feng, Y., Wang, Y., Jun, M., Pu, L., Weisbord, S. D., Bellomo, R., Hong, D., & Gallagher, M. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.13359 · Cited by 65 · see citing works
Fan, Y., Long, E., Cai, L., Cao, Q., Wu, X., & Tong, R. (2021) · Frontiers in Pharmacology · doi:10.3389/fphar.2021.665951 · Cited by 58 · see citing works
External validation of AI-based scoring systems in the ICU: a systematic review and meta-analysis
Rockenschaub, P., Akay, E. M., Carlisle, B. G., Hilbert, A., Wendland, J., Meyer-Eschenbach, F., Näher, A.-F., Frey, D., & Madai, V. I. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02830-7 · Cited by 58 · see citing works
Kim, K., Yang, H., Yi, J., Son, H. E., Ryu, J. Y., Kim, Y. C., Jeong, J. C., Chin, H. J., Na, K. Y., Chae, D., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/24120 · Cited by 57 · see citing works
Ideal algorithms in healthcare: Explainable, dynamic, precise, autonomous, fair, and reproducible
Loftus, T. J., Tighe, P. J., Ozrazgat‐Baslanti, T., Davis, J. P., Ruppert, M. M., Ren, Y., Shickel, B., Kamaleswaran, R., Hogan, W. R., Moorman, J. R., et al. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000006 · Cited by 56 · see citing works
Schwyzer, M., Martini, K., Benz, D. C., Burger, I. A., Ferraro, D. A., Kudura, K., Treyer, V., von Schulthess, G. K., Kaufmann, P. A., Huellner, M. W., et al. (2020) · European Radiology · doi:10.1007/s00330-019-06498-w · Cited by 63 · see citing works
A gender specific risk assessment of coronary heart disease based on physical examination data
Yang, H. Q., Luo, Y., Ma, C.-Y., Zhang, T., Zhou, T., Ren, X., He, X.-L., Deng, K., Yan, D., Tang, H., et al. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00887-8 · Cited by 53 · see citing works
Deasy, J., Liò, P., & Ercole, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-79142-z · Cited by 52 · see citing works
Optimization of critical care pharmacy clinical services: A gap analysis approach
Sikora, A. N., Murray, B., Smith, S. E., Brothers, T. N., Al-Mamun, M. A., Chase, A. M., Rowe, S., Buckley, M. S., Murphy, D. J., & Devlin, J. W. (2021) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/zxab237 · Cited by 53 · see citing works
Lonsdale, H., Jalali, A., Ahumada, L. M., & Matava, C. T. (2020) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2020.02.039 · Cited by 48 · see citing works
Hsu, J., Chang, Y.-F., Cheng, H.-J., Yang, C., Lin, C., Chu, S., Huang, H.-R., Chiang, M., Wang, H.-C., & Tsai, M.-H. (2021) · Journal of Personalized Medicine · doi:10.3390/jpm11080695 · Cited by 47 · see citing works
Therapeutic drug monitoring of β-lactam antibiotics in the ICU
Dhaese, S. A. M., Vooren, S. V., Boelens, J., & Waele, J. J. D. (2020) · Expert Review of Anti-infective Therapy · doi:10.1080/14787210.2020.1788387 · Cited by 47 · see citing works
Temporal patterns of organ dysfunction after severe trauma
Eriksson, J., Nelson, D., Holst, A., Hellgren, E., Friman, O., & Oldner, A. (2021) · Critical Care · doi:10.1186/s13054-021-03586-6 · Cited by 44 · see citing works
Luo, X.-Q., Yan, P., Duan, S.-B., Kang, Y., Deng, Y.-H., Liu, Q., Wu, T., & Wu, X. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.853102 · Cited by 46 · see citing works
Predicting Prolonged Length of ICU Stay through Machine Learning
Wu, J., Lin, Y., Li, P., Hu, Y., Zhang, L., & Kong, G. (2021) · Diagnostics · doi:10.3390/diagnostics11122242 · Cited by 47 · see citing works
Luo, Y., Tang, Z., Hu, X., Lu, S., Miao, B., Hong, S., Bai, H., Sun, C., Qiu, J., Liang, H., et al. (2020) · Annals of Translational Medicine · doi:10.21037/atm.2020.01.09 · Cited by 42 · see citing works
AI in the Intensive Care Unit: Up-to-Date Review
Nguyen, D., Ngo, B., & vanSonnenberg, E. (2020) · Journal of Intensive Care Medicine · doi:10.1177/0885066620956620 · Cited by 42 · see citing works
Lee, S. M., Lee, G., Kim, T. K., Le, T. T., Hao, J., Jung, Y. M., Park, C., Park, J. S., Jun, J. K., Lee, H., et al. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.46637 · Cited by 39 · see citing works
Habets, J., Duits, A., Sijben, L. C. J., de Greef, B. T. A., Mulders, A. E. P., Temel, Y., Kuijf, M. L., Kubben, P., Herff, C., & Janssen, M. L. F. (2020) · PeerJ · doi:10.7717/peerj.10317 · Cited by 43 · see citing works
Liu, C., Liu, X., Mao, Z., HU, P., Li, X., Hu, J., Hong, Q., Geng, X., Chi, K., Zhou, F., et al. (2021) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000002674 · Cited by 41 · see citing works
Wang, Y., Yan, F., Lu, X., Zheng, G., Zhang, X., Wang, C., Zhou, K., Zhang, Y., Li, H., Zhao, Q., et al. (2019) · EBioMedicine · doi:10.1016/j.ebiom.2019.05.040 · Cited by 39 · see citing works
Tsuiki, S., Nagaoka, T., Fukuda, T., Sakamoto, Y., Almeida, F. R., Nakayama, H., Inoue, Y., & Enno, H. (2021) · Sleep And Breathing · doi:10.1007/s11325-021-02301-7 · Cited by 42 · see citing works
Predicting In-Hospital Mortality at Admission to the Medical Ward: A Big-Data Machine Learning Model
Soffer, S., Klang, E., Barash, Y., Grossman, E., & Zimlichman, E. (2020) · The American Journal of Medicine · doi:10.1016/j.amjmed.2020.07.014 · Cited by 38 · see citing works
Barboi, C., Tzavelis, A., & Muhammad, L. N. (2022) · JMIR Medical Informatics · doi:10.2196/35293 · Cited by 38 · see citing works
Liang, Q., Zhao, Q., Xu, X., Zhou, Y., & Huang, M. (2022) · Journal of Global Antimicrobial Resistance · doi:10.1016/j.jgar.2022.03.019 · Cited by 37 · see citing works
Logothetis, S. B., Green, D., Holland, M., & Moubayed, N. A. (2023) · Scientific Reports · doi:10.1038/s41598-023-40661-0 · Cited by 37 · see citing works
Mehta, D., Gonzalez, X. T., Huang, G. T.-W., & Abraham, J. (2024) · British Journal of Anaesthesia · doi:10.1016/j.bja.2024.08.007 · Cited by 38 · see citing works
O’Brien, C. L., Goldstein, B. A., Shen, Y., Phelan, M., Lambert, C., Bedoya, A., & Steorts, R. C. (2020) · MDM Policy & Practice · doi:10.1177/2381468319899663 · Cited by 36 · see citing works
Gbashi, S., Adebo, O. A., Doorsamy, W., & Njobeh, P. B. (2020) · JMIR Medical Informatics · doi:10.2196/22916 · Cited by 36 · see citing works
Jentzer, J. C., Kashou, A. H., & Murphree, D. H. (2023) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2023.100089 · Cited by 36 · see citing works
Artificial Intelligence for Emerging Technology in Surgery: Systematic Review and Validation
Nwoye, E. O., Woo, W. L., Gao, B., & Anyanwu, T. (2022) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2022.3183852 · Cited by 38 · see citing works
Lapp, L., Roper, M., Kavanagh, K., Bouamrane, M.-M., & Schraag, S. (2023) · Journal of Intensive Care Medicine · doi:10.1177/08850666231166349 · Cited by 36 · see citing works
Predicting 6-Month Unfavorable Outcome of Acute Ischemic Stroke Using Machine Learning
Li, X., Pan, X., Jiang, C., Wu, M., Liu, Y., Wang, F., Zheng, X., Yang, J., Sun, C., Zhu, Y., et al. (2020) · Frontiers in Neurology · doi:10.3389/fneur.2020.539509 · Cited by 34 · see citing works
Zhang, K., Zhang, S., Cui, W., Hong, Y., Zhang, G., & Zhang, Z. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2020.609769 · Cited by 34 · see citing works
Development and validation of a deep learning model to predict the survival of patients in ICU
Tang, H., Jin, Z., Deng, J., She, Y., Zhong, Y., Sun, W., Ren, Y., Cao, N., & Chen, C. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac098 · Cited by 34 · see citing works
Al-Mamun, M. A., Brothers, T. N., & Sikora, A. N. (2020) · Annals of Pharmacotherapy · doi:10.1177/1060028020959042 · Cited by 33 · see citing works
Grendas, L. N., Chiapella, L. C., Rodante, D., & Daray, F. M. (2021) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2021.11.029 · Cited by 33 · see citing works
Leukocyte involvement in the signs and symptoms of chronic venous disease. Perspectives for therapy.
Boisseau, M. (2007) · PubMed · Cited by 33 · see citing works
Ueno, R., Xu, L., Uegami, W., Matsui, H., Okui, J., Hayashi, H., Miyajima, T., Hayashi, Y., Pilcher, D. V., & Jones, D. A. (2020) · PLoS ONE · doi:10.1371/journal.pone.0235835 · Cited by 32 · see citing works
Poncette, A.-S., Mosch, L. K., Stablo, L., Spies, C., Schieler, M., Weber‐Carstens, S., Feufel, M. A., & Balzer, F. (2021) · JMIR Human Factors · doi:10.2196/30655 · Cited by 32 · see citing works
Wang, H., Zhao, Q.-Y., Luo, J.-C., Liu, K., Yu, S.-J., Ma, J.-F., Luo, M.-H., Hao, G.-W., Su, Y., Zhang, Y.-J., et al. (2022) · BMC Pulmonary Medicine · doi:10.1186/s12890-022-02096-7 · Cited by 32 · see citing works
Ruan, X., Fu, S., Storlie, C. B., Mathis, K. L., Larson, D. W., & Liu, H. (2022) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2022.104202 · Cited by 30 · see citing works
Zhu, X., Hu, J., Xiao, T., Huang, S., Wen, Y., & Shang, D. (2022) · Frontiers in Pharmacology · doi:10.3389/fphar.2022.975855 · Cited by 33 · see citing works
Machine learning-based mortality prediction model for heat-related illness
Hirano, Y., Kondo, Y., Hifumi, T., Yokobori, S., Kanda, J., Shimazaki, J., Hayashida, K., Moriya, T., Yagi, M., Takauji, S., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-88581-1 · Cited by 30 · see citing works
Machine learning for predicting successful extubation in patients receiving mechanical ventilation
Igarashi, Y., Ogawa, K., Nishimura, K., Osawa, S., Ohwada, H., & Yokobori, S. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.961252 · Cited by 28 · see citing works
Innovative Approaches to Safe Surgery: A Narrative Synthesis of Best Practices
Hussain, A. K., Kakakhel, M. K., Ashraf, M. F., Shahab, M., Ahmad, F., Luqman, F., Ahmad, M., Nour, A. M., Varrassi, G., & Kinger, S. (2023) · Cureus · doi:10.7759/cureus.49723 · Cited by 33 · see citing works
Regulatory T cell-based therapy in type 1 diabetes: Latest breakthroughs and evidence
Huang, Q., & Zhu, J. (2024) · International Immunopharmacology · doi:10.1016/j.intimp.2024.112724 · Cited by 30 · see citing works
Röhrich, S., Hofmanninger, J., Negrin, L. L., Langs, G., & Prosch, H. (2021) · European Radiology · doi:10.1007/s00330-020-07635-6 · Cited by 29 · see citing works
Mosch, L. K., Poncette, A.-S., Spies, C., Weber‐Carstens, S., Schieler, M., Krampe, H., & Balzer, F. (2021) · JMIR Formative Research · doi:10.2196/22866 · Cited by 36 · see citing works
Wanyan, T., Honarvar, H., Azad, A., Ding, Y., & Glicksberg, B. S. (2021) · Data Intelligence · doi:10.1162/dint_a_00097 · Cited by 27 · see citing works
Zeng, Z., Tang, X., Liu, Y., He, Z., & Gong, X. (2022) · BioData Mining · doi:10.1186/s13040-022-00309-7 · Cited by 28 · see citing works
Meyer, L. M., Stead, S., Salge, T. O., & Antons, D. (2024) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2024.123568 · Cited by 28 · see citing works
Tran, T. T. L., Yun, G., & Kim, S. (2024) · BMC Nephrology · doi:10.1186/s12882-024-03793-7 · Cited by 27 · see citing works
Bülow, R. D., Dimitrov, D., Boor, P., & Sáez-Rodríguez, J. (2021) · Seminars in Immunopathology · doi:10.1007/s00281-021-00847-y · Cited by 27 · see citing works
Sung, M., Hahn, S., Han, C. H., Lee, J. M., Lee, J., Yoo, J., Heo, J., Kim, Y. S., & Chung, K. S. (2021) · JMIR Medical Informatics · doi:10.2196/26426 · Cited by 27 · see citing works
Zhang, H., Wang, Y., Wu, S., Ngo, J., Feng, Y., He, X., Zhang, Y., Wu, X., & Hong, D. (2022) · BMC Nephrology · doi:10.1186/s12882-022-03025-w · Cited by 26 · see citing works
Predicting Acute Kidney Injury After Cardiac Surgery Using a Simpler Model
Coulson, T. G., Bailey, M., Pilcher, D., Reid, C. M., Seevanayagam, S., Williams‐Spence, J., & Bellomo, R. (2020) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2020.06.072 · Cited by 26 · see citing works
Song, Z., Yang, Z., Hou, M., & Shi, X. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.951881 · Cited by 26 · see citing works
The quality and utility of artificial intelligence in patient care
Wehkamp, K., Krawczak, M., & Schreiber, S. (2023) · Deutsches Ärzteblatt international · doi:10.3238/arztebl.m2023.0124 · Cited by 26 · see citing works
Dai, A., Zhou, Z., Jiang, F., Guo, Y., Asante, D. O., Feng, Y., Huang, K., Chen, C., Shi, H., Si, Y., et al. (2023) · Annals of Medicine · doi:10.1080/07853890.2023.2266458 · Cited by 24 · see citing works
Zhu, X., Zhang, M., Wen, Y., & Shang, D. (2022) · Frontiers in Pharmacology · doi:10.3389/fphar.2022.994665 · Cited by 24 · see citing works
Predictive Modeling for Readmission to Intensive Care: A Systematic Review
Ruppert, M. M., Loftus, T. J., Small, C., Li, H., Ozrazgat-Baslanti, T., Balch, J., Holmes, R., Tighe, P. J., Upchurch, G. R., Efron, P. A., et al. (2023) · Critical Care Explorations · doi:10.1097/cce.0000000000000848 · Cited by 24 · see citing works
- Authors
- Meyer, A., Zverinski, D., Pfahringer, B., Kempfert, J., Kuehne, T., Sündermann, S. H., Stamm, C., Hofmann, T., Falk, V., & Eickhoff, C.
- Affiliations
- Deutsches Herzzentrum der Charité (DE)
- ETH Zurich (CH)
- Brown University (US)
- Corresponding author
- Meyer, Alexander
- ORCID
- Venue
- The Lancet Respiratory Medicine
- Publisher
- Elsevier BV
- Published
- 2018-12
- Volume
- 6
- Issue
- 12
- Pages
- 905-914
- ISSN
- 2213-2600 (ISSN-L)
- OpenAlex
- W2892592994
- Cited by
- 350
- References
- 41
- Citations by year
- 2026: 29 · 2025: 40 · 2024: 41 · 2023: 51 · 2022: 58 · 2021: 54 · 2020: 57 · 2019: 15 · 2018: 4
- FWCI
- 18.32
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Lancet Respiratory Medicine (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Charité – Universitätsmedizin Berlin
- Berlin Institute of Health
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (174025)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Cardiac, Anesthesia and Surgical Outcomes
- Keywords
- complication prediction, intensive care, cardiothoracic surgery, postoperative mortality, acute kidney injury, renal replacement therapy, postoperative bleeding, real-time prediction, recurrent neural networks, intensive care unit, MIMIC-III, early warning systems, model validation, external validation, clinical decision support, treatment delay, medical errors
- MeSH terms
- Deep Learning, Aged, Female, Cardiac Surgical Procedures, Humans, Intensive Care Units, Male, Middle Aged, Predictive Value of Tests, Retrospective Studies, ROC Curve, Outcome Assessment, Health Care, Postoperative Hemorrhage, Renal Insufficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
