External validation of AI-based scoring systems in the ICU: a systematic review and meta-analysis
Open accessAbstract
Background: Machine learning (ML) is increasingly used to predict clinical deterioration in intensive care unit (ICU) patients through scoring systems. Although promising, such algorithms often overfit their training cohort and perform worse at new hospitals. Thus, external validation is a critical – but frequently overlooked – step to establish the reliability of predicted risk scores to translate them into clinical practice. We systematically reviewed how regularly external validation of ML-based risk scores is performed and how their performance changed in external data. Methods: We searched MEDLINE, Web of Science, and arXiv for studies using ML to predict deterioration of ICU patients from routine data. We included primary research published in English before December 2023. We summarised how many studies were externally validated, assessing differences over time, by outcome, and by data source. For validated studies, we evaluated the change in area under the receiver operating characteristic (AUROC) attributable to external validation using linear mixed-effects models. Results: We included 572 studies, of which 84 (14.7%) were externally validated, increasing to 23.9% by 2023. Validated studies made disproportionate use of open-source data, with two well-known US datasets (MIMIC and eICU) accounting for 83.3% of studies. On average, AUROC was reduced by -0.037 (95% CI -0.052 to -0.027) in external data, with more than 0.05 reduction in 49.5% of studies. Discussion: External validation, although increasing, remains uncommon. Performance was generally lower in external data, questioning the reliability of some recently proposed ML-based scores. Interpretation of the results was challenged by an overreliance on the same few datasets, implicit differences in case mix, and exclusive use of AUROC.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A systematic review showed marked between-hospital variability in performance, indicating that transportability cannot be assumed even for widely deployed systems.
“A systematic review showed marked between-hospital variability in performance, indicating that transportability cannot be assumed even for widely deployed systems [9].”
From Beyond Model Development in Healthcare AI: Post-Development Robustness, Post-Deployment Monitoring, and Lifecycle Governance—A Scoping Review of Reviews (2026) · cites this paper as [9]
The Results section (‘Performance at new hospitals’; page unavailable) reports that external-validation performance varied substantially and was, on average, lower than internal-validation performance. The Discussion says this makes validation at new hospitals important. The review does not establish that the systems showing this variability were widely deployed; it describes widespread clinical adoption as a goal. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The third international consensus definitions for sepsis and septic shock (Sepsis-3).
Singer, M., Deutschman, C. S., Seymour, C. W., Shankar-Hari, M., Annane, D., Bauer, M., Bellomo, R., Bernard, G. R., Chiche, J.-D., Coopersmith, C. M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0287 · Cited by 28,250 · see citing works
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013) · doi:10.1201/b16018 · Cited by 27,276 · see citing works
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856 · see citing works
Conducting Meta-Analyses in <i>R</i> with the <b>metafor</b> Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464 · see citing works
Vincent, J. L., Moreno, R., Takala, J., Willatts, S., De Mendonça, A., Bruining, H., Reinhart, C. K., Suter, P. M., & Thijs, L. G. (1996) · Intensive Care Medicine · doi:10.1007/bf01709751 · Cited by 11,674 · see citing works
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · see citing works
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141 · see citing works
KDIGO Clinical Practice Guidelines for Acute Kidney Injury
Khwaja, A. (2012) · Nephron Clinical Practice · doi:10.1159/000339789 · Cited by 8,039 · see citing works
Vincent, J.-L., de Mendonca, A., Cantraine, F., Moreno, R., Takala, J., Suter, P. M., Sprung, C. L., Colardyn, F., & Blecher, S. (1998) · Critical Care Medicine · doi:10.1097/00003246-199811000-00016 · Cited by 3,463 · see citing works
Key challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136 · see citing works
Serial Evaluation of the SOFA Score to Predict Outcome in Critically Ill Patients
Ferreira, F. L. (2001) · JAMA · doi:10.1001/jama.286.14.1754 · Cited by 2,862 · see citing works
Pollard, T. J., Johnson, A. E. W., Raffa, J. D., Celi, L. A., Mark, R. G., & Badawi, O. (2018) · Scientific Data · doi:10.1038/sdata.2018.178 · Cited by 1,993 · see citing works
Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., et al. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.2626 · Cited by 995 · see citing works
External validation of prognostic models: what, why, how, when and where?
Ramspek, C. L., Jager, K. J., Dekker, F. W., Zoccali, C., & van Diepen, M. (2021) · Clinical Kidney Journal · doi:10.1093/ckj/sfaa188 · Cited by 887 · see citing works
In All Likelihood: Statistical Modelling and Inference Using Likelihood
Pawitan, Y. (2001) · doi:10.1093/oso/9780198507659.001.0001 · Cited by 874 · see citing works
Evaluation of clinical prediction models (part 1): from development to external validation
Collins, G. S., Dhiman, P., Ma, J., Schlussel, M. M., Archer, L., Van Calster, B., Harrell, F. E., Martin, G. P., Moons, K. G. M., van Smeden, M., et al. (2024) · BMJ · doi:10.1136/bmj-2023-074819 · Cited by 798 · see citing works
Clinical review: Scoring systems in the critically ill
Vincent, J.-L., & Moreno, R. (2010) · Critical Care · doi:10.1186/cc8204 · Cited by 634 · see citing works
Evaluation of clinical prediction models (part 2): how to undertake an external validation study
Riley, R., Archer, L., Snell, K., Ensor, J., Dhiman, P., Martin, G. P., Bonnett, L., & Collins, G. S. (2024) · British medical journal · doi:10.1136/bmj-2023-074820 · Cited by 515 · see citing works
Bayesian data analysis, third edition
Gelman, A., Carlin, J. B., Stern, H., Dunson, D., Vehtari, A., & Rubin, D. (2013) · Cited by 488 · see citing works
Early prediction of circulatory failure in the intensive care unit using machine learning
Hyland, S. L., Faltys, M., Hüser, M., Lyu, X., Gumbsch, T., Esteban, C., Bock, C., Horn, M., Moor, M., Rieck, B., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0789-4 · Cited by 477 · see citing works
The myth of generalisability in clinical research and machine learning in health care
Futoma, J., Simons, M., Panch, T., Doshi-Velez, F., & Celi, L. A. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30186-2 · Cited by 467 · see citing works
Hernandez-Boussard, T., Bozkurt, S., Ioannidis, J. P. A., & Shah, N. H. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa088 · Cited by 410 · see citing works
Gerry, S., Bonnici, T., Birks, J., Kirtley, S., Virdee, P. S., Watkinson, P., & Collins, P. G. S. (2020) · BMJ · doi:10.1136/bmj.m1501 · Cited by 398 · see citing works
Early Prediction of Sepsis From Clinical Data: The PhysioNet/Computing in Cardiology Challenge 2019
Reyna, M. A., Josef, C. S., Jeter, R., Shashikumar, S. P., Westover, M. B., Nemati, S., Clifford, G. D., & Sharma, A. (2020) · Critical Care Medicine · doi:10.1097/ccm.0000000000004145 · Cited by 383 · see citing works
Pirracchio, R., Petersen, M. L., Carone, M., Rigon, M. R., Chevret, S., & van der Laan, M. J. (2015) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(14)70239-5 · Cited by 374 · see citing works
Machine learning for real-time prediction of complications in critical care: a retrospective study
Meyer, A., Zverinski, D., Pfahringer, B., Kempfert, J., Kuehne, T., Sündermann, S. H., Stamm, C., Hofmann, T., Falk, V., & Eickhoff, C. (2018) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(18)30300-x · Cited by 350 · see citing works
The development of a machine learning inpatient acute kidney injury prediction model.
Koyner, J. L., Carey, K. A., Edelson, D. P., & Churpek, M. M. (2018) · Critical Care Medicine · doi:10.1097/ccm.0000000000003123 · Cited by 323 · see citing works
There is no such thing as a validated prediction model
Van Calster, B., Steyerberg, E. W., Wynants, L., & van Smeden, M. (2023) · BMC Medicine · doi:10.1186/s12916-023-02779-w · Cited by 286 · see citing works
Thoral, P. J., Peppink, J. M., Driessen, R. H., Sijbrands, E. J. G., Kompanje, E. J. O., Kaplan, L., Bailey, H., Kesecioglu, J., Cecconi, M., Churpek, M., et al. (2021) · Critical Care Medicine · doi:10.1097/ccm.0000000000004916 · Cited by 257 · see citing works
Use of machine learning to analyse routinely collected intensive care unit data: a systematic review
Shillan, D., Sterne, J. A. C., Champneys, A., & Gibbison, B. (2019) · Critical Care · doi:10.1186/s13054-019-2564-9 · Cited by 251 · see citing works
A computational approach to early sepsis detection
Calvert, J. S., Price, D. A., Chettipally, U. K., Barton, C. W., Feldman, M. D., Hoffman, J. L., Jay, M., & Das, R. (2016) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2016.05.003 · Cited by 244 · see citing works
Early Prediction of Sepsis in the ICU Using Machine Learning: A Systematic Review
Moor, M., Rieck, B., Horn, M., Jutzeler, C. R., & Borgwardt, K. M. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.607952 · Cited by 234 · see citing works
The impact of machine learning on patient care: A systematic review
Ben-Israel, D., Jacobs, W. B., Casha, S., Lang, S., Ryu, W. H. A., de Lotbiniere-Bassett, M., & Cadotte, D. W. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2019.101785 · Cited by 207 · see citing works
Silva, I., Moody, G. B., Scott, D., Celi, L. A., & Mark, R. G. (2012) · PubMed · Cited by 167 · see citing works
Sullivan, P. G., Wallach, J. D., & Ioannidis, J. P. A. (2016) · The American Journal of Cardiology · doi:10.1016/j.amjcard.2016.08.024 · Cited by 144 · see citing works
Song, X., Yu, A. S. L., Kellum, J. A., Waitman, L. R., Matheny, M. E., Simpson, S. Q., Hu, Y., & Liu, M. (2020) · Nature Communications · doi:10.1038/s41467-020-19551-w · Cited by 120 · see citing works
Moor, M., Bennett, N., Plečko, D., Horn, M., Rieck, B., Meinshausen, N., Bühlmann, P., & Borgwardt, K. (2023) · eClinicalMedicine · doi:10.1016/j.eclinm.2023.102124 · Cited by 118 · see citing works
Higgins, D., & Madai, V. I. (2020) · Advanced Intelligent Systems · doi:10.1002/aisy.202000052 · Cited by 118 · see citing works
Self-citationArtificial Intelligence in Clinical Diagnosis: Opportunities, Challenges, and Hype
Kulkarni, P. A., & Singh, H. (2023) · JAMA · doi:10.1001/jama.2023.11440 · Cited by 112 · see citing works
Rockenschaub, P., Hilbert, A., Kossen, T., Elbers, P., von Dincklage, F., Madai, V. I., & Frey, D. (2024) · Critical Care Medicine · doi:10.1097/ccm.0000000000006359 · Cited by 91 · see citing works
Self-citationFederated Learning for Electronic Health Records
Dang, T. K., Xiang, L., Weng, J., & Feng, M. (2022) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3514500 · Cited by 90 · see citing works
Vagliano, I., Chesnaye, N. C., Leopold, J. H., Jager, K. J., Abu–Hanna, A., & Schut, M. C. (2022) · Clinical Kidney Journal · doi:10.1093/ckj/sfac181 · Cited by 65 · see citing works
Vincent, J., de Mendonça, A., CANTRAINE, F., MORENO, R., TAKALA, J., SUTER, P., Sprung, C., Colardyn, F., & BLECHER, S. (1998) · Ghent University Academic Bibliography (Ghent University) · Cited by 65 · see citing works
Moor, M., Horn, M., Rieck, B., Roqueiro, D., & Borgwardt, K. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.01659 · Cited by 62 · see citing works
Don't be misled: 3 misconceptions about external validation of clinical prediction models
la Roi-Teeuw, H. M., van Royen, F. S.-A., de Hond, A., Zahra, A., de Vries, S., Bartels, R., Carriero, A., van Doorn, S., Dunias, Z. S., Kant, I. M. J., et al. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111387 · Cited by 57 · see citing works
Hur, S., Ko, R., Yoo, J., Ha, J., Cha, W. C., & Chung, C. R. (2021) · JMIR Medical Informatics · doi:10.2196/23401 · Cited by 37 · see citing works
ricu: R’s interface to intensive care data
Bennett, N., Plečko, D., Ukor, I.-F., Meinshausen, N., & Bühlmann, P. (2022) · GigaScience · doi:10.1093/gigascience/giad041 · Cited by 25 · see citing works
Rockenschaub, P. (2023) · OSF Preprints (OSF Preprints) · doi:10.17605/osf.io/f7j46 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
1 of the 49 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- [14]Hüser M, Faltys M, Lyu X, Barber C, Hyland SL, Merz TM, et al. Early prediction of respiratory failure in the intensive care unit. arXiv [cs.LG]; 2021.
- [23]Carlisle BG. Numbat systematic review manager. Berlin, Germany: The Grey Literature; 2014.
- [27]Gelman A, Jakulin A, Pittau MG, Su Y-S. A weakly informative default prior distribution for logistic and other regression models. aoas. 2008;2:1360–83.
- [47]Arnrich B, Choi E, Fries JA, McDermott MBA, Oh J, Pollard T et al. Medical Event Data Standard (MEDS): Facilitating Machine Learning for Health. In: ICLR 2024 Workshop on Learning from Time Series For Health. 2024.
- [48]van de Water R, Schmidt HNA, Elbers P, Thoral P, Arnrich B, Rockenschaub P. Yet Another ICU Benchmark: A Flexible Multi-Center Framework for Clinical ML. In: The Twelfth International Conference on Learning Representations. 2024.
Cited by
Sorted by citations
Cheah, B. C. J., Vicente, C. R., & Chan, K. R. (2025) · Viruses · doi:10.3390/v17070882 · Cited by 43 · see citing works
A common longitudinal intensive care unit data format (CLIF) for critical illness research
Rojas, J. C., Lyons, P. G., Chhikara, K., Chaudhari, V., Bhavani, S. V., Nour, M., Buell, K. G., Smith, K. D., Gao, C. A., Amagai, S., et al. (2025) · Intensive Care Medicine · doi:10.1007/s00134-025-07848-7 · Cited by 27 · see citing works
Eskandar, K. (2025) · iLiver · doi:10.1016/j.iliver.2025.100205 · Cited by 13 · see citing works
Żerdziński, K., Janiec, J., Jóźwik, K., Łajczak, P. M., & Krzych, Ł. J. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm15010185 · Cited by 12 · see citing works
Data-Driven Quality of Care in the ICU: A Concise Review
Moralez, G. M., Amado, F., Liu, V. X., Tan, S. C., Meyfroidt, G., Stevens, R. D., Pilcher, D., & Salluh, J. I. (2025) · Critical Care Medicine · doi:10.1097/ccm.0000000000006862 · Cited by 10 · see citing works
The value of machine and deep learning in management of critically ill patients: An umbrella review
Tungushpayev, M., Suleimenova, D., Sarria-Santamerra, A., Aimyshev, T., Gaipov, A., & Viderman, D. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106081 · Cited by 10 · see citing works
Xue, M., Yang, J., Wang, H., Yan, Z., Chen, X., Gao, W., Luo, R., Lv, X., & Ye, Z. (2025) · Journal of Anesthesia and Translational Medicine · doi:10.1016/j.jatmed.2025.06.003 · Cited by 7 · see citing works
Applications, image analysis, and interpretation of computer vision in medical imaging
Matsuzaka, Y., & Iyoda, M. (2026) · Frontiers in Radiology · doi:10.3389/fradi.2025.1733003 · Cited by 11 · see citing works
El Arab, R. A., Mustafa, M. H., Almagharbeh, W. T., Saleem, N. H., Al Abdulmohsen, S., Boathab, R., & Bu Washl, M. (2026) · Healthcare · doi:10.3390/healthcare14111459 · Cited by 4 · see citing works
Yadgarov, M. Y., Rebrova, O. Y., Бондоевич, Б. Л., Polyakov, P. A., Kadantseva, K., Yakovlev, A., Grechko, A. V., & Likhvantsev, V. V. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15020777 · Cited by 4 · see citing works
Ramadan, R. Α., Madi, N. K. M., Fageeri, S. O., Yousif, K. A., Al-Daweri, M. S., & Rajhi, W. K. H. A. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.132919 · Cited by 5 · see citing works
van Kouswijk, H. W., Yazid, H., Schoones, J. W., Witlox, A. M. A., Nelissen, R. G. H. H., & de Witte, P. B. (2025) · Children · doi:10.3390/children12050645 · Cited by 4 · see citing works
Khalifa, A., & Hussein, R. (2026) · Journal of Medical Systems · doi:10.1007/s10916-026-02391-4 · Cited by 5 · see citing works
A systematic review on the generative AI applications in human medical genetics
Changalidis, A. I., Barbitoff, Y. A., Nasykhova, Y. A., & Glotov, A. S. (2026) · Frontiers in Genetics · doi:10.3389/fgene.2025.1694070 · Cited by 6 · see citing works
ICU scoring systems: current perspectives and future directions
Salluh, J. I., Moralez, G. M., Tracy, A., & Deliberato, R. O. (2025) · Current Opinion in Critical Care · doi:10.1097/mcc.0000000000001305 · Cited by 4 · see citing works
Multiplex Targeted Proteomic Analysis of Cytokine Ratios for ICU Mortality in Severe COVID-19
Araújo, R., Rekowski, C. P. V., Fonseca, T. A. H., Calado, C. R. C., Ramalhete, L., & Bento, L. F. N. (2025) · Proteomes · doi:10.3390/proteomes13030035 · Cited by 3 · see citing works
Do, D.-K., Rockenschaub, P., Boie, S. D., Kumpf, O., Volk, H., Balzer, F., von Dincklage, F., & Lichtner, G. (2026) · Journal of Medical Internet Research · doi:10.2196/72083 · Cited by 6 · see citing works
Self-citationAllyn, J., Oliver, M., Cerveau, R., Allou, N., Barennes, T., & Ferdynus, C. (2026) · Critical Care · doi:10.1186/s13054-026-06034-5 · Cited by 2 · see citing works
Jagadeesh, A. T., Aramrat, C., Rai, S., Maqsood, F. H., Madhukeshwar, A. K., Bhogadi, S., Lieber, J., Mahajan, H., Banjara, S. K., Lewin, A., et al. (2025) · The Lancet Regional Health - Southeast Asia · doi:10.1016/j.lansea.2025.100644 · Cited by 2 · see citing works
Fourkiotis, K. P., & Tsadiras, A. K. (2025) · Applied Sciences · doi:10.3390/app15169192 · Cited by 1 · see citing works
Tonkaz, D. E., Tonkaz, M., Bekçi, T., & Tonkaz, G. (2025) · Seminars in Ultrasound CT and MRI · doi:10.1053/j.sult.2025.10.002 · Cited by 1 · see citing works
Harnessing Artificial Intelligence in Head and Neck Oncology Practice
Rao, K. N., Kirsch, C., Mahmood, H., & Simon, C. (2025) · Otolaryngologic Clinics of North America · doi:10.1016/j.otc.2025.09.003 · Cited by 2 · see citing works
Chen, H., Gan, W., Zhou, X., Liu, P., Ding, T., Xu, H., Chen, P., & Chen, Y. (2026) · Frontiers in Cellular and Infection Microbiology · doi:10.3389/fcimb.2025.1667176 · Cited by 1 · see citing works
Schwinn, J., Sheikhalishahi, S., Morhart, M., Soto‐Rey, I., Simon, P., Kaspar, M., & Hinske, L. C. (2026) · Digital Health · doi:10.1177/20552076261428383 · Cited by 1 · see citing works
Pandolfi, T., Jiang, K., Mahant, A., Dera, D., Murra, A., Brehman, E., Oliveira, I., Saramya, E., McIntosh, S., Wong, N. D., et al. (2026) · Journal of clinical lipidology · doi:10.1016/j.jacl.2026.05.010 · Cited by 1 · see citing works
Vallée, A. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001610 · Cited by 1 · see citing works
Predicting violence in the intensive care unit: Can artificial intelligence protect those who care?
Johnson, G. U., Towell‐Barnard, A., & Ewens, B. A. (2025) · Intensive and Critical Care Nursing · doi:10.1016/j.iccn.2025.104054 · Cited by 1 · see citing works
Jagadeesh, A., Aramrat, C., Rai, S., Maqsood, F. H., Madhukeshwar, A. K., Bhogadi, S., Lieber, J., Mahajan, H., Banjara, S. K., Lewin, A., et al. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5153451
Early Prediction of ICU Readmissions Using Machine Learning in a Colombian University Hospital
Pazos, J. A. L., Camacho-Cogollo, J. E., & Peñaranda, J. L. M. (2025) · World Congress on Medical Physics and Biomedical Engineering, September 7 - 12, 2009, Munich, Germany · doi:10.1007/978-3-031-96538-8_11
Patel, M. J., Digaswala, F., Vyas, D., Patel, S., Parmar, D., & Patel, U. B. (2025) · Journal of Innovative Image Processing · doi:10.36548/jiip.2025.4.011
Ruz, P. L., Vega-Márquez, B., & Pontes, B. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-10661-2_9
Kharde, P. K., Kharde, Y., & Kanade, T. M. (2026) · Updates in Surgery · doi:10.1007/s13304-026-02553-6
Lee, J., Hwang, S., & Lee, K.-H. (2026) · JMIR Medical Informatics · doi:10.2196/83487
Enhancing Osteosarcoma Survival Predictions
Galoaa, B. M., Ubong, S. E., Girgis, A. G., Gonzalez, M. R., & Lozano‐Calderón, S. A. (2026) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.25.01327
(254423), H. W., Lao, G., He, R., (45625), T. L., Luo, H., Qin, C., Luo, H., Liu, Y., Huang, J., Wei, Z., et al. (2026) · iScience · doi:10.1016/j.isci.2026.116135
Gerbaud, A. F., de la Guardia, M. E. B., Booth, S. C., Adams, L., Cox, M. D., Hunsinger, K., Tan, N., Gulden, M., Menser, T. L., Reichard, R., et al. (2026) · Mayo Clinic Proceedings Innovations Quality & Outcomes · doi:10.1016/j.mayocpiqo.2026.100725 · Cited by 1 · see citing works
How Are Researchers Using MIMIC? A Methods-Focused Review of Studies in Critical CarePreprint
Kenarangi, T., Barnett, A. G., Suen, J. Y., & White, N. M. (2026) · Research Square · doi:10.21203/rs.3.rs-9962098/v1
Eskandar, K. (2026) · World Journal of Nephrology · doi:10.5527/wjn.v15.i2.117719
Bulloni, M., Torquati, M. C., Ahola, R., Bonesi, A., Taconné, M., Vehkaoja, A. T., Tirschmann, F., Kallonen, A., Moreno-Sánchez, P., Lyytikäinen, L., et al. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106595
Al-Ani, A., Alkhawaja, M., Yang, L., Mikhail, D., Connors, L., Milad, D., Ambikkumar, A., Hill, V., Punja, K. G., Costello, F., et al. (2026) · Ophthalmology Science · doi:10.1016/j.xops.2026.101324
Toward Effective Human Oversight in a Human-in-the-Loop AI System: Multi-Model AI and Model Labeling
Chang, J., Oh, Z. X., & Syafiie, S. (2026) · IEEE International Conference on Automatic Control and Intelligent Systems (I2CACIS) · doi:10.1109/i2cacis69435.2026.11600358
PRACTALL 2025: Artificial intelligence—application of allergy and immunology to patient care
Shamji, M. H., Rider, N. L., Adcock, I. M., Pongdee, T., Giacco, S. D., Chang, C., Jaen, M. T., & Williams, P. V. (2026) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2026.04.031
Ruan, H., Lee, J.-K., Xiong, J., Zou, L.-J., & Li, S.-S. (2026) · Digital Health · doi:10.1177/20552076261481028
Lei, Q.-K., Wu, T.-X., Zhang, C.-L., & Zheng, Y. (2026) · Liver Research · doi:10.1016/j.livres.2026.08.001
Reimagining the Intensive Care Unit as a Complex Adaptive System: A Systems Thinking Perspective
Choudhury, A., Joshi, G., Saigal, S., Pakhare, A. P., & Joshi, A. (2026) · Cureus · doi:10.7759/cureus.115305
Serbena, D. R., Cieslack, I. L. F., Ratis, R. C., Chaves, D. P., Oliveira, S., Neves, H. A. F., dos Santos, F. S., Cassar, D. R., da Silva, W. C. F. N., & Bonini, J. S. (2026) · Current Oncology Reports · doi:10.1007/s11912-026-01824-0
Othman, Z. K., Ahmed, M. M., Nazari, R., & Ibrahim, A. M. (2026) · Biotechnology Notes · doi:10.1016/j.biotno.2026.08.003
Toward More Reliable Survival Prediction in Osteosarcoma: The Case for Multi-Data-Set AI
Ruggieri, P. (2026) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.26.00611
Dabbachi, I., Masmoudi, S., Bouzida, I., & NAIFAR, O. (2026) · Intelligent Data Analysis · doi:10.1177/1088467x261484827
How are researchers using MIMIC? A sampling-based methods review
Kenarangi, T., Barnett, A. G., Suen, J. Y., & White, N. M. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106744
Zhang, K., Li, X., Zhang, X., Wang, X., Zhang, H., Liu, S., Gao, Y., Yin, J., Li, X., & Zhang, D. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1948751
Unger, S., Hasan, Y., & Anderle, L. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-02444-0
Ha, N. K., Lee, H.-U., Kim, C., 임찬주, Yoo, S., Kim, J., 허주원, Kim, H., & Son, M. (2026) · Hospitals · doi:10.3390/hospitals3040020
Pathak, S., Si, Y., Sanjaya, J., Haghi, M., Kudrot, N. T., Placencia, G., Alaei, K., & Pishgar, M. (2026) · medRxiv · doi:10.64898/2026.10.11.26365235
Yadgarov, M. Y., Rebrova, O. Y., Shvyrev, S. L., Бондоевич, Б. Л., Polyakov, P. A., Yakovlev, A. А., & Likhvantsev, V. V. (2026) · Vrach i informacionnye tehnologii · doi:10.25881/18110193_2026_3_150
Pennisi, F., Pinto, A., Cozzolino, C., Rezza, G., Signorelli, C., Baldo, V., & Gianfredi, V. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001796
- Authors
- Rockenschaub, P., Akay, E. M., Carlisle, B. G., Hilbert, A., Wendland, J., Meyer-Eschenbach, F., Näher, A.-F., Frey, D., & Madai, V. I.
- Affiliations
- Innsbruck Medical University (AT)
- Charité – Universitätsmedizin Berlin (DE)
- McGill University (CA)
- Ruhr University Bochum (DE)
- Hasso Plattner Institute (DE)
- Birmingham City University (GB)
- Corresponding author
- Madai, Vince Istvan
- ORCID
- Venue
- BMC Medical Informatics and Decision Making
- Published
- 2025-01-06
- Volume
- 25
- Issue
- 1
- Pages
- 5-5
- ISSN
- 1472-6947 (ISSN-L)
- PMC
- PMC11702098
- OpenAlex
- W4406090644
- Cited by
- 58
- References
- 49
- Citations by year
- 2026: 40 · 2025: 18
- FWCI
- 5.53
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- BMC Medical Informatics and Decision Making (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PUBLISSO (German National Library of Medicine) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Charité - Universitätsmedizin Berlin
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Health Sciences › Medicine › Epidemiology › Sepsis Diagnosis and Treatment
- Keywords
- external validation, machine learning, clinical deterioration prediction, intensive care unit, risk score, overfitting, mimics, eICU, open-source dataset, area under the receiver operating characteristic, case mix, generalizability
- MeSH terms
- Machine Learning, Humans, Intensive Care Units, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
