Current challenges in adopting machine learning to critical care and emergency medicine
Open accessAbstract
Over the past decades, the field of machine learning (ML) has made great strides in medicine. Despite the number of ML-inspired publications in the clinical arena, the results and implications are not readily accepted at the bedside. Although ML is very powerful in deciphering hidden patterns in complex critical care and emergency medicine data, various factors including data, feature generation, model design, performance assessment, and limited implementation could affect the utility of the research. In this short review, a series of current challenges of adopting ML models to clinical research will be discussed.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Reasons for the AI chasm include lack of necessary expertise, lack of funding, underappreciation of clinical research, disregard for early stages, and poor reporting and evaluations.
“Reasons proposed for this so-called AI chasm [18] are lack of necessary expertise needed for translating a tool into practice, lack of funding available for translation, underappreciation of clinical research as a translation mechanism, disregard for the potential value of the early stages of clinical evaluation and the analysis of human factors [19], and poor reporting and evaluations [2, 8, 20].”
From Use of artificial intelligence in critical care: opportunities and obstacles (2024) · cites this paper as [2]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856 · see citing works
Saito, T., & Rehmsmeier, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0118432 · Cited by 5,309 · see citing works
Recent advances and applications of machine learning in solid-state materials science
Schmidt, J., Marques, M. R. G., Botti, S., & Marques, M. A. L. (2019) · npj Computational Materials · doi:10.1038/s41524-019-0221-0 · Cited by 2,655 · see citing works
Explainable machine-learning predictions for the prevention of hypoxaemia during surgery
Lundberg, S. M., Nair, B., Vavilala, M. S., Horibe, M., Eisses, M. J., Adams, T., Liston, D. E., Low, D. K.-W., Newman, S.-F., Kim, J., et al. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0304-0 · Cited by 1,967 · see citing works
Sheller, M. J., Edwards, B., Reina, G. A., Martin, J., Pati, S., Kotrotsou, A., Milchenko, M., Xu, W., Marcus, D., Colen, R. R., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-69250-1 · Cited by 1,646 · see citing works
Samek, W., Wiegand, T., & Müller, K.-R. (2017) · arXiv · doi:10.48550/arxiv.1708.08296 · Cited by 1,080 · see citing works
Ten quick tips for machine learning in computational biology
Chicco, D. (2017) · BioData Mining · doi:10.1186/s13040-017-0155-3 · Cited by 875 · see citing works
Federated learning for predicting clinical outcomes in patients with COVID-19
Dayan, I., Roth, H. R., Zhong, A., Harouni, A., Gentili, A., Abidin, A. Z., Liu, A., Costa, A. B., Wood, B. J., Tsai, C.-S., et al. (2021) · Nature Medicine · doi:10.1038/s41591-021-01506-3 · Cited by 842 · see citing works
A Framework of Potential Sources of Harm Throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. V. (2021) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 576 · see citing works
de Hond, A. A. H., Leeuwenberg, A. M., Hooft, L., Kant, I. M. J., Nijman, S. W. J., van Os, H. J. A., Aardoom, J. J., Debray, T. P. A., Schuit, E., van Smeden, M., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-021-00549-7 · Cited by 499 · see citing works
Prediction of the development of acute kidney injury following cardiac surgery by machine learning
Tseng, P.-Y., Chen, Y., Wang, C.-H., Chiu, K., Peng, Y.-S., Hsu, S., Chen, K.-L., Yang, C., & Lee, O. K. (2020) · Critical Care · doi:10.1186/s13054-020-03179-9 · Cited by 448 · see citing works
Machine Learning and Decision Support in Critical Care
Johnson, A. E. W., Ghassemi, M. M., Nemati, S., Niehaus, K. E., Clifton, D. A., & Clifford, G. D. (2016) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2501978 · Cited by 438 · see citing works
A review in radiomics: Making personalized medicine a reality via routine imaging
Guiot, J., Vaidyanathan, A., Deprez, L., Zerka, F., Danthine, D., Frix, A.-N., Lambin, P., Bottari, F., Tsoutzidis, N., Miraglio, B., et al. (2021) · Medicinal Research Reviews · doi:10.1002/med.21846 · Cited by 344 · see citing works
Ayaz, M., Pasha, M. F., Alzahrani, M. Y., Budiarto, R., & Stiawan, D. (2021) · JMIR Medical Informatics · doi:10.2196/21929 · Cited by 284 · cited 1× · see citing works
“For example, Fast Healthcare Interoperability Resources (FHIR) is a preformatted healthcare database that analytics platforms can easily access and deconstruct [8].”
BackgroundDelahanty, R. J., Alvarez, J., Flynn, L. M., Sherwin, R. L., & Jones, S. S. (2019) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2018.11.036 · Cited by 206 · see citing works
Artificial Intelligence in Critical Care Medicine
Yoon, J. H., Pinsky, M. R., & Clermont, G. (2022) · Critical Care · doi:10.1186/s13054-022-03915-3 · Cited by 143 · see citing works
Self-citationRajaraman, S., Ganesan, P., & Antani, S. (2022) · PLoS ONE · doi:10.1371/journal.pone.0262838 · Cited by 96 · see citing works
Radiomics in Lung Diseases Imaging: State-of-the-Art for Clinicians
Frix, A.-N., Cousin, F., Refaee, T., Bottari, F., Vaidyanathan, A., Desir, C., Vos, W. G., Walsh, S., Occhipinti, M., Lovinfosse, P., et al. (2021) · Journal of Personalized Medicine · doi:10.3390/jpm11070602 · Cited by 93 · see citing works
Machine learning in intensive care medicine: ready for take-off?
Collaborators, R. D. R. N., Fleuren, L. M., Thoral, P., Shillan, D., Ercole, A., & Elbers, P. W. G. (2020) · Intensive Care Medicine · doi:10.1007/s00134-020-06045-y · Cited by 88 · see citing works
Ayaz, M., Pasha, M. F., Alzahrani, M. Y., Budiarto, R., & Stiawan, D. (2021) · JMIR Medical Informatics · doi:10.2196/32869 · Cited by 74 · see citing works
Awan, S. E., Bennamoun, M., Sohel, F. A., Sanfilippo, F. M., Chow, B. J. W., & Dwivedi, G. (2019) · PLoS ONE · doi:10.1371/journal.pone.0218760 · Cited by 70 · see citing works
Prediction of hypotension events with physiologic vital sign signatures in the intensive care unit
Yoon, J. H., Jeanselme, V., Dubrawski, A. W., Hravnak, M., Pinsky, M. R., & Clermont, G. (2020) · Critical Care · doi:10.1186/s13054-020-03379-3 · Cited by 54 · see citing works
Self-citationWu, Y.-J., Liu, Y.-C., Liao, C.-Y., Tang, E., & Wu, F. (2021) · Scientific Reports · doi:10.1038/s41598-020-79690-4 · Cited by 39 · see citing works
van der Meijden, S. L., de Hond, A. A. H., Thoral, P. J., Steyerberg, E. W., Kant, I. M. J., Cinà, G., & Arbous, M. S. (2023) · JMIR Human Factors · doi:10.2196/39114 · Cited by 39 · see citing works
Thoral, P. J., Fornasa, M., de Bruin, D. P., Tonutti, M., Hovenkamp, H., Driessen, R. H., Girbes, A. R. J., Hoogendoorn, M., & Elbers, P. W. G. (2021) · Critical Care Explorations · doi:10.1097/cce.0000000000000529 · Cited by 34 · see citing works
Wu, T., Zheng, R. F., Lin, Z. Z., Gong, H. R., & Li, H. (2021) · BMC Emergency Medicine · doi:10.1186/s12873-021-00501-8 · Cited by 31 · see citing works
Abromavičius, V., Plonis, D., Tarasevičius, D., & Serackis, A. (2020) · Electronics · doi:10.3390/electronics9071133 · Cited by 30 · see citing works
Laird, P., Wertz, A., Welter, G., Maslove, D. M., Hamilton, A., Yoon, J. H., Lake, D. E., Zimmet, A. E., Bobko, R., Moorman, J. R., et al. (2021) · Physiological Measurement · doi:10.1088/1361-6579/abfc9b · Cited by 16 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- [19]Davis J Goadrich M The relationship between precision-recall and ROC curves Cohen WW Moore A Proceedings of the 23rd International Conference on Machine Learning 2006 Jun 25-29 Pittsburgh, PA, USA Association for Computing Machinery 2006 233 40
Cited by
Sorted by citations
Use of artificial intelligence in critical care: opportunities and obstacles
Pinsky, M. R., Bedoya, A., Bihorac, A., Celi, L., Churpek, M., Economou-Zavlanos, N. J., Elbers, P., Saria, S., Liu, V., Lyons, P. G., et al. (2024) · Critical Care · doi:10.1186/s13054-024-04860-z · Cited by 160 · see citing works
Explainable artificial intelligence in emergency medicine: an overview
Okada, Y., Ning, Y., & Ong, M. E. H. (2023) · Clinical and Experimental Emergency Medicine · doi:10.15441/ceem.23.145 · Cited by 73 · see citing works
Cheng, R. K.-H., Aggarwal, A. K., Chakraborty, A., Harish, V., McGowan, M., Roy, A., Szulewski, A., & Nolan, B. (2024) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2024.05.020 · Cited by 21 · see citing works
Hwang, S. Y., Kim, I., Jeong, D., Park, J.-E., Lee, G. T., Yoo, J., Choi, K., Shin, T. G., & Kim, K. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12196402 · Cited by 15 · see citing works
Harnessing AI in critical care: opportunities, challenges and key steps for success
Kalimouttou, A., Stevens, R. D., & Pirracchio, R. (2026) · Thorax · doi:10.1136/thorax-2024-222125 · Cited by 12 · see citing works
Irie, K., Minar, P., Reifenberg, J., Boyle, B. M., Noe, J. D., Hyams, J. S., & Mizuno, T. (2025) · Clinical Pharmacokinetics · doi:10.1007/s40262-025-01564-7 · Cited by 10 · see citing works
Clinical support system for triage based on federated learning for the Korea triage and acuity scale
Chang, H., Yu, J. Y., Lee, G. H., Heo, S., Lee, S. U., Hwang, S. Y., Yoon, H. J., Cha, W. C., Shin, T. G., Sim, M. S., et al. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e19210 · Cited by 8 · see citing works
Azzolina, D., Cammarota, G., Boero, E., Berchialla, P., Spadaro, S., Longhini, F., Deana, C., Biasucci, D. G., Stefano D’Incà, Batticci, I., et al. (2025) · Journal of Anesthesia Analgesia and Critical Care · doi:10.1186/s44158-025-00291-6 · Cited by 7 · see citing works
Chang, H., Park, J. E., Lee, D., Lee, K., Jekal, S. Y., Moon, K. T., Heo, S., Kim, D., Lee, G. T., Hwang, S. Y., et al. (2025) · Scientific Reports · doi:10.1038/s41598-025-07511-7 · Cited by 6 · see citing works
Heo, J. H., Suh, G. J., Park, J. H., Kim, J., Kim, K. H., Hwang, S. O., & Shin, S. D. (2023) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2023.07.044 · Cited by 4 · see citing works
Current Perspectives on the Artificial Intelligence in Critical Care Medicine
Lee, J., & Yoon, J. H. (2025) · Anesthesiology Clinics · doi:10.1016/j.anclin.2025.05.005 · Cited by 4 · see citing works
Self-citationHeo, S., Jeong, D., Choi, M., Kim, I., Kim, M., Lee, Y. R., Ko, B. S., Ryoo, S. M., Han, E., Chang, H., et al. (2024) · Emergencias · doi:10.55633/s3me/108.2024 · Cited by 2 · see citing works
Gutiérrez-Esparza, G. O., Martínez-García, M., Amezcua-Guerra, L. M., Rivera, M. M., & Lemus, E. H. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1805107 · Cited by 1 · see citing works
Irie, K., Minar, P., Reifenberg, J., Boyle, B. M., Noe, J. D., Hyams, J. S., & Mizuno, T. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.05.01.651780
Innovation and challenges in the implementation of artificial intelligence in critical care
Díaz, E. L. R., Rojas, M., Osorio-Valencia, J. S., Guerrero, N. A., Celi, L. A., & Pinilla-Forero, M. C. (2025) · Acta Colombiana de Cuidado Intensivo · doi:10.1016/j.acci.2025.04.007
Childhood PTSD and Trauma Treatment Using NLP, V/K Dissociation, and Machine Learning
Panda, S. P., Gupta, T., Kumar, A. R. K., Sachdeva, V., & Dubey, M. (2026) · Springer eBooks · doi:10.1007/978-3-032-09582-4_4
Rios-García, W., Broncano-Rivera, K., MariaFe-Martinez-Acuna, Via-y-Rada-Torres, A. D., Quintana-García, L. A., Marcilla, J. J. M., Mercado, C. A. S., & Rios-Garcia, A. A. (2026) · Pregnancy Hypertension · doi:10.1016/j.preghy.2026.101462
Kim, H. S., Shin, H. S., Goh, J., Jun, S. H., Lee, C. H., Yoon, J. H., & Chung, C. H. (2026) · BMC Anesthesiology · doi:10.1186/s12871-026-03907-8
Self-citationHong, T., Ren, Z., Fang, J., Quan, S., Pan, J., & Lin, Y. (2026) · JMIR Medical Informatics · doi:10.2196/81145
Putra, S. E., Hamidi, B. L., Benedictus, B., Hafizhan, M., Koeshermanto, T., Retnaningsih, R., & Machin, A. (2026) · Majalah Kesehatan Indonesia · doi:10.47679/makein.2026347
- Authors
- Kang, C.-Y., & Yoon, J. H.
- Affiliations
- John H. Stroger, Jr. Hospital of Cook County (US)
- University of Pittsburgh Medical Center (US)
- Corresponding author
- Yoon, Joo Heung
- ORCID
- Yoon, Joo Heung
- Venue
- Clinical and Experimental Emergency Medicine
- Published
- 2023-05-15
- Volume
- 10
- Issue
- 2
- Pages
- 132-137
- ISSN
- 2383-4625 (ISSN-L)
- PMC
- PMC10350350
- OpenAlex
- W4376642915
- Cited by
- 20
- References
- 28
- Citations by year
- 2026: 6 · 2025: 7 · 2024: 3 · 2023: 4
- FWCI
- 1.89
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Clinical and Experimental Emergency Medicine (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- US National National Institutes of Health (K23GM138984)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- machine learning adoption, intensive care, emergency medicine, clinical implementation, clinical data quality, feature construction, model design, performance evaluation, clinical translation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
