Translational Potential and Explainability of Artificial Intelligence-Based Clinical Decision Support for Adults in Intensive Care: A Scoping Review
Open accessThe DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index; 1 of the 59 sources its claims cite were checked for support.
Abstract
This scoping review systematically mapped the translational potential and characteristics of explainable artificial intelligence (XAI) strategies in AI/ML-based clinical decision support tools for adult intensive care unit (ICU) settings. Methods: Following PRISMA-ScR guidelines, we searched Web of Science, PubMed, Scopus, and EBSCOhost up to January 2026. Translational potential was staged using an ICU-adapted, nine-level Technology Readiness Level (TRL) framework, and explainability strategies were classified as post-hoc or inherently interpretable (glass-box) to assess methodological transparency and clinical readiness. Results: A total of 808 records were identified, of which 29 studies met the inclusion criteria. The findings revealed a marked retrospective predominance (86.2%) and reliance on North American data, predominantly MIMIC (Medical Information Mart for Intensive Care). Tree-based ensembles (82.8%) and post-hoc SHAP explanations (86.2%) were dominant, with proposed clinical utility spanning three domains: therapeutic guidance, resource-allocation optimisation, and user-centric design. Most innovations were standalone, web-based prototypes requiring manual data entry (69.0%, TRL 4–5); a further 10.3% were shared only as open-source code, and only 17.2% reported integration with hospital systems. Only one study claimed clinical maturity (TRL 9), although its validation remained retrospective. Conclusion: Accuracy is no longer the primary bottleneck; the constraint has shifted to "last-mile" integration and external validity. Current XAI relies almost entirely on post-hoc methods that risk an "illusion of clarity", while inherently interpretable, glass-box models remain a rare but promising alternative. Future research should prioritise external validation in independent settings, prospective evaluation of clinical impact, and explicit comparison between post-hoc and interpretable approaches.
Overview
CoChat Summary
TL;DR
This scoping review mapped 29 AI/ML clinical decision support studies for adult ICU settings and found that most were retrospective, used tree-based ensembles, and relied on post-hoc SHAP explanations. The main bottleneck was not predictive accuracy but translational readiness: most tools were standalone prototypes, with only one study claiming clinical maturity and only 17.2% reporting hospital-system integration.
Key findings
- From 808 records, 29 studies met inclusion criteria; 25/29 (86.2%) were retrospective, and sample sizes ranged from 237 to 28,819 patients.
- Tree-based ensemble methods dominated the corpus at 24/29 (82.8%); deep learning appeared in 4 studies (13.8%), and only 1 study used a genuinely glass-box model (Explainable Boosting Machine).
- SHAP was the dominant explainability method, used in 25/29 studies (86.2%); most explanations were post-hoc rather than inherently interpretable.
- Translational maturity was low: 20/29 studies (69.0%) were web-based prototypes or online calculators (TRL 4–5), 3/29 (10.3%) shared open-source code, 5/29 (17.2%) integrated with hospital systems (TRL 7), and only 1/29 (3.4%) claimed clinical maturity (TRL 9), though it was validated retrospectively.
Methods
- Scoping review following PRISMA-ScR, searching Web of Science, PubMed/MEDLINE, Scopus, and EBSCOhost up to January 2026, with backward/forward citation tracking.
- Included primary quantitative studies on adult ICU AI/ML decision support that reported at least one translational indicator (eg, code, web tool, dashboard, calculator, or workflow/EHR integration); 2 reviewers screened/extracted, with additional review and supervision.
- Narrative synthesis across four domains: model architecture, explainability strategy, translational potential mapped to a nine-level ICU-adapted TRL framework, and proposed clinical utility.
Limitations
- Stated in the paper: no formal critical appraisal of study quality or risk of bias was performed; no meta-analysis was possible because of heterogeneity; no Cohen’s kappa/inter-reviewer agreement statistic was calculated.
- Stated in the paper: eligibility required at least one translational indicator, so the review may understate early-stage, non-deployed AI research in ICU XAI.
- Stated in the paper: the search strategy did not include explicit terms for inherently interpretable/glass-box models, so such studies may have been under-ascertained relative to post-hoc methods.
- Evident from the text: the evidence base is concentrated in a narrow set of datasets and countries, heavily relying on MIMIC/eICU and Chinese studies, which the authors note limits generalisability.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
1 Valid · 0 Not valid · 44 Not checked · 89 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Tree-based ensemble methods dominated, accounting for 24 of 29 studies (82.8%)”
§ Translational Potential and Explainability of Artificial Intelligence-Based Clinical Decision Support for Adults in Intensive Care: A Scoping Review > Result > Model Architecture
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Retrospective cohort designs predominated (25 studies, 86.2%)”
§ Translational Potential and Explainability of Artificial Intelligence-Based Clinical Decision Support for Adults in Intensive Care: A Scoping Review > Result > Characteristics of Included Studies
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“sample sizes ranged widely from 237 to 28,819 patients”
§ Translational Potential and Explainability of Artificial Intelligence-Based Clinical Decision Support for Adults in Intensive Care: A Scoping Review > Result > Characteristics of Included Studies
plainn = 28,819- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“20 studies (69.0%) supplemented these with local clinical data from the researchers’ institutions or regional collaborations”
§ Translational Potential and Explainability of Artificial Intelligence-Based Clinical Decision Support for Adults in Intensive Care: A Scoping Review > Result > Characteristics of Included Studies
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“23 studies (79.3%) drew on MIMIC-III/IV”
§ Translational Potential and Explainability of Artificial Intelligence-Based Clinical Decision Support for Adults in Intensive Care: A Scoping Review > Result > Characteristics of Included Studies
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
61 more claims
Create a free account to see all 134 claims and every source verdict.
- Probably not a substantial claim. 28
- It may state two separate claims as one. 51
- The quote only partly states this claim. 1
Citations
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation
Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., et al. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0850 · Cited by 43,565
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01626-4 · Cited by 13,874
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“12,13 Visual transparency for interdisciplinary ICU teams is currently delivered almost exclusively through the latter, with SHAP and LIME being the most widely adopted approaches; both extract explanations only after the underlying model has been trained and have been applied extensively to ICU…”
Sources of bias in artificial intelligence that perpetuate healthcare disparities—A global review
Celi, L. A., Cellini, J., Charpignon, M.-L., Dee, E. C., Dernoncourt, F., Eber, R., Mitchell, W. G., Moukheiber, L., Schirmer, J., Situ, J., et al. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000022 · Cited by 576 · cited 2×
“57,61 Federated learning has been proposed as a promising avenue for building more inclusive models across institutions without compromising patient data privacy, 62–64 although its application to ICU-based XAI remains largely unexplored in the studies reviewed.”
A Systematic Review of the Barriers to the Implementation of Artificial Intelligence in Healthcare
Ahmed, M. I., Spooner, B., Isherwood, J., Lane, M., Orrock, E., & Dennison, A. (2023) · Cureus · doi:10.7759/cureus.46454 · Cited by 364 · cited 1×
“5,9 The Current literature is highly heterogeneous in terms of model types, methodologies, and input data.”
Valentin, A., Ferdinande, P., & ESICM Working Group on Quality Improvement (2011) · Intensive Care Medicine · doi:10.1007/s00134-011-2300-7 · Cited by 331
Teo, Z. L., Jin, L., Liu, N., Li, S., Miao, D., Zhang, X., Ng, W. Y., Tan, T. F., Lee, D. M., Chua, K. J., et al. (2024) · Cell Reports Medicine · doi:10.1016/j.xcrm.2024.101419 · Cited by 255 · cited 1×
“57,61 Federated learning has been proposed as a promising avenue for building more inclusive models across institutions without compromising patient data privacy, 62–64 although its application to ICU-based XAI remains largely unexplored in the studies reviewed.”
Abbas, Q., Jeong, W., & Lee, S. W. (2025) · Healthcare · doi:10.3390/healthcare13172154 · Cited by 236 · cited 1×
“4 The findings of this review suggest that ICU-based AI development is beginning to reflect this concern, with a growing, though still partial, emphasis on decision-support tools that are more transparent, auditable, and oriented toward clinicians’ bedside needs, rather than on predictive…”
Should AI models be explainable to clinicians?
Abgrall, G., Holder, A. L., Chelly Dagdia, Z., Zeitouni, K., & Monnet, X. (2024) · Critical Care · doi:10.1186/s13054-024-05005-y · Cited by 175 · cited 1×
“52 Even within this review’s own corpus, SHAP outputs were frequently visualised through dense summaries and beeswarm plots that, without a sufficiently intuitive interface, risk adding to the cognitive burden of already fatigued ICU clinicians rather than relieving it.”
Interpretable and explainable machine learning: A methods‐centric overview with concrete examples
Marcinkevičs, R., & Vogt, J. E. (2023) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1493 · Cited by 161 · cited 1×
“13,15,16 This distinction is especially consequential in the ICU, where the accountability demanded of clinical decisions calls for a deeper understanding than a mere summary of model patterns or trends.”
I Think I Get Your Point, AI! The Illusion of Explanatory Depth in Explainable AI
Chromik, M., Eiband, M., Buchner, F., Krüger, A., & Butz, A. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450644 · Cited by 144 · cited 1×
“13,15,16 This distinction is especially consequential in the ICU, where the accountability demanded of clinical decisions calls for a deeper understanding than a mere summary of model patterns or trends.”
Romero-Brufau, S., Wyatt, K. D., Boyum, P., Mickelson, M., Moore, M., & Cognetta-Rieke, C. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.104072 · Cited by 143 · cited 1×
“65,66 In a fast-paced ICU environment, the need for manual data entry adds further cognitive burden and disrupts workflows that should remain centred on the patient.”
McWilliams, C. J., Lawson, D. J., Santos-Rodriguez, R., Gilchrist, I. D., Champneys, A., Gould, T. H., Thomas, M. J., & Bourdeaux, C. P. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-025925 · Cited by 89 · cited 6×
“5,32,46 The sheer volume of data generated by continuous monitoring often contributes to cognitive fatigue and administrative burden among clinical staff, underscoring the need for data-driven decision support systems that can process this information effectively.”
InfluentialSjoding, M. W., Taylor, D., Motyka, J., Lee, E., Co, I., Claar, D., McSparron, J. I., Ansari, S., Kerlin, M. P., Reilly, J. P., et al. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00056-x · Cited by 81 · cited 3×
“…to temporal or encounter-level data (SepsisFormer 48 and TECO 42 ) while two applied convolutional neural networks (CNNs) to different data types: Sjoding et al (2021) 10 used a DenseNet-121 CNN on chest radiographs for ARDS detection, the corpus’s only true imaging application, whereas Jia et…”
Prediction of weaning from mechanical ventilation using Convolutional Neural Networks
Jia, Y., Kaul, C., Lawton, T., Murray-Smith, R., & Habli, I. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102087 · Cited by 79 · cited 5×
“…networks (CNNs) to different data types: Sjoding et al (2021) 10 used a DenseNet-121 CNN on chest radiographs for ARDS detection, the corpus’s only true imaging application, whereas Jia et al (2021) 45 applied a CNN to ventilator waveform and physiological data to predict extubation readiness.”
InfluentialArtificial Intelligence in Intensive Care Medicine: Bibliometric Analysis
Tang, R., Zhang, S., Ding, C., Zhu, M., & Gao, Y. (2022) · Journal of Medical Internet Research · doi:10.2196/42185 · Cited by 79 · cited 1×
“54 55–57 Models trained on relatively homogeneous populations often generalise poorly when deployed in hospitals or countries with different genetic backgrounds, clinical workflows, and resource availability.”
Nguyen, T. V., Dakka, M. A., Diakiw, S. M., VerMilyea, M. D., Perugini, M., Hall, J. M. M., & Perugini, D. (2022) · Scientific Reports · doi:10.1038/s41598-022-12833-x · Cited by 77
Hong, N., Liu, C., Gao, J., Han, L., Chang, F., Gong, M., & Su, L. (2022) · JMIR Medical Informatics · doi:10.2196/28781 · Cited by 76 · cited 2×
“Confirming real-world benefits will still require prospective testing in ICU settings 73,74 built on local registries rather than continued reliance on external datasets such as MIMIC.”
To warrant clinical adoption AI models require a multi-faceted implementation evaluation
van de Sande, D., Chung, E. F. F., Oosterhoff, J., van Bommel, J., Gommers, D., & van Genderen, M. E. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01064-1 · Cited by 76
Berkhout, W. E. M., van Wijngaarden, J. J., Workum, J. D., van de Sande, D., Hilling, D. E., Jung, C., Meyfroidt, G., Gommers, D., Buijsman, S. N. R., & van Genderen, M. E. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.22866 · Cited by 75
Rockenschaub, P., Hilbert, A., Kossen, T., Elbers, P., von Dincklage, F., Madai, V. I., & Frey, D. (2024) · Critical Care Medicine · doi:10.1097/ccm.0000000000006359 · Cited by 73 · cited 1×
“56,60 This pattern is clearly visible within the present corpus: 22 of the 29 included studies (75.9”
InfluentialMachine learning identifies ICU outcome predictors in a multicenter COVID-19 cohort
Magunia, H., Lederer, S., Verbuecheln, R., Gilot, B. J., Koeppen, M., Haeberle, H. A., Mirakaj, V., Hofmann, P., Marx, G., Bickenbach, J., et al. (2021) · Critical Care · doi:10.1186/s13054-021-03720-4 · Cited by 65 · cited 7×
“…this landscape, employed in 25 of the 29 included studies (86.2%), 20–22, 24, 25, 35, 38, One study departed from this post-hoc paradigm entirely: Magunia et al (2021) 44 employed an Explainable Boosting Machine, a generalised additive model that is inherently interpretable by design and…”
InfluentialWhy do probabilistic clinical models fail to transport between sites
Lasko, T. A., Strobl, E. V., & Stead, W. W. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01037-4 · Cited by 63 · cited 1×
“58,59 This mismatch can produce substantial data drift and a corresponding drop in accuracy when applied to patients from a different institution altogether, posing a risk to patient safety wherever models are deployed without rigorous local validation.”
Caterson, J., Lewin, A., & Williamson, E. (2024) · Digital Health · doi:10.1177/20552076241272657 · Cited by 58 · cited 1×
“54 55–57 Models trained on relatively homogeneous populations often generalise poorly when deployed in hospitals or countries with different genetic backgrounds, clinical workflows, and resource availability.”
Artificial Intelligence in Sepsis Management: An Overview for Clinicians
Bignami, E. G., Berdini, M., Panizzi, M., Domenichetti, T., Bezzi, F., Allai, S., Damiano, T., & Bellini, V. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14010286 · Cited by 53
Algor-ethics: charting the ethical path for AI in critical care
Montomoli, J., Bitondo, M. M., Cascella, M., Rezoagli, E., Romeo, L., Bellini, V., Semeraro, F. F., Gamberini, E., Frontoni, E., Agnoletti, V., et al. (2024) · Journal of Clinical Monitoring and Computing · doi:10.1007/s10877-024-01157-y · Cited by 53 · cited 1×
“56,60 This pattern is clearly visible within the present corpus: 22 of the 29 included studies (75.9”
InfluentialYang, J., Clifton, L., Dung, N. T., Phong, N. T., Yen, L. M., Thy, D. B. X., Soltan, A. A. S., Thwaites, L., & Clifton, D. A. (2024) · Scientific Reports · doi:10.1038/s41598-024-64210-5 · Cited by 52 · cited 2×
“58,59 This mismatch can produce substantial data drift and a corresponding drop in accuracy when applied to patients from a different institution altogether, posing a risk to patient safety wherever models are deployed without rigorous local validation.”
Peek, N. B., Capurro, D., Rozova, V. S., & van der Veer, S. N. (2024) · Yearbook of Medical Informatics · doi:10.1055/s-0044-1800729 · Cited by 52
Lin, T., Chung, H., Jian, M., Chang, C.-K., Lin, H.-H., Yen, C.-T., Tang, S.-H., Pan, P.-C., Perng, C., Chang, F., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/56155 · Cited by 45
Zhu, H., Bai, J., Li, N., Li, X., Liu, D., Buckeridge, D. L., & Li, Y. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01661-8 · Cited by 43 · cited 1×
“Two applied transformer-based models to temporal or encounter-level data (SepsisFormer 48 and TECO 42 ) while two applied convolutional neural networks (CNNs) to different data types: Sjoding et al (2021) 10 used a DenseNet-121 CNN on chest radiographs for ARDS detection, the corpus’s only true…”
Is artificial intelligence prepared for the 24-h shifts in the ICU?
Gonzalez, F. A., Santonocito, C., Lamas, T., Costa, P. H. P., Vieira, S. M., Ferreira, H. A., & Sanfilippo, F. (2024) · Anaesthesia Critical Care & Pain Medicine · doi:10.1016/j.accpm.2024.101431 · Cited by 39
Rong, R., Gu, Z., Lai, H., Nelson, T. L., Keller, T., Walker, C., Jin, K. W., Chen, C., Návar, A. M., Velasco, F. T., et al. (2025) · JAMIA Open · doi:10.1093/jamiaopen/ooaf026 · Cited by 39
Machine Learning for Acute Kidney Injury Prediction in the Intensive Care Unit
Gottlieb, E. R., Samuel, M. C., Bonventre, J. V., Celi, L. A., & Mattie, H. (2022) · Advances in Chronic Kidney Disease · doi:10.1053/j.ackd.2022.06.005 · Cited by 38
Wang, S., Liu, X., Yuan, S., Bian, Y., Wu, H. M., & Ye, Q. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01643-w · Cited by 35
Hegselmann, S., Ertmer, C., Volkert, T., Gottschalk, A., Dugas, M., & Varghese, J. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.960296 · Cited by 32 · cited 1×
“53 This distinction matters: the future utility of AI in hospital settings will likely depend less on marginal accuracy gains and more on a system’s capacity to offer clinical reasoning that is honest, verifiable, and genuinely usable at the bedside.”
Embracing Artificial Intelligence: Revolutionizing Nursing Documentation for a Better Future
Yadav, S. (2024) · Cureus · doi:10.7759/cureus.57725 · Cited by 29 · cited 1×
“71 Taken together, the evidence synthesised in this review suggests that predictive accuracy is no longer the principal barrier to clinical AI in the ICU; the more pressing challenge lies in achieving seamless workflow integration and demonstrating robust external validity across diverse clinical…”
Hu, W., Jin, T., Pan, Z., Xu, H., Yu, L., Chen, T., Zhang, W., Jiang, H., Yang, W., Xu, J., et al. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107577 · Cited by 25 · cited 2×
“25,27,28,31 The TOPSIS-based classification fusion model of S. Wang et al (2025) was similarly classified here, as its core remains that of a gradient boosting decision tree.”
InfluentialDing, R., Deng, M., Wei, H., Zhang, Y., Wei, L., Jiang, G., Zhu, H., Huang, X., Fu, H., Zhao, S., et al. (2024) · CNS Neuroscience & Therapeutics · doi:10.1111/cns.14848 · Cited by 25 · cited 2×
“29,36”
InfluentialLeveraging interpretable machine learning in intensive care
Bohlen, L., Rosenberger, J., Zschech, P., & Kraus, M. (2024) · Annals of Operations Research · doi:10.1007/s10479-024-06226-8 · Cited by 24 · cited 1×
“50 Bohlen et al (2025) 51 reported that the performance gap between transparent and state-of-the-art black-box models is typically as small as 0.2–0.9%, a margin that is arguably too narrow to justify trading interpretability for accuracy alone.”
InfluentialGe, X., Chen, W., Shi, J., Zhang, J. F., Tai, H., Zhang, Y., Wang, B., Liu, W., Chen, S., & Han, H. (2025) · Journal of Medical Internet Research · doi:10.2196/73840 · Cited by 22 · cited 5×
“Tree-based ensemble methods dominated, accounting for 24 of 29 studies (82.8%), including XGBoost, 23,29,32,34,36,38,39 CatBoost, 22,30,33,35 LightGBM, 20,37,43 Random Forest, 24,46 and several stacking or fusion variants.”
InfluentialKim, Y. K., Koo, J. H., Lee, S. J., Song, H. S., & Lee, M. (2023) · Journal of Medical Internet Research · doi:10.2196/48244 · Cited by 22
Contreras, M., Silva, B., Shickel, B., Davidson, A., Ozrazgat‐Baslanti, T., Ren, Y., Guan, Z., Balch, J. A., Zhang, J., Bandyopadhyay, S., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-62121-1 · Cited by 20 · cited 1×
“67,69,70 Yet the reliability of any such integration remains contingent on the quality of bedside documentation performed by nursing staff, since AI-based decision support is only as trustworthy as the clinical data entered at the point of care.”
Dhami, A., Onyeukwu, K. A., Sattar, S., Batra, A., Mostafa, Y., Haris, M., Iqbal, A., Bokhari, S. F. H., & Siddique, M. U. (2025) · Cureus · doi:10.7759/cureus.90465 · Cited by 19 · cited 1×
“49 In such a high-pressure environment, clinicians tend to favour transparent, auditable models over black-box alternatives that offer only marginal accuracy improvements.”
Liu, Y., Xu, Y., Guo, L., Chen, Z., Xia, X., Chen, F., Tang, L., Jiang, H., & Xie, C. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03082-9 · Cited by 19 · cited 3×
“29,36”
InfluentialZhu, L., Chen, Z., Zhang, H., Chen, H., Liu, L., Yu, W., Wu, K., Chen, Y., Tao, X., Yu, Z., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-65365-z · Cited by 19 · cited 5×
“19,20 The analysis identified three main domains as the focus of development.”
InfluentialChen, J., Hou, D., & Song, Y. (2025) · Respiratory Research · doi:10.1186/s12931-025-03348-w · Cited by 18
Transforming critical care: the digital revolution's impact on intensive care units
Vernic, C., Tamaș, T.-P., Petre, I., & Ursoniu, S. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1664382 · Cited by 17 · cited 1×
“3 However, the greatest barrier to bedside adoption is not predictive accuracy alone but model transparency: many clinicians remain reluctant to trust AI systems they perceive as an impenetrable “black box”.”
Wei, S., Dong, H., Yao, W., Chen, Y., Wang, X., ji, W., Zhang, Y., & Guo, S. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03033-4 · Cited by 16 · cited 3×
“…tools requiring manual data entry, positioning them as early indicators of translational intent rather than as evidence of workflow integration, 22,35 and reducing this dependence on manual input will likely be necessary before such tools can function efficiently in time-pressured ICU settings.”
InfluentialWang, Z., Li, F. Y., Cai, J., Xue, Z., Zhou, Y., & Wang, Z. (2024) · Clinics and Research in Hepatology and Gastroenterology · doi:10.1016/j.clinre.2024.102507 · Cited by 14 · cited 2×
“Liu et al (2023) 39 released a ready-to-use application for triaging wild mushroom poisoning, S. Wang et al (2025) 41 provided code for a septic shock mortality model, and Jia et al (2021) 45 published the source code underlying their ventilator-weaning readiness model.”
Yang, M., Zhuang, J., Hu, W., Li, J., Wang, Y., Zhang, Z., Liu, C., & Chen, H. (2024) · Journal of Medical Internet Research · doi:10.2196/54621 · Cited by 14
Role of Artificial Intelligence in Critical Care Medicine: A Literature Review
ABDELBAKY, A., Elmasry, W. G., Awad, A. H., & Khan, S. (2025) · Cureus · doi:10.7759/cureus.90149 · Cited by 14
Gupta, J., Majumder, A. K., Sengupta, D., Sultana, M., & Bhattacharya, S. D. (2024) · Journal of Intensive Medicine · doi:10.1016/j.jointm.2024.04.006 · Cited by 12
Gu, K., & Lu, S. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03054-z · Cited by 12 · cited 1×
“25,27,28,31 The TOPSIS-based classification fusion model of S. Wang et al (2025) was similarly classified here, as its core remains that of a gradient boosting decision tree.”
Hong, Y., Zhang, X., Liu, K., Liu, Z., Chen, W., Liu, B., Ngai, E. W. T., & Hu, Y. Q. (2025) · Digital Health · doi:10.1177/20552076251360861 · Cited by 11 · cited 1×
“57,61 Federated learning has been proposed as a promising avenue for building more inclusive models across institutions without compromising patient data privacy, 62–64 although its application to ICU-based XAI remains largely unexplored in the studies reviewed.”
Żerdziński, K., Janiec, J., Jóźwik, K., Łajczak, P., & Krzych, Ł. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm15010185 · Cited by 11 · cited 5×
“5,9 The Current literature is highly heterogeneous in terms of model types, methodologies, and input data.”
InfluentialHuang, J., Liu, X., & Jin, W. (2024) · Digital Health · doi:10.1177/20552076241280126 · Cited by 10 · cited 1×
“8,11 Explainable AI (XAI) strategies for clinical decision support fall into two broad paradigms: models designed from the outset to be interpretable and post-hoc explanation methods applied to otherwise opaque “black-box” models.”
Athukorala, V. S., & Ilmini, W. M. K. S. (2026) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-026-03344-0 · Cited by 10 · cited 3×
“Yang et al (2026) 17 mapped 86 multimodal ICU AI studies but treated XAI only as a secondary theme, while Athukorala et al (2026) 14 focused specifically on XAI but restricted their scope to three clinical outcomes (sepsis onset, ICU readmission, and mortality).”
Early Triage of Critically Ill Adult Patients With Mushroom Poisoning: Machine Learning Approach
Liu, Y., Lyu, X., Yang, B., Fang, Z., Hu, D., Shi, L., Wu, B., Tian, Y., Zhang, E., & Yang, Y. (2023) · JMIR Formative Research · doi:10.2196/44666 · Cited by 10 · cited 3×
“Liu et al (2023) 39 released a ready-to-use application for triaging wild mushroom poisoning, S. Wang et al (2025) 41 provided code for a septic shock mortality model, and Jia et al (2021) 45 published the source code underlying their ventilator-weaning readiness model.”
InfluentialCurrent perspectives on the use of artificial intelligence in critical patient safety
Barea‐Mendoza, J. A., Fernández, M. V., Pardo, À., & Álvarez, J. G. (2024) · Medicina Intensiva (English Edition) · doi:10.1016/j.medine.2024.04.002 · Cited by 10 · cited 2×
“Automatic EHR integration is a starting point, ensuring that information reaches the team without manual-entry delays, 72,75 bedside documentation these systems depend on; management must prioritise investment in local data registries; and IT teams need to embed such solutions directly into…”
Ge, Y., Wang, G., Liu, T., Ji, W., Sun, J., & Zhang, Y. (2025) · BMC Pulmonary Medicine · doi:10.1186/s12890-025-03881-w · Cited by 8 · cited 2×
“Findings from this corpus reinforce that tree-based algorithms remain the dominant and pragmatic choice for modelling structured ICU data, accounting for 24 of the 29 included studies (82.8%), including XGBoost, 22,29,36 CatBoost, 30,35 and LightGBM 20,43 compared with only four studies employing deep learning archite…”
InfluentialRong, R., Gu, Z., Lai, H., Nelson, T. L., Keller, T., Walker, C., Jin, K. W., Chen, C., Navar, A., Velasco, F. T., et al. (2025) · medRxiv · doi:10.1101/2025.01.21.25320916 · Cited by 8 · cited 4×
“Two applied transformer-based models to temporal or encounter-level data (SepsisFormer 48 and TECO 42 ) while two applied convolutional neural networks (CNNs) to different data types: Sjoding et al (2021) 10 used a DenseNet-121 CNN on chest radiographs for ARDS detection, the corpus’s only true imaging application, wh…”
InfluentialAddressing Bias in AI Algorithms for Health Applications
Agnes, K. (2025) · IAA Journal of Biological Sciences · doi:10.59298/iaajb/2025/1313743 · Cited by 8 · cited 1×
“57,61 Federated learning has been proposed as a promising avenue for building more inclusive models across institutions without compromising patient data privacy, 62–64 although its application to ICU-based XAI remains largely unexplored in the studies reviewed.”
Wang, S., Qiu, K., Zheng, Q.-R., Zhou, B.-J., Li, M.-Y., Zhong, H., Chen, Y., & Yuan, S. (2025) · Frontiers in Cellular and Infection Microbiology · doi:10.3389/fcimb.2025.1586087 · Cited by 7 · cited 2×
“Liu et al (2023) 39 released a ready-to-use application for triaging wild mushroom poisoning, S. Wang et al (2025) 41 provided code for a septic shock mortality model, and Jia et al (2021) 45 published the source code underlying their ventilator-weaning readiness model.”
Cheng, Y., Guo, Y. S., Zhao, Y., Wang, C., Zhao, X., Yu, Q., Huang, J., Zhang, Y., Zhang, J., Liu, X., et al. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106203 · Cited by 4
Multimodal Artificial Intelligence for Precision Critical Care: A Scoping Review
Yang, M., Shi, N., Chen, H., Po, E. Y. P., Ng, P. Y., Lai, P. C. K., Wu, Y.-C., & Fong, D. Y. T. (2026) · Health Data Science · doi:10.34133/hds.0356 · Cited by 3 · cited 1×
“Yang et al (2026) 17 mapped 86 multimodal ICU AI studies but treated XAI only as a secondary theme, while Athukorala et al (2026) 14 focused specifically on XAI but restricted their scope to three clinical outcomes (sepsis onset, ICU readmission, and mortality).”
Liang, X., Zhao, W., Liufu, W., Qian, J., Xiang, N., Zhang, X., Zhou, J., & Cui, H. (2025) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2025.106082 · Cited by 3 · cited 2×
“22,34,39 User-Centric Design and Usability The successful implementation of AI depends heavily on clinician comfort and trust as end-users.”
InfluentialChen, M., Li, P., Xu, Y., Li, Z., Xiong, Y. Q., Wu, J., Pandya, C. J., Wang, Y., & Huang, G. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03301-3 · Cited by 3 · cited 1×
“21,43”
Bai, X., Huang, S., Huang, S., Wang, B., Zhu, A., Qi, S., Gao, Y., Zhu, H., Jiang, T., Zhang, B., et al. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106260 · Cited by 2 · cited 3×
“32,46 The implementation of this principle in everyday practice depends on the coordinated efforts of the care team.”
InfluentialBuilding Interpretable AI Models for Healthcare Decision Support
Pasupuleti, M. K. (2025) · International Journal of Academic and Industrial Research Innovations(IJAIRI) · doi:10.62311/nesx/rphcr16 · Cited by 2 · cited 1×
“50 Bohlen et al (2025) 51 reported that the performance gap between transparent and state-of-the-art black-box models is typically as small as 0.2–0.9%, a margin that is arguably too narrow to justify trading interpretability for accuracy alone.”
InfluentialYang, R., Huang, T., Yao, R.-Q., Wang, D., Hu, Y., Ren, L., Li, S., Zhao, Y., & Dai, Z. (2025) · Clinical Medicine · doi:10.1016/j.clinme.2025.100299 · Cited by 2 · cited 2×
“Shapley Additive exPlanations (SHAP) dominated this landscape, employed in 25 of the 29 included studies (86.2%), 20–22, 24, 25, 35, 38, One study departed from this post-hoc paradigm entirely: Magunia et al (2021) 44 employed an Explainable Boosting Machine, a generalised additive model that is…”
InfluentialWang, Y., Li, W., Cui, J., Wang, Z., & Li, Y. (2025) · Injury · doi:10.1016/j.injury.2025.112949 · Cited by 2 · cited 1×
“25,27,28,31 The TOPSIS-based classification fusion model of S. Wang et al (2025) was similarly classified here, as its core remains that of a gradient boosting decision tree.”
Artificial intelligence – new horizons in intensive care
Kiran, Q., & Naseer, A. (2025) · Anaesthesia Pain & Intensive Care · doi:10.35975/apic.v29i5.2848 · Cited by 1 · cited 2×
“Automatic EHR integration is a starting point, ensuring that information reaches the team without manual-entry delays, 72,75 bedside documentation these systems depend on; management must prioritise investment in local data registries; and IT teams need to embed such solutions directly into…”
Yang, C., Ma, H., Zeng, X., Yang, J., Xiao, B., Tan, R., Liu, Y., & Zeng, Q. (2026) · Clinical and Applied Thrombosis/Hemostasis · doi:10.1177/10760296251408357 · Cited by 1 · cited 2×
“Yang et al (2026) 17 mapped 86 multimodal ICU AI studies but treated XAI only as a secondary theme, while Athukorala et al (2026) 14 focused specifically on XAI but restricted their scope to three clinical outcomes (sepsis onset, ICU readmission, and mortality).”
InfluentialAn examination of the justification for post hoc explanations of artificial intelligence
Hu, D. (2025) · AI & Society · doi:10.1007/s00146-025-02742-8 · Cited by 1 · cited 1×
“13,15,16 This distinction is especially consequential in the ICU, where the accountability demanded of clinical decisions calls for a deeper understanding than a mere summary of model patterns or trends.”
Ma, M., Liu, J., Li, C., Chen, Y., Jia, H., Hou, A., & Xu, H. (2025) · European journal of medical research · doi:10.1186/s40001-025-02543-1 · Cited by 1 · cited 6×
“Tree-based ensemble methods dominated, accounting for 24 of 29 studies (82.8%), including XGBoost, 23,29,32,34,36,38,39 CatBoost, 22,30,33,35 LightGBM, 20,37,43 Random Forest, 24,46 and several stacking or fusion variants.”
InfluentialHorvat, C. M., Suresh, S., & Clark, R. S. B. (2018) · Informatics · doi:10.3390/informatics5030037 · Cited by 1 · cited 1×
“67,69,70 Yet the reliability of any such integration remains contingent on the quality of bedside documentation performed by nursing staff, since AI-based decision support is only as trustworthy as the clinical data entered at the point of care.”
Cited by
cheng, Z., Ma, X., Wang, H., xiao, B., Cui, J., & Liang, Z. (2026) · Research Square · doi:10.21203/rs.3.rs-10947791/v1
Metadata
- Authors
- Pranata, Y., Mirwanti, R., Bahtiar, Y. A., & Purwanti, H.
- Affiliations
- Indonesia Open University (ID)
- Padjadjaran University (ID)
- Corresponding author
- Mirwanti, Ristina
- Venue
- Journal of Multidisciplinary Healthcare
- Publisher
- Informa UK Limited
- Published
- 2026-07
- Volume
- Volume 19
- Pages
- 1-16
- ISSN
- 1178-2390 (ISSN-L)
- PMC
- PMC13387200
- OpenAlex
- W7169497460
- Semantic Scholar
- 9f9d0c2f31c571aad0a8b13c0422f7ad4e223710
- Cited by
- 1
- References
- 69
- Citations by year
- 2026: 1
- FWCI
- 0.46
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Journal of Multidisciplinary Healthcare (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by-nc)
- DOAJ (DOAJ: Directory of Open Access Journals) (acceptedVersion)
- Dove Medical Press (Taylor and Francis Group) (submittedVersion, cc-by-nc)
- Funders
- Lembaga Pengelola Dana Pendidikan (4303/ B3/DT.03.08/2025, 3927/UN6. RKT/HK.07.00/2025)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- explainable AI, clinical decision support, intensive care unit, technology readiness level, translational potential, post-hoc explanation, SHAP, tree-based ensemble models, external validation, clinical integration, resource allocation, adult critical care
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

