To explain or not to explain?—Artificial intelligence explainability in clinical decision support systems
Open accessThe DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index.
Abstract
Explainability for artificial intelligence (AI) in medicine is a hotly debated topic. Our paper presents a review of the key arguments in favor and against explainability for AI-powered Clinical Decision Support System (CDSS) applied to a concrete use case, namely an AI-powered CDSS currently used in the emergency call setting to identify patients with life-threatening cardiac arrest. More specifically, we performed a normative analysis using socio-technical scenarios to provide a nuanced account of the role of explainability for CDSSs for the concrete use case, allowing for abstractions to a more general level. Our analysis focused on three layers: technical considerations, human factors, and the designated system role in decision-making. Our findings suggest that whether explainability can provide added value to CDSS depends on several key questions: technical feasibility, the level of validation in case of explainable algorithms, the characteristics of the context in which the system is implemented, the designated role in the decision-making process, and the key user group(s). Thus, each CDSS will require an individualized assessment of explainability needs and we provide an example of how such an assessment could look like in practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

![Fig 2 Z-Inspection® process. This figure has been reproduced from Zicari RV, Brodersen J, Brusseau J, Düdder B, Eichhorn T, Ivanov T, et al. Z-Inspection®: A Process to Assess Trustworthy AI. IEEE Trans Technol Soc. 2021 Jun;2(2):83–97. [56].](https://figures.cochat.ai/paper-figures/c71a414b2c984bfea375d14b7dd18a96/to-explain-or-not-to-explain-artificial-intelligence-explainability-in-clinical-figure-2.jpg)
![Fig 3 Ethical issue identified during the initial assessment of the use case [31].](https://figures.cochat.ai/paper-figures/4314a4502f78427c86c49fe0ba120ef6/to-explain-or-not-to-explain-artificial-intelligence-explainability-in-clinical-figure-3.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While interpretability is increasingly recognized as essential for the clinical integration of AI, further prospective and real-world studies are needed to quantify its impact on patient outcomes and define implementation best practices429,430.”
From Artificial intelligence-driven multi-omics approaches in Alzheimer's disease: Progress, challenges, and future directions (2025) · cites this paper as [429]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In particular, the framework in [137] aims at avoiding situations that can have dire consequences for patients.”
From Connecting the dots in trustworthy Artificial Intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulation (2023) · cites this paper as [137]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The ongoing debate about explainability trade-offs [52,53], building on earlier work [43-46], underscores that selecting suitable XAI methods requires careful consideration of the context-balancing interpretability needs against performance constraints and workflow pressures.”
From Explainable AI for Clinical Decision Support Systems: Literature Review, Key Gaps, and Research Synthesis (2025) · cites this paper as [52]
The cited source says what the claim attributes to it.
In the citing paper
“as Amann et al. (2022) point out, transparency is not one-dimensional: “Explainability is but one measure of achieving transparency; other measures include detailed documentation of datasets and algorithms, as well as open communication of the system’s capabilities and limitations” (Amann et al., 2022).”
From An auditable and source-verified framework for clinical AI decision support: integrating retrieval-augmented generation with data provenance (2026) · cites this paper as Amann et al., 2022
The cited source says what the claim attributes to it.
In the citing paper
“leading studies have repeatedly identified that lack of transparency is a major barrier to AI-CDS adoption (Tun et al., 2025; Amann et al., 2022).”
From An auditable and source-verified framework for clinical AI decision support: integrating retrieval-augmented generation with data provenance (2026) · cites this paper as Amann et al., 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · cited 4×
“An explanation for a black-box model can then, for example, be obtained through an interpretable approximation (see, e.g., SHAP [32] or LIME [33]).”
MethodInfluentialGauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 3×
“Due to these limitations in current techniques for generating explanations, some experts suggest the use of inherently transparent models such as regression, decision trees, or rule lists [45,50].”
BackgroundBarredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 1×
“We aimed to capture the various disciplinary perspectives on explainability, ranging from philosophy, to medicine, to computer science, and law [14,17,18,59,60].”
BackgroundThe global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 1×
“An often referenced way to foster trust is to increase the system’s transparency [12], and an important part of increasing transparency is the application of explainability [13].”
BackgroundLipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427 · cited 5×
“When referring to different algorithms that can achieve explainability, there are unfortunately a multitude of different terms in the field that are frequently used interchangeably and without a fixed definition, especially across disciplines [22,25,28].”
BackgroundInfluentialExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Research works printed but not linked: 3
- Karimi A-H, Barthe G, Balle B, Valera I. Model-Agnostic Counterfactual Explanations for Consequential Decisions. In: International Conference on Artificial Intelligence and Statistics [Internet]. PMLR; 2020 [cited 2021 Jun 15]. p. 895–905. Available from: http://proceedings.mlr.press/v108/karimi20a.html
- Aubakirova M, Bansal M. Interpreting Neural Networks to Improve Politeness Comprehension. ArXiv161002683 Cs [Internet]. 2016 Oct 9 [cited 2021 Apr 13]; Available from: http://arxiv.org/abs/1610.02683 · arXiv
- Karlekar S, Niu T, Bansal M. Detecting Linguistic Characteristics of Alzheimer’s Dementia by Interpreting Neural Models. ArXiv180406440 Cs [Internet]. 2018 Apr 17 [cited 2021 Apr 13]; Available from: http://arxiv.org/abs/1804.06440 · arXiv
Cited by
Díaz-Rodríguez, N., Del Ser, J., Coeckelbergh, M., López de Prado, M., Herrera-Viedma, E., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101896 · Cited by 800
A Scoping Review of Reporting Gaps in FDA-Approved AI Medical Devices
Muralidharan, V., Adewale, B. A., Huang, C. J., Nta, M. T., Ademiju, P. O., Pathmarajah, P., Hang, M. K., Adesanya, O., Abdullateef, R. O., Babatunde, A. O., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01270-x · Cited by 201
Gala, D., Behl, H., Shah, M., & Makaryus, A. N. (2024) · Healthcare · doi:10.3390/healthcare12040481 · Cited by 183
“Explainable AI methods aim to provide transparency in AI-driven decision making, offering insights into how algorithms arrive at specific conclusions [61].”
Solving the Explainable AI Conundrum by Bridging Clinicians’ Needs and Developers’ Goals
Bienefeld, N., Boss, J. M., Lüthy, R., Brodbeck, D., Azzati, J., Blaser, M., Willms, J., & Keller, E. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00837-4 · Cited by 159
Caruccio, L., Cirillo, S., Polese, G., Solimando, G., Sundaramurthy, S., & Tortora, G. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121186 · Cited by 153
Sivaraman, V., Bukowski, L. A., Levin, J., Kahn, J. M., & Perer, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581075 · Cited by 141
“However, none of these methods appear to work universally across contexts, particularly in health care; in some domains, they may infate or undermine confdence [31, 40, 90], while in others they may be poorly-aligned with human decision-making processes [2, 32].”
BackgroundDey, M. T. (2024) · Advances in Environmental Engineering and Green Technologies · doi:10.4018/979-8-3693-7822-9.ch012 · Cited by 131
EXplainable Artificial Intelligence (XAI)—From Theory to Methods and Applications
Ortigossa, E. S., Gonçalves, T., & Nonato, L. G. (2024) · IEEE Access · doi:10.1109/access.2024.3409843 · Cited by 124
“Amann et al. [16] claimed such an interpretability reduction might be related to the overhead induced by the large amount of information users are exposed to during the understanding process, even in transparent models.”
BackgroundMethodInfluentialAI for Decision Support: Balancing Accuracy, Transparency, and Trust Across Sectors
Kővári, A. (2024) · Information · doi:10.3390/info15110725 · Cited by 213
Amann, J., Vayena, E., Ormond, K. E., Frey, D., Madai, V. I., & Blasimme, A. (2023) · PLOS ONE · doi:10.1371/journal.pone.0279088 · Cited by 106
Self-citationZhang, S., Yu, J., Xu, X., Yin, C., Lu, Y., Yao, B., Tory, M., Padilla, L. M., Caterino, J., Zhang, P., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642343 · Cited by 99
The prospect of artificial intelligence to personalize assisted reproductive technology
Hanassab, S., Abbara, A., Yeung, A. C., Voliotis, M., Tsaneva-Atanasova, K. T., Kelsey, T., Trew, G. H., Nelson, S. M., Heinis, T., & Dhillo, W. S. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01006-x · Cited by 125
Deep Learning Approaches for Medical Image Analysis and Diagnosis
Thakur, G. K., Thakur, A., Kulkarni, S., Khan, N., & Khan, S. (2024) · Cureus · doi:10.7759/cureus.59507 · Cited by 167
Ren, F., Wei, J., Chen, Q., Hu, M., Yu, L., Mi, J., Zhou, X., Qin, D., Wu, J., & Wu, A. (2025) · Acta Pharmaceutica Sinica B · doi:10.1016/j.apsb.2025.07.030 · Cited by 97
What Is the Role of Explainability in Medical Artificial Intelligence? A Case-Based Approach
Hildt, E. (2025) · Bioengineering · doi:10.3390/bioengineering12040375 · Cited by 83
“Overall, the debate is complicated by the fact that none of these concepts are well defined [9,25,28,34].”
InfluentialMedical artificial intelligence for clinicians: the lost cognitive perspective
Tikhomirov, L., Semmler, C., McCradden, M., Searston, R., Ghassemi, M., & Oakden-Rayner, L. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00095-5 · Cited by 63
Neri, E., Aghakhanyan, G., Zerunian, M., Gandolfo, N. G., Grassi, R., Miele, V., Giovagnoni, A., Laghi, A., & SIRM expert group on Artificial Intelligence (2023) · La radiologia medica · doi:10.1007/s11547-023-01634-5 · Cited by 71
“An interesting approach to the evaluation of the explainability of an AI system is the one called Z-Inspection; an initiative to assess trustworthy AI in practice [31, 32].”
BackgroundThe use of AI in mental health services to support decision-making: scoping review
Auf, H., Svedberg, P., Nygren, J., Nair, M., & Lundgren, L. E. (2025) · Journal of Medical Internet Research · doi:10.2196/63548 · Cited by 56
“It highlights the need for efficient and explainable AI that can facilitate communication within a short span of time and tailored to the individualized care processes considering both technical and end-user preferences [79].”
Bashkirova, A., & Krpan, D. (2024) · Computers in Human Behavior: Artificial Humans · doi:10.1016/j.chbah.2024.100066 · Cited by 55
Health Care Professionals’ Experience of Using AI: Systematic Review With Narrative Synthesis
Ayorinde, A., Mensah, D. O., Walsh, J., Ghosh, I., Ibrahim, S. A., Hogg, J., Peek, N., & Griffiths, F. (2024) · Journal of Medical Internet Research · doi:10.2196/55766 · Cited by 55
Artificial intelligence to revolutionize IBD clinical trials: a comprehensive review
Sedaño, R., Solitano, V., Vuyyuru, S. K., Yuan, Y., Hanžel, J., Ma, C., Nardone, O. M., & Jairath, V. (2025) · Therapeutic Advances in Gastroenterology · doi:10.1177/17562848251321915 · Cited by 68
Bayor, A. A., Li, J., Yang, I. A., & Varnfield, M. (2025) · Journal of Medical Internet Research · doi:10.2196/63733 · Cited by 46
“…systems offer promising solutions to address trust and validation challenges in CDSS. XAI enhances transparency by making AI systems’ inner workings more interpretable for health care professionals, enabling them to identify potential flaws and assess the trustworthiness of system outputs [83].”
Fedele, A., Punzi, C., & Tramacere, S. (2024) · Computer law & security review · doi:10.1016/j.clsr.2024.105986 · Cited by 76
Dalvi‐Esfahani, M., Mosharaf‐Dehkordi, M., Leong, L. W., Ramayah, T., & Kanaan-Jebna, A. M. J. (2023) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2023.122768 · Cited by 53
Iacucci, M., Santacroce, G., Maeda, Y., & Ghosh, S. (2025) · Gastroenterology · doi:10.1053/j.gastro.2025.03.005 · Cited by 56
Perumal, M. K. K., Renuka, R. R., Kumar, S. S., & Natarajan, P. M. (2025) · Frontiers in Oral Health · doi:10.3389/froh.2025.1592428 · Cited by 72
A review on artificial intelligence for the diagnosis of fractures in facial trauma imaging
Pham, T. D., Holmes, S., & Coulthard, P. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1278529 · Cited by 47
Grand challenges in industrial and systems engineering
Karwowski, W., Salvendy, G., Albert, L. A., Kim, W. C., Denton, B. T., Dessouky, M. M., Dolgui, A. B., Duffy, V. G., Kumara, S. R. T., Li, J., et al. (2025) · International Journal of Production Research · doi:10.1080/00207543.2024.2432463 · Cited by 52
“For instance, AI plays a crucial role in supporting decision-making in healthcare and Medicare (Vasey et al. 2022; Amann et al. 2022; Čartolovni, Tomičić, and Mosler 2022), enabling more accurate diagnoses, personalised treatments, and efficient patient care.”
Alfeo, A. L., Zippo, A. G., Catrambone, V., Cimino, M. G. C. A., Toschi, N., & Valenza, G. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107550 · Cited by 38
Intelligent decision support systems for dementia care: A scoping review
Andargoli, A. E., Ulapane, N. N. B., Nguyen, T. A., Shuakat, M. N., Zelcer, J., & Wickramasinghe, N. S. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102815 · Cited by 50
Sahoo, R. K., Sahoo, K. C., Negi, S., Baliarsingh, S. K., Panda, B., & Pati, S. (2025) · Patient Education and Counseling · doi:10.1016/j.pec.2025.108680 · Cited by 51
Salimparsa, M., Sedig, K., Lizotte, D. J., Abdullah, S. S., Chalabianloo, N., & Muanda, F. T. (2025) · Informatics · doi:10.3390/informatics12040119 · Cited by 39
Huo, W., Zhang, Z., Qu, J., Yan, J., Yan, S., Yan, J., & Shi, B. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2023.2176985 · Cited by 35
“However, it may also unintentionally cause many issues, such as algorithmic “ black boxes ” and a lack of transparency (Amann et al., 2022; Holzinger et al., 2022).”
BackgroundGiacobbe, D. R., Marelli, C., Guastavino, S., Mora, S., Rosso, N., Signori, A., Campi, C., Giacomini, M. L., & Bassetti, M. (2024) · Clinical Therapeutics · doi:10.1016/j.clinthera.2024.02.010 · Cited by 38
Miedema, E., Waschull, S., & Emmanouilidis, C. (2025) · Computers in Industry · doi:10.1016/j.compind.2025.104409 · Cited by 48
Artificial Intelligence in the Pathology of Gastric Cancer
Choi, S., & Kim, S. (2023) · Journal of the Korean Gastric Cancer Association · doi:10.5230/jgc.2023.23.e25 · Cited by 40
“Despite the sophistication of the algorithm, the mainstay of DL algorithms remains recognized as a “black box” and can be affected by the Clever Hans effect [96-98]; this hinders the analysis of errors even by software developers and cannot account for the legal issues.”
BackgroundRecommendations for using artificial intelligence in clinical flow cytometry
Ng, D. P., Simonson, P. D., Tárnok, A., Lucas, F. M., Kern, W., Rolf, N., Bogdanoski, G., Green, C., Brinkman, R. R., & Czechowska, K. (2024) · Cytometry Part B Clinical Cytometry · doi:10.1002/cyto.b.22166 · Cited by 38
“Within the confines of diagnostic clinical laboratory flow cytometry, some clinical pathologists feel that explainability is not strictly necessary (Amann et al., 2022; Ghassemi et al., 2021).”
Painting the Black Box White: Experimental Findings from Applying XAI to an ECG Reading Setting
Cabitza, F., Campagner, A., Natali, C., Parimbelli, E., Ronzio, L., & Cameli, M. (2023) · Machine Learning and Knowledge Extraction · doi:10.3390/make5010017 · Cited by 41
Lessons Learned from Assessing Trustworthy AI in Practice
Vetter, D., Amann, J., Bruneault, F., Coffee, M. P., Düdder, B., Gallucci, A., Gilbert, T. K., Hagendorff, T., van Halem, I., Hickman, E., et al. (2023) · Digital Society · doi:10.1007/s44206-023-00063-1 · Cited by 43
Self-citationYu, Y., Gomez-Cabello, C. A., Haider, S. A., Genovese, A., Prabha, S., Trabilsy, M., Collaco, B. G., Wood, N. G., Bagaria, S., Tao, C., et al. (2025) · Diagnostics · doi:10.3390/diagnostics15172204 · Cited by 32
“…responses, this study addresses one of the most significant impediments to the effective implementation of AI-decision support tools: low clinician trust due to uncertainty about their accuracy and reliability, while considering confidence calibrations to mitigate clinician overreliance [22,23].”
Bongurala, A. R., Save, D., & Virmani, A. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1533910 · Cited by 32
State-of-the-Art in Responsible, Explainable, and Fair AI for Medical Image Analysis
Amirian, S., Gao, F., Littlefield, N. G., Hill, J. H., Yates, A. J., Plate, J. F., Pantanowitz, L., Rashidi, H. H., & Tafti, A. P. (2025) · IEEE Access · doi:10.1109/access.2025.3555543 · Cited by 37
“In this way, REF-AI explainability bridges the gap between complex AI systems and end-users and makes more ethical and informed use of AI, supporting decision-making in high-stakes areas like medical imaging analysis [22], [135], [136], [137], [138], [139].”
AI-Supported Shared Decision-Making (AI-SDM): Conceptual Framework
As’ad, M., Faran, N., & Joharji, H. (2025) · JMIR AI · doi:10.2196/75866 · Cited by 50
“This technical transparency, though important for trust [6], does not equate to the human-centered, contextual reasoning required for SDM.”
Giacobbe, D. R., Zhang, Y., & de la Fuente, J. (2023) · Annals of Medicine · doi:10.1080/07853890.2023.2286336 · Cited by 31
Coupland, H., Scheidwasser, N., Katsiferis, A., Davies, M., Flaxman, S., Hulvej Rod, N., Mishra, S., Bhatt, S., & Unwin, H. J. T. (2025) · BMC Public Health · doi:10.1186/s12889-025-22705-4 · Cited by 26
“However, XAI methods remain limited in their ability to establish causal relationships, an issue that is especially pronounced in time-series data where confounding and sparsity complicate interpretability [33].”
Artificial intelligence and its application in clinical microbiology
Mairi, A., Hamza, L., & Touati, A. (2025) · Expert Review of Anti-infective Therapy · doi:10.1080/14787210.2025.2484284 · Cited by 43
“For AI to be trusted in healthcare, particularly for critical decisions, enhancing transparency in how these models function is essential [142].”
Azadi, A., & García‐Peñalvo, F. J. (2025) · JMIR Human Factors · doi:10.2196/69333 · Cited by 45
Piloting a Survey-Based Assessment of Transparency and Trustworthiness with Three Medical AI Tools
Fehr, J., Jaramillo-Gutierrez, G., Oala, L., Gröschel, M. I., Bierwirth, M., Balachandran, P., Leite, A. W., & Lippert, C. (2022) · Healthcare · doi:10.3390/healthcare10101923 · Cited by 37
“Similar to a previous study [35], it was challenging to judge if all the potential sources of bias, causes of harm, and caveats for deployment were sufficiently investigated.”
MethodResultGiacobbe, D. R., Marelli, C., Manna, B. L., Padua, D., Malva, A., Guastavino, S., Signori, A., Mora, S., Rosso, N., Campi, C., et al. (2025) · npj Antimicrobials and Resistance · doi:10.1038/s44259-025-00084-5 · Cited by 32
The quality and utility of artificial intelligence in patient care
Wehkamp, K., Krawczak, M., & Schreiber, S. (2023) · Deutsches Ärzteblatt international · doi:10.3238/arztebl.m2023.0124 · Cited by 37
Vila-Blanco, N., Varas-Quintana, P., Aneiros-Ardao, Á., Tomás, I., & Carreira, M. J. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106072 · Cited by 24
Zuheros, C., Martínez‐Cámara, E., Herrera‐Viedma, E., Katib, I. A., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101821 · Cited by 23
Human-Controllable AI: Meaningful Human Control
Liu, C., & Xu, W. (2025) · Springer eBooks · doi:10.1007/978-981-97-8440-0_58-1 · Cited by 26
Tan, R., Cobuccio, L. G., Beynon, F., Levine, G. A., Vaezipour, N., Luwanda, L. B., Mangu, C., Vonlanthen, A., Santis, O. D., Salim, N., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000170 · Cited by 25
AI-driven multimodal colorimetric analytics for biomedical and behavioral health diagnostics
Hagos, D. H., Aryal, S. K., Ymele‐Leki, P., & Burge, L. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.05.015 · Cited by 29
Nobile, M. S., Capitoli, G., Sowirono, V., Clerici, F., Piga, I., van Abeelen, K. C. J., Magni, F., Pagni, F., Galimberti, S., Cazzaniga, P., et al. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.119296 · Cited by 20
Thomas, J., Lucht, A., Segler, J., Wundrack, R., Miché, M., Lieb, R., Kuchinke, L., & Meinlschmidt, G. (2024) · JMIR Public Health and Surveillance · doi:10.2196/63809 · Cited by 28
Leonardsen, E. H., Persson, K., Grødem, E., Dinsdale, N. K., Schellhorn, T., Roe, J. M., Dídac Vidal-piñeiro, A., Sørensen, Ø., Kaufmann, T., Westman, E., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01123-7 · Cited by 33
Clinical Applications of Machine Learning
Mateussi, N., Rogers, M. P., Grimsley, E. A., Read, M. D., Parikh, R., Pietrobon, R. S., & Kuo, P. C. (2024) · Annals of Surgery Open · doi:10.1097/as9.0000000000000423 · Cited by 21
“8,9 ML-powered AI methods are increasingly applied in the form of CDSSs to assist health-care professionals in predicting patient outcomes 9,10 as well as to evaluate the implementation of CDSSs. 8”
van Buchem, M. M., de Hond, A., Fanconi, C., Shah, V. B., Schuessler, M., Kant, I. M. J., Steyerberg, E. W., & Hernandez‐Boussard, T. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae188 · Cited by 24
Can large language models assist with pediatric dosing accuracy?
Levin, C., Orkaby, B., Kerner, E., & Saban, M. (2025) · Pediatric Research · doi:10.1038/s41390-025-03980-8 · Cited by 20
“It underscores the importance of model-speci fi c evaluation rather than generalizing capabilities across all LLMs. 21 The signi fi cant differences observed between nurses and LLMs in their responses to most medication calculation questions highlight the complexity of this task and the varying…”
InfluentialVelandia, H., García, A. P., Vera, M. I., & Vera, M. (2025) · Bioengineering · doi:10.3390/bioengineering12111248 · Cited by 34
Nittas, V., Daniore, P., Landers, C., Gille, F., Amann, J., Hubbs, S., Puhan, M. A., Vayena, E., & Blasimme, A. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000189 · Cited by 20
Self-citationOrtuño-Miró, S., Molina‐Rodríguez, S., Belmonte, C., & Ibáñez-Ballesteros, J. (2023) · Journal of Neural Engineering · doi:10.1088/1741-2552/acad2b · Cited by 20
Health technology assessment framework for artificial intelligence-based technologies
Bidino, R. D., Daugbjerg, S. B., Papavero, S. C., Haraldsen, I., Cicchetti, A., & Sacchini, D. (2024) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462324000308 · Cited by 31
Gawade, V., & Chen, M. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113095 · Cited by 17
Căliman-Sturdza, O. A., Filip, F., Baitan, M. T., & Dimian, M. (2025) · Diagnostics · doi:10.3390/diagnostics15141830 · Cited by 18
Tourab, H., Lopez-Perez, L., Arroyo, P., Georga, E. I., Rujas, M., Ponziani, F. R., Torrego-Ellacuría, M., Barbancho, B. M., Dei, N. N., Ciuti, G., et al. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3593198 · Cited by 21
“Moreover, as it has been mentioned before, explainability is a measure for achieving transparency, with other measures that include giving a thorough documentation of the datasets and algorithms employed [92].”
Real-time AI in Clinical Decision Support: Bridging Research and Practice
Veernapu, K., Patil, B. K., Tharayil, A. S., Sindhura, C. L., Andy, A., Budhewar, A. S., Somwanshi, S. K., Kavya, K., Bharadwaj, M., & Raj, P. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-6400-1.ch015 · Cited by 16
Advancing cardiovascular care through actionable AI innovation
Biondi‐Zoccai, G., Mahajan, A., Powell, D., Peruzzi, M., Carnevale, R., & Frati, G. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01621-2 · Cited by 21
Justice and the Normative Standards of Explainability in Healthcare
Kempt, H., Freyer, N., & Nagel, S. K. (2022) · Philosophy & Technology · doi:10.1007/s13347-022-00598-0 · Cited by 15
Algorithmic legitimacy in clinical decision-making
Holm, S. (2023) · Ethics and Information Technology · doi:10.1007/s10676-023-09709-7 · Cited by 18
“, Amann et al. (2022), Lipton (2018) and Rudin (2019) for discussion.”
BackgroundSelf-citationTransforming clinical cardiology through neural networks and deep learning: A guide for clinicians
Sutanto, H. (2024) · Current Problems in Cardiology · doi:10.1016/j.cpcardiol.2024.102454 · Cited by 17
Explaining decisions without explainability? Artificial intelligence and medicolegal accountability
McCradden, M. D., & Stedman, I. (2024) · Future Healthcare Journal · doi:10.1016/j.fhj.2024.100171 · Cited by 18
“12 We disagree, for the reasons listed above.”
BackgroundAlu, F. F., & Oluwadare, S. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1737532 · Cited by 22
“This opacity can erode trust; indeed, leading studies have repeatedly identified that lack of transparency is a major barrier to AI-CDS adoption (Tun et al., 2025; Amann et al., 2022).”
InfluentialUihlein, A., Beissel, L., Ajlani, A. H., Orzechowski, M., Leinert, C., Kocar, T. D., Pankratz, C., Schuetze, K., Gebhard, F. T., Steger, F., et al. (2024) · JMIR Aging · doi:10.2196/57899 · Cited by 19
“Acceptance of an AI-CDSS is closely connected to explainability, accountability, and trust [39].”
Buhr, E., Onder, O., Rudra, P., & Ursin, F. (2025) · Ethics and Information Technology · doi:10.1007/s10676-025-09867-w · Cited by 18
“Amann et al. (2022) adopt a similar view, while Ferrario and Loi (2022) further stress that explainability promotes trust only when it enables justified paradigmatic trust by reducing the need for monitoring.”
McCradden, M. D., Thai, K., Assadi, A., Tonekaboni, S., Stedman, I., Joshi, S. D., Zhang, M., Chevalier, F., & Goldenberg, A. (2025) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2024-112919 · Cited by 21
Roychowdhury, S., Lanfranchi, V., & Mazumdar, S. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110944 · Cited by 21
Liao, X., Yao, C., Zhang, J., & Liu, L. Z. (2023) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12549 · Cited by 19
Machine learning innovations in CPR: a comprehensive survey on enhanced resuscitation techniques
Islam, S., Rjoub, G., Elmekki, H., Bentahar, J., Pedrycz, W., & Cohen, R. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11214-w · Cited by 16
“This fits well with the concept of algorithm-based or algorithm-driven deci - sions (and not what is known as algorithm-determined decisions) per a distinction made by the German Data Ethics Committee (Amann et al. 2022 ) and reflected on at length in the FDA report of the US (Food and (FDA) 2021 ).”
Yesankar, P., Puri, C., & Gote, P. M. (2025) · International Conference on Machine Learning and Autonomous Systems (ICMLAS) · doi:10.1109/icmlas64557.2025.10969014 · Cited by 12
“In the critical care units, CDSS monitors patients on intensive therapies and predicts potential deteriorations and enables clinicians to make changes in real time to prevent life-threatening events [14].”
The transformative role of Artificial Intelligence in genomics: Opportunities and challenges
Dara, M., Dianatpour, M., Azarpira, N., & Tanideh, N. (2025) · Gene Reports · doi:10.1016/j.genrep.2025.102314 · Cited by 18
Use of Computation Ecosystems to Analyze the Kidney-Heart Crosstalk
Wu, Z., Lohmöller, J., Kuhl, C. K., Wehrle, K., & Jankowski, J. (2023) · Circulation Research · doi:10.1161/circresaha.123.321765 · Cited by 12
“180 Differentiating the initiator from the overall effects of an organ crosstalk interaction is largely sidestepped due to the sparsity of data in the temporal dimension.”
BackgroundBenzinger, L., Epping, J., Ursin, F., & Salloch, S. (2024) · BMC Medical Ethics · doi:10.1186/s12910-024-01079-z · Cited by 15
“Explicability is a highly debated topic regarding CDSS in general [42].”
BackgroundMeirelles, A. L. S., Kurç, T., Kong, J., Ferreira, R., Saltz, J. H., & Teodoro, G. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.894430 · Cited by 15
“Also, applications developed with deep learning should consider explainability to improve confidence in their use (13, 14).”
BackgroundAdvancing Alzheimer's Therapy: Computational strategies and treatment innovations
Paul, J. K., Malik, A., Azmal, M., Gulzar, T., Afghan, M. T. R., Talukder, O. F., Shahzadi, S., & Ghosh, A. (2025) · IBRO Neuroscience Reports · doi:10.1016/j.ibneur.2025.02.002 · Cited by 10
Development of a Machine Learning-Based Model to Predict Timed-Up-and-Go Test in Older Adults
Kraus, M., Stumpf, U. C., Keppler, A. M., Neuerburg, C., Böcker, W., Wackerhage, H., Baumbach, S. F., & Saller, M. M. (2023) · Geriatrics · doi:10.3390/geriatrics8050099 · Cited by 12
“Statements about the most important influencing factors in the utilized models could also be made, thus fulfilling the demand for explainable AI in clinical decision support systems [50].”
BackgroundSankar, S., & Sathyalakshmi, S. (2024) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2024.048408 · Cited by 12
Statistical learning and big data applications
Witte, H., Blatter, T. U., Nagabhushana, P., Schär, D., Ackermann, J., Cadamuro, J., & Leichtle, A. B. (2023) · Journal of Laboratory Medicine · doi:10.1515/labmed-2023-0037 · Cited by 9
Nzeako, T. R., Elendu, C., Echefu, G. C., Olanisa, O., Kiladejo, A., & Bob-Manuel, E. D. (2025) · Annals of Medicine and Surgery · doi:10.1097/ms9.0000000000003602 · Cited by 11
“Sensitivity: 85%; AUC: 79.7%; accuracy: 75% – strong mortality stratification Generalizability concerns and need to address data bias [64] Functional assessment via cardiac MRI Automated strain analysis for GLS and GCS”
InfluentialLee, H., Kim, J., Kwak, S., Rehman, A., Park, S. M., & Chang, J. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01957-9 · Cited by 9
Unveiling the Algorithm: The Role of Explainable Artificial Intelligence in Modern Surgery
Lopes, S., Mascarenhas, M., Fonseca, J., Fernandes, G., & Leite-Moreira, A. F. (2025) · Healthcare · doi:10.3390/healthcare13243208 · Cited by 14
The Future of Antibiotics and Artificial Intelligence: Some Thoughts from Discovery to Bedside
Giacobbe, D. R., Grossi, A. A., Bassetti, M., & de la Fuente‐Núñez, C. (2025) · Infectious Diseases and Therapy · doi:10.1007/s40121-025-01288-y · Cited by 12
“Explainability essentially means providing reliable explanations as to why a model made a certain prediction or provided a certain suggestion (e.g., administering a specific antibiotic to a specific patient) [57].”
Thomsen, C. H. N., Kronborg, T., Hangaard, S., Vestergaard, P., Hejlesen, O. K., & Jensen, M. H. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105758 · Cited by 7
Eneva, Y., & Doğan, B. (2025) · Journal of Lifestyle and SDGs Review · doi:10.47172/2965-730x.sdgsreview.v5.n02.pe03545 · Cited by 10
Artificial Intelligence in Cardiovascular Diseases
Bashir, S., Ahanger, A. N., Assad, A., Bhat, M. R., & Macha, M. A. (2025) · Springer eBooks · doi:10.1007/978-981-96-8176-1_10 · Cited by 11
A Human-Centered Approach to Interpretable Machine Learning in Clinical Decision Support Systems
Nimbalkar, V., Chawla, L., Adnan, M. M., Bhansali, A., Gupta, M., & Kalra, R. (2025) · International Conference on Artificial Intelligence for Innovations in Healthcare Industries (ICAIIHI) · doi:10.1109/icaiihi67124.2025.11403473 · Cited by 7
Consorti, G., Catarzi, L., Frosolini, A., Vaira, L. A., Committeri, U., & Cirignaco, G. (2026) · International Journal of Oral and Maxillofacial Surgery · doi:10.1016/j.ijom.2026.02.020 · Cited by 11
When AI is wrong: addressing liability challenges in women's healthcare
Marotta, A. (2022) · Journal of Computer Information Systems · doi:10.1080/08874417.2022.2089773 · Cited by 7
“However, unlike traditional systems, CDSS are highly complex, and their functioning is generally hard to explain.(39) This complexity raises concerns about misuse of AI tools and consequent responsibility attribution.”
Background
Metadata
- Authors
- Amann, J., Vetter, D., Blomberg, S. N., Christensen, H. C., Coffee, M., Gerke, S., Gilbert, T. K., Hagendorff, T., Holm, S., Livne, M., Spezzatti, A., Strümke, I., Zicari, R. V., Madai, V. I., & initiative, O. B. O. T. Z.-I.
- Affiliations
- ETH Zurich (CH)
- Goethe University Frankfurt (DE)
- University of Copenhagen (DK)
- New York University (US)
- Pennsylvania State University (US)
- Cornell Tech (US)
- University of Tübingen (DE)
- Google (United Kingdom) (GB)
- University of California, Berkeley (US)
- Norwegian University of Science and Technology (NO)
- Seoul National University (KR)
- Birmingham City University (GB)
- Corresponding author
- Amann, Julia
- ORCID
- Venue
- PLOS Digital Health
- Publisher
- Public Library of Science (PLoS)
- Published
- 2022-02-17
- Volume
- 1
- Issue
- 2
- Pages
- e0000016
- ISSN
- 2767-3170 (ISSN-L)
- PMC
- PMC9931364
- OpenAlex
- W4212942200
- Cited by
- 251
- References
- 62
- Citations by year
- 2026: 75 · 2025: 70 · 2024: 55 · 2023: 35 · 2022: 16
- FWCI
- 14.49
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLOS Digital Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research at the University of Copenhagen (University of Copenhagen) (publishedVersion, cc-by)
- eYLS (Yale Law School) (submittedVersion, cc-by)
- eYLS (Yale Law School) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
- Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
- ETH Zürich Research Collection
- Funders
- Horizon 2020 Framework Programme (777107, 101016233, H2020-SC1-PHE-CORONAVIRUS-2020-2-RTD)
- Connecting Europe Facility (INEA/CEF/ICT/A2020/2276680, 2020-EU-IA-0098)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- clinical decision support systems, cardiac arrest prediction, normative analysis, human factors, explainable algorithms, user groups
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.