Explainability, transparency and black box challenges of AI in radiology: impact on patient care in cardiovascular radiology
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
The integration of artificial intelligence (AI) in cardiovascular imaging has revolutionized the field, offering significant advancements in diagnostic accuracy and clinical efficiency. However, the complexity and opacity of AI models, particularly those involving machine learning (ML) and deep learning (DL), raise critical legal and ethical concerns due to their "black box" nature. This manuscript addresses these concerns by providing a comprehensive review of AI technologies in cardiovascular imaging, focusing on the challenges and implications of the black box phenomenon. We begin by outlining the foundational concepts of AI, including ML and DL, and their applications in cardiovascular imaging. The manuscript delves into the "black box" issue, highlighting the difficulty in understanding and explaining AI decision-making processes. This lack of transparency poses significant challenges for clinical acceptance and ethical deployment. The discussion then extends to the legal and ethical implications of AI's opacity. The need for explicable AI systems is underscored, with an emphasis on the ethical principles of beneficence and non-maleficence. The manuscript explores potential solutions such as explainable AI (XAI) techniques, which aim to provide insights into AI decision-making without sacrificing performance. Moreover, the impact of AI explainability on clinical decision-making and patient outcomes is examined. The manuscript argues for the development of hybrid models that combine interpretability with the advanced capabilities of black box systems. It also advocates for enhanced education and training programs for healthcare professionals to equip them with the necessary skills to utilize AI effectively. Patient involvement and informed consent are identified as critical components for the ethical deployment of AI in healthcare. Strategies for improving patient understanding and engagement with AI technologies are discussed, emphasizing the importance of transparent communication and education. Finally, the manuscript calls for the establishment of standardized regulatory frameworks and policies to address the unique challenges posed by AI in healthcare. By fostering interdisciplinary collaboration and continuous monitoring, the medical community can ensure the responsible integration of AI into cardiovascular imaging, ultimately enhancing patient care and clinical outcomes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“This has raised concerns among some experts who suggest that the use of unexplainable models should be stopped due to the significant problems they may pose [38, 42].”
BackgroundHigh-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450 · cited 1×
“Such indications can pose challenges in clinical settings because clinicians may not fully understand or trust the AI’s outputs, which can impact patient care [25, 26].”
BackgroundBarredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Energy and Policy Considerations for Deep Learning in NLP
Strubell, E., Ganesh, A., & McCallum, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1355 · Cited by 3,684 · cited 1×
“The "black box" nature in AI models Despite these benefits, the complexity of AI models, particularly deep learning methods, poses significant challenges [23, 24].”
BackgroundRevolutionizing healthcare: the role of artificial intelligence in clinical practice
Alowais, S. A., Alghamdi, S. S., Alsuhebany, N., Alqahtani, T., Alshaya, A. I., Almohareb, S. N., Aldairem, A., Alrashed, M., Bin Saleh, K., Badreldin, H. A., et al. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04698-z · Cited by 3,146 · cited 1×
“Future work should explore standardized consent frameworks that can be adapted across various healthcare settings to ensure consistency and thorough-ness [76].”
BackgroundAn Introduction to Variational Autoencoders
Kingma, D. P., & Welling, M. (2019) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000056 · Cited by 2,609 · cited 1×
“Variational autoencoders (VAEs) are a type of generative model that learns a latent representation of input data and can reconstruct input data from a compressed latent space [65].”
BackgroundThe practical implementation of artificial intelligence technologies in medicine
He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2019) · Nature Medicine · doi:10.1038/s41591-018-0307-0 · Cited by 2,246 · cited 1×
“Such indications can pose challenges in clinical settings because clinicians may not fully understand or trust the AI’s outputs, which can impact patient care [25, 26].”
BackgroundExplainability for artificial intelligence in healthcare: a multidisciplinary perspective
the Precise4Q consortium, Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01332-6 · Cited by 2,203 · cited 1×
“Clinicians need to understand not only how to use AI tools but also how these tools work, their limitations, and the ethical considerations involved [69].”
BackgroundDefinitions, methods, and applications in interpretable machine learning
Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900654116 · Cited by 2,167 · cited 2×
“Although efforts are underway to develop mechanisms for contex-tualizing ML predictions based on user needs, achieving full comprehension of AI predictions remains an evolving research frontier [43].”
BackgroundExplaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3060483 · Cited by 1,477 · cited 1×
“This ambiguity can undermine healthcare practitioners’ confidence in relying on AI predictions, particularly when they conflict with conventional clinical judgment [40].”
BackgroundExplainable artificial intelligence (XAI) in deep learning-based medical image analysis
van der Velden, B. H. M., Kuijf, H. J., Gilhuijs, K. G. A., & Viergever, M. A. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102470 · Cited by 1,328
Artificial intelligence in cardiology
Johnson, K. W., Torres Soto, J., Glicksberg, B. S., Shameer, K., Miotto, R., Ali, M., Ashley, E., & Dudley, J. T. (2018) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2018.03.521 · Cited by 1,202
Olah, C., Mordvintsev, A., & Schubert, L. (2017) · Distill · doi:10.23915/distill.00007 · Cited by 880
Artificial Intelligence and Black‐Box Medical Decisions: Accuracy versus Explainability
London, A. J. (2019) · Hastings Center Report · doi:10.1002/hast.973 · Cited by 801
Machine learning in cardiovascular medicine: are we there yet?
Shameer, K., Johnson, K. W., Glicksberg, B. S., Dudley, J. T., & Sengupta, P. P. (2018) · Heart · doi:10.1136/heartjnl-2017-311198 · Cited by 555 · cited 1×
“The rise of smart devices and electronic medical records has expanded data availability, enhancing ML algorithm performance despite data complexity [6].”
BackgroundAl’Aref, S. J., Anchouche, K., Singh, G., Slomka, P. J., Kolli, K. K., Kumar, A., Pandey, M., Maliakal, G., van Rosendael, A. R., Beecy, A. N., et al. (2019) · European Heart Journal · doi:10.1093/eurheartj/ehy404 · Cited by 534
What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use
Tonekaboni, S., Joshi, S., McCradden, M. D., & Goldenberg, A. (2019) · arXiv · doi:10.48550/arxiv.1905.05134 · Cited by 318
Cestonaro, C., Delicati, A., Marcante, B., Caenazzo, L., & Tozzo, P. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1305756 · Cited by 231
Explainable and interpretable artificial intelligence in medicine: a systematic bibliometric review
Frasca, M., La Torre, D., Pravettoni, G., & Cutica, I. (2024) · Discover Artificial Intelligence · doi:10.1007/s44163-024-00114-7 · Cited by 190
AI in Medical Education: Global situation, effects and challenges
Zhang, W., Cai, M., Lee, H. J., Evans, R., Zhu, C., & Ming, C. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12009-8 · Cited by 148
Stakeholders in Explainable AI
Preece, A., Harborne, D., Braines, D., Tomsett, R., & Chakraborty, S. (2018) · arXiv · doi:10.48550/arxiv.1810.00184 · Cited by 117
Application of mobile health, telemedicine and artificial intelligence to echocardiography
Seetharam, K., Kagiyama, N., & Sengupta, P. P. (2019) · Echo Research & Practice · doi:10.1530/erp-18-0081 · Cited by 111 · cited 1×
“Advances in computational power and big data have propelled ML’s application in healthcare [5].”
BackgroundGlobal and Local Interpretability for Cardiac MRI Classification
Clough, J. R., Oksuz, I., Puyol-Antón, E., Ruijsink, B., King, A. P., & Schnabel, J. A. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-32251-9_72 · Cited by 73
Artificial Intelligence in Cardiovascular Imaging: “Unexplainable” Legal and Ethical Challenges?
Lang, M., Bernier, A., & Knoppers, B. M. (2022) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2021.10.009 · Cited by 37
Cited by
Onciul, R., Tataru, C.-I., Dumitru, A. V., Crivoi, C., Serban, M., Covache-Busuioc, R.-A., Radoi, M. P., & Toader, C. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14020550 · Cited by 113
“Efforts to incorporate XAI tools, such as integrated gradients and SHAP (Shapley Additive Explanations), are underway but have yet to see widespread application in single-cell research [164].”
Explainable Artificial Intelligence in Radiological Cardiovascular Imaging—A Systematic Review
Haupt, M., Maurer, M. H., & Thomas, R. P. (2025) · Diagnostics · doi:10.3390/diagnostics15111399 · Cited by 58
Explainable and Interpretable AI for Voice and Speech Analysis in Clinical Care: Systematic Review
Ebraheem, M., Toghranegar, J., Bridge2AI-Voice Consortium, Bensoussan, Y., & Templeton, J. M. (2026) · Journal of Medical Internet Research · doi:10.2196/83790
Metadata
- Authors
- Marey, A., Arjmand, P., Alerab, A. D. S., Eslami, M. J., Saad, A. M., Sanchez, N., & Umair, M.
- Venue
- Egyptian Journal of Radiology and Nuclear Medicine
- Published
- 2024-09-13
- Volume
- 55
- Issue
- 1
- ISSN
- 0378-603X (ISSN-L)
- 2090-4762
- OpenAlex
- W4402524908
- Cited by
- 196
- References
- 25
- FWCI
- 29.17
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.