Explainable AI (XAI) In Biomedical Signal and Image Processing: Promises and Challenges
Abstract
Artificial intelligence has become pervasive across disciplines and fields, and biomedical image and signal processing is no exception. The growing and widespread interest on the topic has triggered a vast research activity that is reflected in an exponential research effort. Through study of massive and diverse biomedical data, machine and deep learning models have revolutionized various tasks such as modeling, segmentation, registration, classification and synthesis, outperforming traditional techniques. However, the difficulty in translating the results into biologically/clinically interpretable information is preventing their full exploitation in the field. Explainable AI (XAI) attempts to fill this translational gap by providing means to make the models interpretable and providing explanations. Different solutions have been proposed so far and are gaining increasing interest from the community. This paper aims at providing an overview on XAI in biomedical data processing and points to an upcoming Special Issue on Deep Learning in Biomedical Image and Signal Processing of the IEEE Signal Processing Magazine that is going to appear in March 2022.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The FDA has already signaled interest in adaptive AI that learns over time, proposing monitoring approaches to ensure safety is maintained during algorithm updates.
“The FDA has already signaled interest in adaptive AI that learns over time, proposing monitoring approaches to ensure safety is maintained during algorithm updates [58].”
From A Review of Multi-Agent AI Systems for Biological and Clinical Data Analysis (2026) · cites this paper as [58]
Cites
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · see citing works
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · see citing works
Explainable AI: a review of machine learning interpretability methods
Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. (2020) · Entropy · doi:10.3390/e23010018 · Cited by 2,922 · see citing works
Causability and explainability of artificial intelligence in medicine
Holzinger, A., Langs, G., Denk, H., Zatloukal, K., & Müller, H. (2019) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1312 · Cited by 1,885 · see citing works
Yang, G., Ye, Q., & Xia, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.07.016 · Cited by 717 · see citing works
Self-citationModel-Agnostic Interpretability of Machine Learning
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1606.05386 · Cited by 699 · see citing works
Spatial omics and multiplexed imaging to explore cancer biology
Lewis, S. M., Asselin-Labat, M.-L., Nguyen, Q. H., Berthelet, J., Tan, X., Wimmer, V. C., Mérino, D., Rogers, K. L., & Naik, S. H. (2021) · Nature Methods · doi:10.1038/s41592-021-01203-6 · Cited by 609 · see citing works
Nauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y., Schlötterer, J., van Keulen, M., & Seifert, C. (2023) · ACM Computing Surveys · doi:10.1145/3583558 · Cited by 590 · see citing works
Common brain disorders are associated with heritable patterns of apparent aging of the brain
Kaufmann, T., van der Meer, D., Doan, N. T., Schwarz, E., Lund, M. J., Agartz, I., Alnæs, D., Barch, D. M., Baur‐Streubel, R., Bertolino, A., et al. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0471-7 · Cited by 559 · see citing works
Cole, J. H. (2020) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2020.03.014 · Cited by 379 · see citing works
Böhle, M., Eitel, F., Weygandt, M., & Ritter, K. (2019) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2019.00194 · Cited by 291 · see citing works
Gray Matter Age Prediction as a Biomarker for Risk of Dementia
Wang, J. Z., Knol, M. J., Tiulpin, A., Dubost, F., de Bruijne, M., Vernooij, M. W., Adams, H. H. H., Ikram, M. A., Niessen, W. J., & Roshchupkin, G. V. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1902376116 · Cited by 211 · see citing works
Smith, S. M., Elliott, L. T., Alfaro‐Almagro, F., McCarthy, P., Nichols, T. E., Douaud, G., & Miller, K. L. (2020) · eLife · doi:10.7554/elife.52677 · Cited by 194 · see citing works
Toward a unified framework for interpreting machine-learning models in neuroimaging
Kohoutová, L., Heo, J., Cha, S., Lee, S., Moon, T., Wager, T. D., & Woo, C. (2020) · Nature Protocols · doi:10.1038/s41596-019-0289-5 · Cited by 168 · see citing works
The four dimensions of contestable AI diagnostics - A patient-centric approach to explainable AI
Ploug, T., & Holm, S. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101901 · Cited by 152 · see citing works
Explainable AI meets Healthcare: A Study on Heart Disease DatasetPreprint
Dave, D., Naik, H., Singhal, S., & Patel, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.03195 · Cited by 104 · see citing works
Levakov, G., Rosenthal, G., Shelef, I., Raviv, T. R., & Avidan, G. (2020) · Human Brain Mapping · doi:10.1002/hbm.25011 · Cited by 100 · see citing works
Feng, X., Lipton, Z. C., Yang, J., Small, S. A., & Provenzano, F. A. (2020) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2020.02.009 · Cited by 96 · see citing works
Accelerated MRI-predicted brain ageing and its associations with cardiometabolic and brain disorders
Kolbeinsson, A., Filippi, S. L., Panagakis, Y., Matthews, P. M., Elliott, P., Dehghan, A., & Tzoulaki, I. (2020) · Scientific Reports · doi:10.1038/s41598-020-76518-z · Cited by 63 · see citing works
Robust weakly supervised learning for COVID-19 recognition using multi-center CT images
Ye, Q., Gao, Y., Ding, W., Niu, Z., Wang, C., Jiang, Y., Wang, M., Fang, E. F., Menpes-Smith, W., Xia, J., et al. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.108291 · Cited by 37 · see citing works
Self-citationDiagnostic biases in translational bioinformatics
Han, H. (2015) · BMC Medical Genomics · doi:10.1186/s12920-015-0116-y · Cited by 24 · see citing works
Interpretable deep learning as a means for decrypting disease signature in multiple sclerosis
Cruciani, F., Brusini, L., Zucchelli, M., Pinheiro, G. R., Setti, F., Galazzo, I. B., Rachid Deriche, B., Rittner, L., Calabrese, M., & Menegaz, G. (2021) · Journal of Neural Engineering · doi:10.1088/1741-2552/ac0f4b · Cited by 24 · see citing works
Self-citationCGAT: Cell Graph ATtention Network for Grading of Pancreatic Disease Histology Images
Baranwal, M., Krishnan, S., Oneka, M. D., Frankel, T. L., & Rao, A. (2021) · Frontiers in Immunology · doi:10.3389/fimmu.2021.727610 · Cited by 12 · see citing works
Self-citationEditorial: Defining the Spatial Organization of Immune Responses to Cancer and Viruses In Situ
Phillips, D. J., Rodig, S. J., & Jiang, S. (2022) · Frontiers in Immunology · doi:10.3389/fimmu.2022.847582 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Review of Multi-Agent AI Systems for Biological and Clinical Data Analysis
Spieser, J., Balapour, A., Meller, J., Patra, K. C., & Shamsaei, B. (2026) · Methods and Protocols · doi:10.3390/mps9020033 · Cited by 1 · see citing works
“Research in XAI (explainable AI) is beginning to explore how to generate user-friendly explanations from multi-agent processes [58].”
- Authors
- Yang, G., Rao, A., Fernandez-Maloigne, C., Calhoun, V., & Menegaz, G.
- Affiliations
- Imperial College London (GB)
- University of Michigan (US)
- University of Poitiers and CNRS,I3M Laboratory,France
- Georgia Institute of Technology (US)
- University of Verona (IT)
- Venue
- 2022 IEEE International Conference on Image Processing (ICIP)
- Publisher
- IEEE
- Published
- 2022-10-16
- Pages
- 1531-1535
- ISSN
- 1522-4880 (ISSN-L)
- OpenAlex
- W4308235820
- Cited by
- 18
- References
- 24
- Citations by year
- 2026: 7 · 2025: 3 · 2024: 4 · 2023: 3 · 2022: 1
- FWCI
- 1.52
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - International Conference on Image Processing (publishedVersion)
- Funders
- University of Michigan
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Keywords
- explainable AI, biomedical image processing, biomedical signal processing, model interpretability, clinical interpretability, deep learning, medical image segmentation, medical image registration, medical image classification, medical image synthesis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
