A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Open accessThe DOI matches its Crossref record; 193 of the 193 references it cites were found in a registry or an index.
Abstract
Recently, artificial intelligence and machine learning in general have demonstrated remarkable performances in many tasks, from image processing to natural language processing, especially with the advent of deep learning (DL). Along with research progress, they have encroached upon many different fields and disciplines. Some of them require high level of accountability and thus transparency, for example, the medical sector. Explanations for machine decisions and predictions are thus needed to justify their reliability. This requires greater interpretability, which often means we need to understand the mechanism underlying the algorithms. Unfortunately, the blackbox nature of the DL is still unresolved, and many machine decisions are still poorly understood. We provide a review on interpretabilities suggested by different research works and categorize them. The different categories show different dimensions in interpretability research, from approaches that provide "obviously" interpretable information to the studies of complex patterns. By applying the same categorization to interpretability in medical research, it is hoped that: 1) clinicians and practitioners can subsequently approach these methods with caution; 2) insight into interpretability will be born with more considerations for medical practices; and 3) initiatives to push forward data-based, mathematically grounded, and technically grounded medical education are encouraged.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“XAI methods, therefore, hold significant potential to enhance the implementation, utility, and adoption of AI in CDSS.”
From Explainable AI for Clinical Decision Support Systems: Literature Review, Key Gaps, and Research Synthesis (2025) · cites this paper as [9]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A crucial aspect of these advancements is the explainability of AI-driven decisions, which is essential for guiding IoDT development and refining AI algorithms [140,141].”
From Digital Twins in the IIoT: Current Practices and Future Directions Toward Industry 5.0 (2025) · cites this paper as [140]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While substantial progress has been made in model explainability for image-related tasks, fewer studies address explainability in text processing or multimodal tasks in medicine29.”
From Large language models and multimodal foundation models for precision oncology (2024) · cites this paper as [29]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This trust can exist on two levels: trust in the model versus trust in the prediction [14].”
From Explainable Artificial Intelligence (XAI): Concepts and Challenges in Healthcare (2023) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This represents a critical knowledge gap, as the explainability challenges of modern LLMs differ fundamentally from those covered by existing healthcare XAI reviews, which predominantly focus on traditional machine learning methods applied to structured clinical data or medical imaging (e.g., [34]).”
From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [34]
Experimental — accuracy not yet measured
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Gauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation
Milletari, F., Navab, N., & Ahmadi, S.-A. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.79 · Cited by 11,931
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · cited 2×
“implementations of machine learning models; consider [10], [14]–[18].”
BackgroundMethod3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
Çiçek, Ö., Abdulkadir, A., Lienkamp, S. S., Brox, T., & Ronneberger, O. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_49 · Cited by 7,945
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
(2017) · International Conference on Machine Learning · Cited by 6,382
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
U-Net: Convolutional Networks for Biomedical Image SegmentationPreprint
Ronneberger, O., Fischer, P., & Brox, T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.04597 · Cited by 5,282
Artificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087
Striving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447
Hastie, T., & Tibshirani, R. (1986) · Statistical Science · doi:10.1214/ss/1177013604 · Cited by 3,409
Canonical Correlation Analysis: An Overview with Application to Learning Methods
Hardoon, D. R., Szedmak, S., & Shawe-Taylor, J. (2004) · Neural Computation · doi:10.1162/0899766042321814 · Cited by 3,202
Key challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121
Learning Important Features Through Propagating Activation Differences
Shrikumar, A., Greenside, P., & Kundaje, A. (2017) · arXiv · doi:10.48550/arxiv.1704.02685 · Cited by 2,761
Methods for interpreting and understanding deep neural networks
Montavon, G., Samek, W., & Müller, K.-R. (2018) · Digital Signal Processing · doi:10.1016/j.dsp.2017.10.011 · Cited by 2,665
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507
Causability and explainability of artificial intelligence in medicine
Holzinger, A., Langs, G., Denk, H., Zatloukal, K., & Müller, H. (2019) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1312 · Cited by 1,885
Understanding deep image representations by inverting them
Mahendran, A., & Vedaldi, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299155 · Cited by 1,827
Explaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731
Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission
Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788613 · Cited by 1,685
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514
SmoothGrad: removing noise by adding noise
Smilkov, D., Thorat, N., Kim, B., Viégas, F., & Wattenberg, M. (2017) · arXiv · doi:10.48550/arxiv.1706.03825 · Cited by 1,467
Quantum-chemical insights from deep tensor neural networks
Schütt, K. T., Arbabzadah, F., Chmiela, S., Müller, K. R., & Tkatchenko, A. (2017) · Nature Communications · doi:10.1038/ncomms13890 · Cited by 1,445
Deep learning and the information bottleneck principle
Tishby, N., & Zaslavsky, N. (2015) · 2015 IEEE Information Theory Workshop (ITW) · doi:10.1109/itw.2015.7133169 · Cited by 1,409
On the interpretation of weight vectors of linear models in multivariate neuroimaging
Haufe, S., Meinecke, F., Görgen, K., Dähne, S., Haynes, J.-D., Blankertz, B., & Bießmann, F. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.10.067 · Cited by 1,389
Network Dissection: Quantifying Interpretability of Deep Visual Representations
Bau, D., Zhou, B., Khosla, A., Oliva, A., & Torralba, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.354 · Cited by 1,389
Understanding Neural Networks Through Deep Visualization
Yosinski, J., Clune, J., Nguyen, A. M., Fuchs, T. J., & Lipson, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06579 · Cited by 1,358
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276
Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., & Sayres, R. (2017) · arXiv · doi:10.48550/arxiv.1711.11279 · Cited by 1,240
Evaluating the Visualization of What a Deep Neural Network Has Learned
Samek, W., Binder, A., Montavon, G., Lapuschkin, S., & Muller, K.-R. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2599820 · Cited by 1,164
Interpretation of Neural Networks is FragilePreprint
Ghorbani, A., Abid, A., & Zou, J. Y. (2017) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.1710.10547 · Cited by 1,153
Explainable artificial intelligence: A survey
Dosilovic, F. K., Brcic, M., & Hlupic, N. (2018) · 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) · doi:10.23919/mipro.2018.8400040 · Cited by 1,147
Hierarchical Question-Image Co-Attention for Visual Question Answering
Lu, J., Yang, J., Batra, D., & Parikh, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00061 · Cited by 1,129
Interpretable Explanations of Black Boxes by Meaningful Perturbation
Fong, R. C., & Vedaldi, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.371 · Cited by 1,097
Samek, W., Wiegand, T., & Müller, K.-R. (2017) · arXiv · doi:10.48550/arxiv.1708.08296 · Cited by 1,080
Unmasking Clever Hans predictors and assessing what machines really learn
Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., & Müller, K.-R. (2019) · Nature Communications · doi:10.1038/s41467-019-08987-4 · Cited by 1,047
Designing Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906
Olah, C., Mordvintsev, A., & Schubert, L. (2017) · Distill · doi:10.23915/distill.00007 · Cited by 880
Visualizing and Understanding Recurrent Networks
Karpathy, A., Johnson, J., & Fei-Fei, L. (2015) · arXiv · doi:10.48550/arxiv.1506.02078 · Cited by 845
Artificial Intelligence and Black‐Box Medical Decisions: Accuracy versus Explainability
London, A. J. (2019) · Hastings Center Report · doi:10.1002/hast.973 · Cited by 801
Rationalizing Neural Predictions
Lei, T., Barzilay, R., & Jaakkola, T. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1011 · Cited by 777
Letham, B., Rudin, C., McCormick, T. H., & Madigan, D. (2015) · The Annals of Applied Statistics · doi:10.1214/15-aoas848 · Cited by 766
Interpretation of Neural Networks Is Fragile
Ghorbani, A., Abid, A., & Zou, J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013681 · Cited by 765
Vellido, A. (2020) · Neural Computing and Applications · doi:10.1007/s00521-019-04051-w · Cited by 635
Ma, W., Cheng, F., Xu, Y., Wen, Q., & Liu, Y. (2019) · Advanced Materials · doi:10.1002/adma.201901111 · Cited by 615
The Building Blocks of Interpretability
Olah, C., Satyanarayan, A., Johnson, I., Carter, S., Schubert, L., Ye, K., & Mordvintsev, A. (2018) · Distill · doi:10.23915/distill.00010 · Cited by 591
Understanding intermediate layers using linear classifier probes
Alain, G., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1610.01644 · Cited by 581
The (Un)reliability of Saliency Methods
Kindermans, P.-J., Hooker, S., Adebayo, J., Alber, M., Schütt, K. T., Dähne, S., Erhan, D., & Kim, B. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6_14 · Cited by 580
Unmasking Clever Hans predictors and assessing what machines really learn
Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., & Müller, K. (2019) · RePEc: Research Papers in Economics · Cited by 580
VQA: Visual Question Answering: www.visualqa.org
Agrawal, A., Lu, J., Antol, S., Mitchell, M., Zitnick, C. L., Parikh, D., & Batra, D. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0966-6 · Cited by 570
Accurate intelligible models with pairwise interactions
Lou, Y., Caruana, R., Gehrke, J., & Hooker, G. (2013) · Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2487575.2487579 · Cited by 564
Intelligible models for classification and regression
Lou, Y., Caruana, R., & Gehrke, J. (2012) · Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2339530.2339556 · Cited by 547
Visualizing and Understanding Convolutional Networks
Zeiler, M. D., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1311.2901 · Cited by 522
Detecting Cancer Metastases on Gigapixel Pathology Images
Liu, Y., Gadepalli, K., Norouzi, M., Dahl, G. E., Kohlberger, T., Boyko, A., Venugopalan, S., Timofeev, A., Nelson, P. Q., Corrado, G. S., et al. (2017) · arXiv · doi:10.48550/arxiv.1703.02442 · Cited by 506
3D U-Net: Learning Dense Volumetric Segmentation from Sparse AnnotationPreprint
Çiçek, Ö., Abdulkadir, A., Lienkamp, S. S., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.06650 · Cited by 488
Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
Zintgraf, L. M., Cohen, T. S., Adel, T., & Welling, M. (2017) · arXiv · doi:10.48550/arxiv.1702.04595 · Cited by 487
Hastie, T. J., & Tibshirani, R. J. (2017) · Routledge eBooks · doi:10.1201/9780203753781 · Cited by 454
Das, A., Kottur, S., Gupta, K., Singh, A., Jain, D., Moura, J. M. F., Parikh, D., & Batra, D. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · Cited by 453
Das, A., Kottur, S., Gupta, K., Singh, A., Yadav, D., Moura, J. M. F., Parikh, D., & Batra, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.121 · Cited by 445
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301590 · Cited by 392
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · arXiv · doi:10.48550/arxiv.1901.07031 · Cited by 392
Towards Automatic Concept-based Explanations
Ghorbani, A., Wexler, J., Zou, J., & Kim, B. (2019) · arXiv · doi:10.48550/arxiv.1902.03129 · Cited by 378
Langlotz, C. P., Allen, B., Erickson, B. J., Kalpathy-Cramer, J., Bigelow, K., Cook, T. S., Flanders, A. E., Lungren, M. P., Mendelson, D. S., Rudie, J. D., et al. (2019) · Radiology · doi:10.1148/radiol.2019190613 · Cited by 368
Nguyen, A. T., Yosinski, J., & Clune, J. (2016) · arXiv.org · doi:10.48550/arxiv.1602.03616 · Cited by 365
Explaining Recurrent Neural Network Predictions in Sentiment Analysis
Arras, L., Montavon, G., Müller, K.-R., & Samek, W. (2017) · Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis · doi:10.18653/v1/w17-5221 · Cited by 361
Hierarchical Question-Image Co-Attention for Visual Question Answering
Lu, J. (2024) · TIB Data Manager · doi:10.57702/uhdmfu1r · Cited by 357
Brain–machine interfaces in neurorehabilitation of stroke
Soekadar, S. R., Birbaumer, N., Slutzky, M. W., & Cohen, L. G. (2015) · Neurobiology of Disease · doi:10.1016/j.nbd.2014.11.025 · Cited by 335
Raghu, M., Gilmer, J., Yosinski, J., & Sohl-Dickstein, J. (2017) · arXiv · doi:10.48550/arxiv.1706.05806 · Cited by 335
Interpreting Deep Visual Representations via Network Dissection
Zhou, B., Bau, D., Oliva, A., & Torralba, A. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858759 · Cited by 330
What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use
Tonekaboni, S., Joshi, S., McCradden, M. D., & Goldenberg, A. (2019) · arXiv · doi:10.48550/arxiv.1905.05134 · Cited by 318
Representer Point Selection for Explaining Deep Neural NetworksPreprint
Yeh, C., Kim, J. S., Yen, I. E.-H., & Ravikumar, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.09720 · Cited by 312
Grad-CAM: Why did you say that?
Selvaraju, R. R., Das, A., Vedantam, R., Cogswell, M., Parikh, D., & Batra, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.07450 · Cited by 309
Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
Brundage, M., Avin, S., Wang, J., Belfield, H., Krueger, G., Hadfield, G., Khlaaf, H., Yang, J., Toner, H., Fong, R., et al. (2020) · arXiv · doi:10.48550/arxiv.2004.07213 · Cited by 306
Fully automatic acute ischemic lesion segmentation in DWI using convolutional neural networks
Chen, L., Bentley, P., & Rueckert, D. (2017) · NeuroImage: Clinical · doi:10.1016/j.nicl.2017.06.016 · Cited by 304
Weakly Supervised Instance Segmentation Using Class Peak Response
Zhou, Y., Zhu, Y., Ye, Q., Qiu, Q., & Jiao, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00399 · Cited by 296
Explainable artificial intelligence for breast cancer: A visual case-based reasoning approach
Lamy, J.-B., Sekar, B., Guezennec, G., Bouaud, J., & Séroussi, B. (2019) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2019.01.001 · Cited by 282
Deep into the Brain: Artificial Intelligence in Stroke Imaging
Lee, E.-J., Kim, Y.-H., Kim, N., & Kang, D.-W. (2017) · Journal of Stroke · doi:10.5853/jos.2017.02054 · Cited by 264
Learning how to explain neural networks: PatternNet and PatternAttribution
Kindermans, P.-J., Schütt, K. T., Alber, M., Müller, K. R., Erhan, D., Kim, B., & Dähne, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.05598 · Cited by 264
"What is relevant in a text document?": An interpretable machine learning approach
Arras, L., Horn, F., Montavon, G., Müller, K.-R., & Samek, W. (2017) · PLOS ONE · doi:10.1371/journal.pone.0181142 · Cited by 257
Faithful and Customizable Explanations of Black Box Models
Lakkaraju, H., Kamar, E., Caruana, R., & Leskovec, J. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314229 · Cited by 256
Singh, S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7475259 · Cited by 252
"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations
Lakkaraju, H., & Bastani, O. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375833 · Cited by 241
Fernandez, A., Herrera, F., Cordon, O., Jose del Jesus, M., & Marcelloni, F. (2019) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2881645 · Cited by 232
Tang, Z., Chuang, K. V., DeCarli, C., Jin, L.-W., Beckett, L., Keiser, M. J., & Dugger, B. N. (2019) · Nature Communications · doi:10.1038/s41467-019-10212-1 · Cited by 229
Generalizability vs. Robustness: Investigating Medical Imaging Networks Using Adversarial Examples
Paschali, M., Conjeti, S., Navarro, F. A., & Navab, N. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00928-1_56 · Cited by 226
Understanding Convolutional Neural Networks for Text Classification
Jacovi, A., Sar Shalom, O., & Goldberg, Y. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5408 · Cited by 226
Kuhlmann, L., Karoly, P., Freestone, D. R., Brinkmann, B. H., Temko, A., Barachant, A., Li, F., Titericz, G., Lang, B. W., Lavery, D., et al. (2018) · Brain · doi:10.1093/brain/awy210 · Cited by 195
Application of t-SNE to human genetic data
Li, W., Cerise, J. E., Yang, Y., & Han, H. (2017) · Journal of Bioinformatics and Computational Biology · doi:10.1142/s0219720017500172 · Cited by 194
Pereira, S., Meier, R., Alves, V., Reyes, M., & Silva, C. A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-02628-8_12 · Cited by 179
A causal framework for explaining the predictions of black-box sequence-to-sequence models
Alvarez-Melis, D., & Jaakkola, T. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1042 · Cited by 178
How far have we come? Artificial intelligence for chest radiograph interpretation
Kallianos, K., Mongan, J., Antani, S., Henry, T., Taylor, A., Abuya, J., & Kohli, M. (2019) · Clinical Radiology · doi:10.1016/j.crad.2018.12.015 · Cited by 173
Explanations can be manipulated and geometry is to blame
Dombrowski, A.-K., Alber, M., Anders, C. J., Ackermann, M., Müller, K.-R., & Kessel, P. (2019) · arXiv · doi:10.48550/arxiv.1906.07983 · Cited by 171
Understanding Deep Image Representations by Inverting Them
Mahendran, A., & Vedaldi, A. (2014) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.48550/arxiv.1412.0035 · Cited by 169
Ablation Studies in Artificial Neural Networks
Meyes, R., Lu, M., de Puiseau, C. W., & Meisen, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08644 · Cited by 168
Deep convolutional neural networks for interpretable analysis of EEG sleep stage scoring
Vilamala, A., Madsen, K. H., & Hansen, L. K. (2017) · 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP) · doi:10.1109/mlsp.2017.8168133 · Cited by 162
Brain age from the electroencephalogram of sleep
Sun, H., Paixão, L., Oliva, J. T., Goparaju, B., Carvalho, D. Z., van Leeuwen, K. G., Akeju, O., THOMAS, R. J., Cash, S. S., Bianchi, M. T., et al. (2018) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2018.10.016 · Cited by 161
The (Un)reliability of saliency methodsPreprint
Kindermans, P.-J., Hooker, S., Adebayo, J., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00867 · Cited by 160
Artificial intelligence applications in dermatology: Where do we stand?
Gomolin, A., Netchiporouk, E., Gniadecki, R., & Litvinov, I. V. (2020) · Frontiers in Medicine · doi:10.3389/fmed.2020.00100 · Cited by 142
Autofocus Layer for Semantic Segmentation
Qin, Y., Kamnitsas, K., Ancha, S., Nanavati, J., Cottrell, G., Criminisi, A., & Nori, A. (2018) · arXiv · doi:10.48550/arxiv.1805.08403 · Cited by 135
Autofocus Layer for Semantic Segmentation
Qin, Y., Kamnitsas, K., Ancha, S., Nanavati, J., Cottrell, G., Criminisi, A., & Nori, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00931-1_69 · Cited by 135
Overview of ImageCLEF 2018 Medical Domain Visual Question Answering Task.
Hasan, S. A., Yuan, L., Farri, O., Liu, J., Müller, H., & Lungren, M. P. (2018) · ArODES (HES-SO (https://www.hes-so.ch/)) · Cited by 128
Yan, Y., Zhu, J., Duda, M., Solarz, E., Sripada, C., & Koutra, D. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330921 · Cited by 127
An Evaluation of the Human-Interpretability of Explanation
Lage, I., Chen, E., He, J., Narayanan, M., Kim, B., Gershman, S., & Doshi-Velez, F. (2019) · arXiv · doi:10.48550/arxiv.1902.00006 · Cited by 125
Interpreting and Explaining Deep Neural Networks for Classification of Audio SignalsPreprint
Becker, S., Ackermann, M. R., Lapuschkin, S., Müller, K., & Samek, W. (2018) · arXiv (Cornell University) · Cited by 124
Cheng, J., & Bernstein, M. S. (2015) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2675133.2675214 · Cited by 117
Understanding Convolutional Neural Networks for Text ClassificationPreprint
Jacovi, A., Shalom, O. S., & Goldberg, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.08037 · Cited by 116
Kocadağlı, O., & Langari, R. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.07.020 · Cited by 115
Analyzing Neuroimaging Data Through Recurrent Deep Learning Models
Thomas, A. W., Heekeren, H. R., Müller, K., & Samek, W. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.01321 · Cited by 113
Your evidence? Machine learning algorithms for medical diagnosis and prediction
Heinrichs, B., & Eickhoff, S. B. (2019) · Human Brain Mapping · doi:10.1002/hbm.24886 · Cited by 113
Regression Concept Vectors for Bidirectional Explanations in Histopathology
Graziani, M., Andrearczyk, V., & Müller, H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-02628-8_14 · Cited by 112
Deep Neural Network or Dermatologist?
Young, K. C., Booth, G., Simpson, B., Dutton, R., & Shrapnel, S. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33850-3_6 · Cited by 112
VQA-Med: Overview of the Medical Visual Question Answering Task at ImageCLEF 2019
Abacha, A. B., Hasan, S. A., Datla, V. V., Liu, J., Demner‐Fushman, D., & Müller, H. (2024) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.10499038 · Cited by 109
Building and Interpreting Deep Similarity Models
Eberle, O., Büttner, J., Kräutli, F., Müller, K., Valleriani, M., & Montavon, G. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3020738 · Cited by 98
Eitel, F., Ritter, K., & Initiative, A. D. N. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33850-3_1 · Cited by 96
Brain Biomarker Interpretation in ASD Using Deep Learning and fMRI
Li, X., Dvornek, N. C., Zhuang, J., Ventola, P. E., & Duncan, J. S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00931-1_24 · Cited by 93
Townsend, J., Chaton, T., & Monteiro, J. M. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2944672 · Cited by 91
Biffi, C., Oktay, O., Tarroni, G., Bai, W., de Marvao, A., Doumou, G., Rajchl, M., Bedair, R., Prasad, S. K., Cook, S. A., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00934-2_52 · Cited by 83
Multiple Instance Learning for Heterogeneous Images: Training a CNN for Histopathology
Couture, H. D., Marron, J. S., Perou, C. M., Troester, M. A., & Niethammer, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00934-2_29 · Cited by 82
VQA-Med: Overview of the Medical Visual Question Answering Task at ImageCLEF 2019
Abacha, A. B., Hasan, S. A., Datla, V. V., et al. (2024) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.10499039 · Cited by 81
Papanastasopoulos, Z., Samala, R. K., Chan, H., Hadjiiski, L. M., Paramagul, C., Helvie, M. A., & Neal, C. H. (2020) · SPIE Medical Imaging: Computer-Aided Diagnosis · doi:10.1117/12.2549298 · Cited by 78
Inverting Convolutional Networks with Convolutional Networks.Preprint
Dosovitskiy, A., & Brox, T. (2015) · arXiv (Cornell University) · Cited by 75
Visualizing Convolutional Neural Networks to Improve Decision Support for Skin Lesion Classification
Molle, P. V., Strooper, M. D., Verbelen, T., Vankeirsbilck, B., Simoens, P., & Dhoedt, B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-02628-8_13 · Cited by 73
An Interpretable Neural Network for Configuring Programmable Wireless Environments
Liaskos, C., Tsioliaridou, A., Nie, S., Pitsillides, A., Ioannidis, S., & Akyildiz, I. F. (2019) · IEEE International Workshop on Signal Processing Advances in Wireless Communications (SPAWC) · doi:10.1109/spawc.2019.8815428 · Cited by 65
Prentzas, N., Nicolaides, A. N., Kyriacou, E. C., Kakas, A. C., & Pattichis, C. S. (2019) · IEEE International Conference on Bioinformatics and Bioengineering (BIBE) · doi:10.1109/bibe.2019.00152 · Cited by 63
Outlining the Design Space of Explainable Intelligent Systems for Medical DiagnosisPreprint
Xie, Y., Gao, G., & Chen, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06019 · Cited by 62
Zhang, T., Chen, W., & Li, M. (2018) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2018.08.038 · Cited by 60
Explainable Sentiment Analysis with Applications in Medicine
Zucco, C., Liang, H., Fatta, G. D., & Cannataro, M. (2018) · 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2018.8621359 · Cited by 59
Croskerry, P., Cosby, K. S., Graber, M. L., & Singh, H. (2017) · CRC Press eBooks · doi:10.1201/9781315116334 · Cited by 58
ProtoAttend: Attention-Based Prototypical LearningPreprint
Arık, S. Ö., & Pfister, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06292 · Cited by 58
Dementia in the Elderly: An Analysis of Medical Responsibility
Cassel, C. K., & JAMETON, A. L. (1981) · Annals of Internal Medicine · doi:10.7326/0003-4819-94-6-802 · Cited by 55
Respond-CAM: Analyzing Deep Models for 3D Imaging Data by Visualizations
Zhao, G., Zhou, B., Wang, K., Jiang, R., & Xu, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00928-1_55 · Cited by 55
How to Explain Individual Classification Decisions
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1859912 · Cited by 54
Interpreting the Predictions of Complex ML Models by Layer-wise Relevance PropagationPreprint
Samek, W., Montavon, G., Binder, A., Lapuschkin, S., & Müller, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.08191 · Cited by 54
Codella, N. C. F., Lin, C.-C., Halpern, A., Hind, M., Feris, R., & Smith, J. R. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-02628-8_11 · Cited by 52
Ensemble of Deep Convolutional Neural Networks for Prognosis of Ischemic Stroke
Choi, Y., Kwon, Y., Lee, H., Kim, B. J., Paik, M. C., & Won, J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-55524-9_22 · Cited by 51
Magnetic Resonance Spectroscopy Quantification Using Deep Learning
Hatami, N., Sdika, M., & Ratiney, H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00928-1_53 · Cited by 49
Interpretable human action recognition in compressed domain
Srinivasan, V., Lapuschkin, S., Hellge, C., Müller, K., & Samek, W. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952445 · Cited by 47
Quantum-Chemical Insights from Interpretable Atomistic Neural Networks
Schütt, K. T., Gastegger, M., Tkatchenko, A., & Müller, K. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-28954-6_17 · Cited by 47
Enabling an Open Data Ecosystem for the Neurosciences
Wiener, M., Sommer, F. T., Ives, Z. G., Poldrack, R. A., & Litt, B. (2016) · Neuron · doi:10.1016/j.neuron.2016.10.037 · Cited by 46
Lee, H., Kim, S. T., & Ro, Y. M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33850-3_3 · Cited by 46
Izadyyazdanabadi, M., Belykh, E. G., Cavallo, C., Zhao, X., Gandhi, S., Moreira, L. B., Eschbacher, J. M., Nakaji, P., Preul, M. C., & Yang, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00934-2_34 · Cited by 45
Weakly Supervised Instance Segmentation using Class Peak ResponsePreprint
Zhou, Y., Zhu, Y., Ye, Q., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.00880 · Cited by 38
UBS: A Dimension-Agnostic Metric for Concept Vector Interpretability Applied to Radiomics
Yèche, H., Harrison, J., & Berthier, T. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33850-3_2 · Cited by 34
Ulaş, Ç., Tetteh, G., Kaczmarz, S., Preibisch, C., & Menze, B. H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00928-1_4 · Cited by 31
Diagnosis : Interpreting the Shadows
Croskerry, P., Cosby, K. S., Graber, M. L., & Singh, H. (2017) · Cited by 29
Meyes, R., de Puiseau, C. W., Posada-Moreno, A. F., & Meisen, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.01254 · Cited by 29
Xu, W., Jiang, X., Hu, X. T., & Li, G. (2014) · BMC Medical Genomics · doi:10.1186/1755-8794-7-s2-s1 · Cited by 28
Guideline-Based Additive Explanation for Computer-Aided Diagnosis of Lung Nodules
Zhu, P., & Ogino, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33850-3_5 · Cited by 26
Low-Rank Representation for Multi-center Autism Spectrum Disorder Identification
Wang, M., Zhang, D., Huang, J., Shen, D. G., & Liu, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00928-1_73 · Cited by 25
Incorporating Task-Specific Structural Knowledge into CNNs for Brain Midline Shift Detection
Pisov, M., Goncharov, M., Kurochkina, N., Morozov, S. P., Gombolevskiy, V. А., Chernina, V., Vladzymyrskyy, A. V., Zamyatina, K., Chesnokova, A., Pronin, I., et al. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33850-3_4 · Cited by 24
Group-Driven Reinforcement Learning for Personalized mHealth Intervention
Zhu, F., Guo, J. Y., Xu, Z., Liao, P., Liu, Y., & Huang, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00928-1_67 · Cited by 22
Jansen, C., Penzel, T., Hodel, S., Breuer, S., Spott, M., & Krefting, D. (2019) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5128003 · Cited by 21
Explaining Motion Relevance for Activity Recognition in Video Deep Learning ModelsPreprint
Hiley, L., Preece, A. D., Hicks, Y., Chakraborty, S., Gurram, P. K., & Tomsett, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.14285 · Cited by 21
Meacham, S., Isaac, G., Nauck, D. D., & Virginas, B. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-22871-2_67 · Cited by 20
Hazarika, A., Barthakur, M., Dutta, L., & Bhuyan, M. (2018) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2018.08.010 · Cited by 19
Predicting Stroke Lesion and Clinical Outcome with Random Forests
Maier, O., & Heinz Handels, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-55524-9_21 · Cited by 18
Tang, Z., Chuang, K. V., DeCarli, C. S., et al. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/454793 · Cited by 17
Kim, B., Chacha, C. M., & Shah, J. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8548 · Cited by 16
Reduced acquisition time PET pharmacokinetic modelling using simultaneous ASL–MRI: proof of concept
Scott, C. J., Jiao, J., Melbourne, A., Burgos, N., Cash, D. M., Vita, E. D., Markiewicz, P. J., O’Connor, A., Thomas, D. L., Weston, P. S. J., et al. (2018) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1177/0271678x18797343 · Cited by 15
Structuring Neural Networks for More Explainable Predictions
Rieger, L., Chormai, P., Montavon, G., Hansen, L. K., & Müller, K. (2018) · The Springer series on challenges in machine learning · doi:10.1007/978-3-319-98131-4_5 · Cited by 15
Ulloa, A., vanMaanen, D. P., Jing, L., Stough, J. V., Patel, A. A., Haggerty, C. M., Fornwalt, B. K., & Pattichis, M. S. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3529320 · Cited by 14
Neural Network Interpretation via Fine Grained Textual Summarization.Preprint
Guo, P., Anderson, C., Pearson, K., & Farrell, R. J. (2018) · arXiv (Cornell University) · Cited by 13
Understanding the (un)interpretability of natural image distributions using generative modelsPreprint
Krusinga, R., Shah, S., Zwicker, M., Goldstein, T., & Jacobs, D. W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.01499 · Cited by 13
Kim, B., Chacha, C. M., & Shah, J. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1306.0963 · Cited by 11
Varol, E., Sotiras, A., Zeng, K., & Davatzikos, C. A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00931-1_62 · Cited by 11
Arık, S. Ö., & Pfister, T. (2019) · arXiv (Cornell University) · Cited by 9
An Interpretable Generative Model for Handwritten Digits Synthesis
Zhu, Y., Suri, S., Kulkarni, P., Chen, Y., Duan, J., & Kuo, C. ‐. J. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803129 · Cited by 9
Fuzzy Systems with Sigmoid-Based Membership Functions as Interpretable Neural Networks
Bede, B. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-21920-8_15 · Cited by 8
Enabling an Open Data Ecosystem for the Neurosciences
Wiener, M., Sommer, F. T., Ives, Z. G., Poldrack, R. A., & Litt, B. (2016) · Neuron · doi:10.1016/j.neuron.2016.11.009 · Cited by 8
AudioMNIST: Exploring Explainable Artificial Intelligence for Audio Analysis on a Simple Benchmark
Becker, S., Vielhaben, J., Ackermann, M., Müller, K.-R., Lapuschkin, S., & Samek, W. (2018) · arXiv · doi:10.48550/arxiv.1807.03418 · Cited by 7
Interpretable Dynamics Models for Data-Efficient Reinforcement LearningPreprint
Kaiser, M., Otte, C., Runkler, T. A., & Ek, C. H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.04902 · Cited by 6
Biffi, C., Oktay, O., Tarroni, G., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.06843
Magnetic Resonance Spectroscopy Quantification using Deep Learning
Hatami, N., Sdika, M., & Ratiney, H. (2018) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.48550/arxiv.1806.07237
Semantic Network InterpretationPreprint
Guo, P., & Farrell, R. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.08969
Opening the Black Box of Deep Neural Networks via Information
Shwartz-Ziv, R., & Tishby, N. (2017) · arXiv · doi:10.48550/arxiv.1703.00810
Research works printed but not linked: 8
- 2016, Can We Open the Black Box of AI?
- 2019, Ethics guidelines for trustworthy ai
- 2016, Deep Taylor Decomposition of Neural Networks
- 2018, Machine Learning and AI for the Sciences—Towards Interpretability
- cerna, 2019, Interpretable neural networks for predicting mortality risk using multi-modal electronic health records, CoRR, abs 1901 8125, 3
- lipton, 2016, The Mythos of model interpretability, CoRR, abs 1606 3490, 1
- karpathy, 2014, t-sne visualization of cnn codes
- carter, 2019, Exploring Neural Networks With Activation Atlases
Cited by
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., & Hussain, A. (2024) · Cognitive Computation · doi:10.1007/s12559-023-10179-8 · Cited by 2,495
“Tjoa et al. [39] proposed two major classes of interpretability, i.e., perceptive interpretability and interpretability by mathematical structures.”
BackgroundDeep Neural Networks and Tabular Data: A Survey
Borisov, V., Leemann, T., Seßler, K., Haug, J., Pawelczyk, M., & Kasneci, G. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3229161 · Cited by 1,025
A Survey on the Explainability of Supervised Machine Learning
Burkart, N., & Huber, M. F. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12228 · Cited by 1,008
Explainable artificial intelligence: a comprehensive review
Minh, D., Wang, H. X., Li, Y. F., & Nguyen, T. N. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10088-y · Cited by 983
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957
“60 Biomedical and health data can be subject to major variations in the real world (both expected and unexpected), which can affect the performance of AI tools.”
Metadata
- Authors
- Tjoa, E., & Guan, C.
- Affiliations
- Nanyang Technological University (SG)
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2021-11
- Volume
- 32
- Issue
- 11
- Pages
- 4793-4813
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W2958089299
- Semantic Scholar
- 38f23fe236b152cd4983c8f30d305a568afd0d3e
- Cited by
- 2,524
- References
- 178
- Citations by year
- 2026: 606 · 2025: 527 · 2024: 440 · 2023: 421 · 2022: 285 · 2021: 195
- FWCI
- 92.80
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- IEEE Transactions on Neural Networks and Learning Systems (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- DR-NTU (Nanyang Technological University) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- Health-AI
- DAMO Academy
- Alibaba Group Holding Ltd.
- Alibaba-NTU Talent Program
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- explainable AI, interpretability, black-box models, deep learning, transparency, accountability, explainable machine learning, medical education
- MeSH terms
- Machine Learning, Artificial Intelligence, Humans, Image Processing, Computer-Assisted, Pattern Recognition, Automated, Surveys and Questionnaires, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.