The (Un)reliability of Saliency Methods

ImageNet Large Scale Visual Recognition Challenge

Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380

Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370

On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation

Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524

Striving for Simplicity: The All Convolutional Net

Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447

Explaining nonlinear classification decisions with deep taylor decomposition

Montavon, G., Lapuschkin, S., Binder, A., Samek, W., & Müller, K.-R. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.11.008 · Cited by 1,357

Evaluating the Visualization of What a Deep Neural Network Has Learned

Samek, W., Binder, A., Montavon, G., Lapuschkin, S., & Muller, K.-R. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2599820 · Cited by 1,164

A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI

Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524

Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence

Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762

Explainable AI: Interpreting, Explaining and Visualizing Deep Learning

(2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6 · Cited by 1,377

Explainable artificial intelligence: a comprehensive review

Minh, D., Wang, H. X., Li, Y. F., & Nguyen, T. N. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10088-y · Cited by 983

A Survey on Neural Network Interpretability

Zhang, Y., Tino, P., Leonardis, A., & Tang, K. (2021) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2021.3100641 · Cited by 923

A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems

Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785

A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems

Mohseni, S., Zarei, N., & Ragan, E. D. (2018) · arXiv · doi:10.48550/arxiv.1811.11839 · Cited by 783

Algorithmic fairness in artificial intelligence for medicine and healthcare

Chen, R. J., Wang, J. J., Williamson, D. F. K., Chen, T. Y., Lipkova, J., Lu, M. Y., Sahai, S., & Mahmood, F. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01056-8 · Cited by 750

Explainability for Large Language Models: A Survey

Zhao, H., Chen, H., Yang, F., Liu, N., Deng, H., Cai, H., Wang, S., Yin, D., & Du, M. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3639372 · Cited by 681

Notions of explainability and evaluation approaches for explainable artificial intelligence

Vilone, G., & Longo, L. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.009 · Cited by 598

The Building Blocks of Interpretability

Olah, C., Satyanarayan, A., Johnson, I., Carter, S., Schubert, L., Ye, K., & Mordvintsev, A. (2018) · Distill · doi:10.23915/distill.00010 · Cited by 591

Explainability Methods for Graph Convolutional Neural Networks

Pope, P. E., Kolouri, S., Rostami, M., Martin, C. E., & Hoffmann, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01103 · Cited by 553

Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey

Das, A., & Rad, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11371 · Cited by 504

Explaining anomalies detected by autoencoders using Shapley Additive Explanations

Antwarg, L., Miller, R. M., Shapira, B., & Rokach, L. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115736 · Cited by 496

Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness?

Jacovi, A., & Goldberg, Y. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.386 · Cited by 503

Towards Automatic Concept-based Explanations

Ghorbani, A., Wexler, J., Zou, J., & Kim, B. (2019) · arXiv · doi:10.48550/arxiv.1902.03129 · Cited by 378

Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims

Brundage, M., Avin, S., Wang, J., Belfield, H., Krueger, G., Hadfield, G., Khlaaf, H., Yang, J., Toner, H., Fong, R., et al. (2020) · arXiv · doi:10.48550/arxiv.2004.07213 · Cited by 306

Open Problems in Mechanistic Interpretability

Sharkey, L., Chughtai, B., Batson, J., Lindsey, J., Wu, J., Bushnaq, L., Goldowsky-Dill, N., Heimersheim, S., Ortega, A., Bloom, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.16496 · Cited by 291

Four principles of explainable artificial intelligence

Phillips, P. J., Hahn, C. A., Fontana, P. C., Yates, A. N., Greene, K., Broniatowski, D. A., & Przybocki, M. A. (2021) · doi:10.6028/nist.ir.8312 · Cited by 262

A Survey of Data-driven and Knowledge-aware eXplainable AI

Li, X.-H., Cao, C. C., Shi, Y., Bai, W., Gao, H., Qiu, L., Wang, C., Gao, Y., Zhang, S., Xue, X., et al. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2983930 · Cited by 224

Explainable AI for clinical and remote health applications: A survey on tabular and time series data

Di Martino, F., & Delmastro, F. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10304-3 · Cited by 216

Explanations can be manipulated and geometry is to blame

Dombrowski, A.-K., Alber, M., Anders, C. J., Ackermann, M., Müller, K.-R., & Kessel, P. (2019) · arXiv · doi:10.48550/arxiv.1906.07983 · Cited by 171

Acquisition of chess knowledge in AlphaZero

McGrath, T., Kapishnikov, A., Tomašev, N., Pearce, A., Wattenberg, M., Hassabis, D., Kim, B., Paquet, U., & Kramnik, V. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2206625119 · Cited by 166

Explainable artificial intelligence for cybersecurity: a literature survey

Charmet, F., Tanuwidjaja, H. C., Ayoubi, S., Gimenez, P.-F., Han, Y., Jmila, H., Blanc, G., Takahashi, T., & Zhang, Z. (2022) · Annals of Telecommunications · doi:10.1007/s12243-022-00926-7 · Cited by 167

Local interpretable model-agnostic explanations for classification of lymph node metastases

Palatnik de Sousa, I., Maria Bernardes Rebuzzi Vellasco, M., & Costa da Silva, E. (2019) · Sensors · doi:10.3390/s19132969 · Cited by 155

Ethical Dimensions of Visualization Research

Correll, M. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300418 · Cited by 150

Interpretable machine learning for dementia: a systematic review

Martin, S. A., Townend, F. J., Barkhof, F., & Cole, J. H. (2023) · Alzheimer's & Dementia · doi:10.1002/alz.12948 · Cited by 149

Measuring Association Between Labels and Free-Text Rationales

Wiegreffe, S., Marasović, A., & Smith, N. A. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.804 · Cited by 119

Stakeholders in Explainable AI

Preece, A., Harborne, D., Braines, D., Tomsett, R., & Chakraborty, S. (2018) · arXiv · doi:10.48550/arxiv.1810.00184 · Cited by 117

XAI Systems Evaluation: A Review of Human and Computer-Centred Methods

Lopes, P., Silva, E., Braga, C., Oliveira, T., & Rosado, L. (2022) · Applied Sciences · doi:10.3390/app12199423 · Cited by 112

Concept attribution: explaining cnn decisions to physicians

M., G., V., A., S., M.-M., & H., M. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.103865 · Cited by 95

Explainable Artificial Intelligence: a Systematic Review

Vilone, G., & Longo, L. (2020) · arXiv · doi:10.48550/arxiv.2006.00093 · Cited by 269

A systematic review on the use of explainability in deep learning systems for computer aided diagnosis in radiology: Limited use of explainable AI?

Groen, A. M., Kraan, R., Amirkhan, S. F., Daams, J. G., & Maas, M. (2022) · European Journal of Radiology · doi:10.1016/j.ejrad.2022.110592 · Cited by 89

Exact and Consistent Interpretation for Piecewise Linear Neural Networks: A Closed Form Solution

Chu, L., Hu, X., Hu, J., Wang, L., & Pei, J. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220063 · Cited by 87

Flexible and Context-Specific AI Explainability: A Multidisciplinary Approach

Beaudouin, V., Bloch, I., Bounie, D., Clémençon, S., d'Alché-Buc, F., Eagan, J., Maxwell, W., Mozharovskyi, P., & Parekh, J. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3559477 · Cited by 61

Establishing and evaluating trustworthy AI: overview and research challenges

Kowald, D., Scher, S., Pammer-Schindler, V., Müllner, P., Waxnegger, K., Demelius, L., Fessl, A., Toller, M., Mendoza Estrada, I. G., Šimić, I., et al. (2024) · Frontiers in Big Data · doi:10.3389/fdata.2024.1467222 · Cited by 59

Interpreting deep learning models for weak lensing

Matilla, J. M. Z., Sharma, M., Hsu, D., & Haiman, Z. (2020) · Physical Review D · doi:10.1103/physrevd.102.123506 · Cited by 48

Efficient Shapley Explanation for Features Importance Estimation Under Uncertainty

Li, X., Zhou, Y., Dvornek, N. C., Gu, Y., Ventola, P., & Duncan, J. S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-59710-8_77 · Cited by 45

Curve Detectors

Cammarata, N., Goh, G., Carter, S., Schubert, L., Petrov, M., & Olah, C. (2020) · Distill · doi:10.23915/distill.00024.003 · Cited by 41

Understanding RL vision

Hilton, J., Cammarata, N., Carter, S., Goh, G., & Olah, C. (2020) · Distill · doi:10.23915/distill.00029 · Cited by 37

Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Prediction

Chu, E., Roy, D., & Andreas, J. (2020) · arXiv · doi:10.48550/arxiv.2007.12248 · Cited by 34

Future Shock: Generative AI and the International AI Policy and Governance Crisis

Leslie, D., & Perini, A. M. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.88b4cc98 · Cited by 33

Shortcomings of Top-Down Randomization-Based Sanity Checks for Evaluations of Deep Neural Network Explanations

Binder, A., Weber, L., Lapuschkin, S., Montavon, G., Müller, K.-R., & Samek, W. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01549 · Cited by 21

Distinct brain morphometry patterns revealed by deep learning improve prediction of post-stroke aphasia severity

Teghipco, A., Newman-Norlund, R., Fridriksson, J., Rorden, C., & Bonilha, L. (2024) · Communications Medicine · doi:10.1038/s43856-024-00541-8 · Cited by 11

Discriminative attribution from paired images

Eckstein, N., Bukhari, H., Bates, A. S., Jefferis, G. S. X. E., & Funke, J. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-25069-9_27 · Cited by 5

Sanity Checks for Saliency Metrics

Tomsett, R., Harborne, D., Chakraborty, S., Gurram, P., & Preece, A. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6064 · Cited by 3

EXplainable Neural-Symbolic Learning (X-NeSyL) methodology to fuse deep learning representations with expert knowledge graphs: The MonuMAI cultural heritage use case

Díaz-Rodríguez, N., Lamas, A., Sanchez, J., Franchi, G., Donadello, I., Tabik, S., Filliat, D., Cruz, P., Montes, R., & Herrera, F. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.09.022 · Cited by 2

Explainable and Interpretable AI for Voice and Speech Analysis in Clinical Care: Systematic Review

Ebraheem, M., Toghranegar, J., Bridge2AI-Voice Consortium, Bensoussan, Y., & Templeton, J. M. (2026) · Journal of Medical Internet Research · doi:10.2196/83790