The DOI matches its DataCite record; 24 of the 27 references it cites were found in a registry or an index.
Explanations can be manipulated and geometry is to blame
Open accessAbstract
Explanation methods aim to make neural networks more trustworthy and interpretable. In this paper, we demonstrate a property of explanation methods which is disconcerting for both of these purposes. Namely, we show that explanations can be manipulated arbitrarily by applying visually hardly perceptible perturbations to the input that keep the network's output approximately constant. We establish theoretically that this phenomenon can be related to certain geometrical properties of neural networks. This allows us to derive an upper bound on the susceptibility of explanations to manipulations. Based on this result, we propose effective mechanisms to enhance the robustness of explanations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Adversarial Vulnerability: Recent research demonstrates that explanations can be manipulated to appear reasonable while maintaining incorrect predictions [34].”
From Beyond Post hoc Explanations: A Comprehensive Framework for Accountable AI in Medical Imaging Through Transparency, Interpretability, and Explainability (2025) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524
Self-citation
Research works printed but not linked: 7
- [6][6] Ramprasaath R. Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Batra. Grad-cam: Why did you say that? visual explanations from deep networks via gradient-based localization. CoRR , abs/1610.02391, 2016. - · arXiv
- [14][14] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017 , pages 3145-3153, 2017. -
- [15][15] Ruth C Fong and Andrea Vedaldi. Interpretable explanations of black boxes by meaningful perturbation. In 2017 IEEE international conference on computer vision (ICCV) , pages 3449-3457. IEEE, 2017. -
- [22][22] Julius Adebayo, Justin Gilmer, Michael Muelly, Ian J. Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. CoRR , abs/1810.03292, 2018. - · arXiv
- [23][23] Maximilian Alber, Sebastian Lapuschkin, Philipp Seegerer, Miriam Hägele, Kristof T. Schütt, Grégoire Montavon, Wojciech Samek, Klaus-Robert Müller, Sven Dähne, and Pieter-Jan Kindermans. innvestigate neural networks! Journal of Machine Learning Research 20 , 2019. -
- [29][29] Gao Huang, Zhuang Liu, and Kilian Q. Weinberger. Densely connected convolutional networks. CoRR , abs/1608.06993, 2016. - · arXiv
- [31][31] Loring W Tu. Differential geometry: connections, curvature, and characteristic classes , volume 275. Springer, 2017.
Cited by
Sorted by citations
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524
Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3060483 · Cited by 1,477
“Relatedly, even when the model remains unchanged, inputs could be perturbed imperceptibly to produce arbitrary explanations [39].”
BackgroundSelf-citationA Unifying Review of Deep and Shallow Anomaly Detection
Ruff, L., Kauffmann, J., Vandermeulen, R. A., Montavon, G., Samek, W., Kloft, M., Dietterich, T. G., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3052449 · Cited by 1,184
A Survey on Neural Network Interpretability
Zhang, Y., Tino, P., Leonardis, A., & Tang, K. (2021) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2021.3100641 · Cited by 923
Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods
Slack, D., Hilgard, S., Jia, E., Singh, S., & Lakkaraju, H. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375830 · Cited by 900
Explainable machine learning in deployment
Bhatt, U., Xiang, A., Sharma, S., Weller, A., Taly, A., Jia, Y., Ghosh, J., Puri, R., Moura, J. M. F., & Eckersley, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375624 · Cited by 633
Obtaining genetics insights from deep learning via explainable artificial intelligence
Novakovsky, G., Dexter, N., Libbrecht, M. W., Wasserman, W. W., & Mostafavi, S. (2022) · Nature Reviews Genetics · doi:10.1038/s41576-022-00532-2 · Cited by 415
A Survey of Algorithmic Recourse: Contrastive Explanations and Consequential Recommendations
Karimi, A.-H., Barthe, G., Schölkopf, B., & Valera, I. (2023) · ACM Computing Surveys · doi:10.1145/3527848 · Cited by 301
Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation
Lee, J., Kim, E., & Yoon, S. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00406 · Cited by 297
Higher-Order Explanations of Graph Neural Networks via Relevant Walks
Schnake, T., Eberle, O., Lederer, J., Nakajima, S., Schütt, K. T., Müller, K., & Montavon, G. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3115452 · Cited by 322
Self-citationThe Disagreement Problem in Explainable Machine Learning: A Practitioner's PerspectivePreprint
Krishna, S., Han, T., Gu, A., Jabbari, S., Wu, Z. S., & Lakkaraju, H. (2023) · Research Square · doi:10.21203/rs.3.rs-2963888/v1 · Cited by 404
Rawal, A., McCoy, J., Rawat, D. B., Sadler, B. M., & Amant, R. S. (2021) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3133846 · Cited by 257
"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations
Lakkaraju, H., & Bastani, O. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375833 · Cited by 241
Schramowski, P., Stammer, W., Teso, S., Brugger, A., Herbert, F., Shao, X., Luigs, H.-G., Mahlein, A.-K., & Kersting, K. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0212-3 · Cited by 207
Adversarial attacks and defenses in explainable artificial intelligence: A survey
Baniecki, H., & Biecek, P. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102303 · Cited by 234
Evaluating Explanation Methods for Deep Learning in Security
Warnecke, A., Arp, D., Wressnegger, C., & Rieck, K. (2020) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp48549.2020.00018 · Cited by 167
Baur, L., Ditschuneit, K., Schambach, M., Kaymakci, C., Wollmann, T., & Sauer, A. (2024) · Energy and AI · doi:10.1016/j.egyai.2024.100358 · Cited by 152
Toward Explainable Artificial Intelligence for Regression Models: A methodological perspective
Letzgus, S., Wagner, P., Lederer, J., Samek, W., Müller, K., & Montavon, G. (2022) · IEEE Signal Processing Magazine · doi:10.1109/msp.2022.3153277 · Cited by 139
Self-citationBlack Box Attacks on Explainable Artificial Intelligence(XAI) methods in Cyber Security
Kuppa, A., & Le-Khac, N.-A. (2020) · 2020 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9206780 · Cited by 123
Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks
Räuker, T., Ho, A., Casper, S., & Hadfield-Menell, D. (2023) · 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00039 · Cited by 122
Robust Explainability: A tutorial on gradient-based attribution methods for deep neural networks
Nielsen, I. E., Dera, D., Rasool, G., Ramachandran, R. P., & Bouaynaya, N. C. (2022) · IEEE Signal Processing Magazine · doi:10.1109/msp.2022.3142719 · Cited by 135
Guided Integrated Gradients: an Adaptive Path Method for Removing Noise
Kapishnikov, A., Venugopalan, S., Avci, B., Wedin, B., Terry, M. A., & Bolukbasi, T. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00501 · Cited by 133
Survey on Explainable AI: Techniques, challenges and open issues
Abusitta, A., Li, M. Q., & Fung, B. C. M. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124710 · Cited by 98
Reliable Post hoc Explanations: Modeling Uncertainty in ExplainabilityPreprint
Slack, D., Hilgard, S., Singh, S., & Lakkaraju, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.05030 · Cited by 95
Explaining Explanations: Axiomatic Feature Interactions for Deep NetworksPreprint
Janizek, J. D., Sturmfels, P., & Lee, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.04138 · Cited by 95
SoK: Explainable Machine Learning for Computer Security Applications
Nadeem, A., Vos, D., Cao, C., Pajola, L., Dieck, S., Baumgartner, R., & Verwer, S. (2023) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp57164.2023.00022 · Cited by 79
Singh, Y., Hathaway, Q. A., Keishing, V., Salehi, S., Wei, Y., Horvat, N., Vera-Garcia, D. V., Choudhary, A., Mula Kh, A., Quaia, E., et al. (2025) · Bioengineering · doi:10.3390/bioengineering12080879 · Cited by 72
“Adversarial Vulnerability: Recent research demonstrates that explanations can be manipulated to appear reasonable while maintaining incorrect predictions [34].”
Invertible Concept-based Explanations for CNN Models with Non-negative Concept Activation Vectors
Zhang, R., Madumal, P., Miller, T., Ehinger, K. A., & Rubinstein, B. I. P. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17389 · Cited by 87
Toward Interpretable Machine Learning: Transparent Deep Neural Networks and BeyondPreprint
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Müller, K. (2020) · arXiv (Cornell University) · Cited by 70
Self-citationMohamed, E., Sirlantzis, K., & Howells, G. (2022) · Displays · doi:10.1016/j.displa.2022.102239 · Cited by 67
Exploring Evaluation Methods for Interpretable Machine Learning: A Survey
Alangari, N., Menaï, M. E. B., Mathkour, H. I., & Almosallam, I. A. (2023) · Information · doi:10.3390/info14080469 · Cited by 72
Application and Potential of Artificial Intelligence in Heart Failure: Past, Present, and Future
Yoon, M., Park, J. J., Hur, T., Hua, C.-H., Hussain, M., Lee, S., & Choi, D. (2023) · International Journal of Heart Failure · doi:10.36628/ijhf.2023.0050 · Cited by 66
Shapley explainability on the data manifoldPreprint
Frye, C., de Mijolla, D., Begley, T., Cowton, L., Stanley, M., & Feige, I. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.01272 · Cited by 104
Fairness via Explanation Quality
Dai, J. C., Upadhyay, S., Aïvodji, U., Bach, S. H., & Lakkaraju, H. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534159 · Cited by 72
You shouldn’t trust me: Learning models which conceal unfairness from multiple explanation methods.
Dimanov, B., Bhatt, U., Jamnik, M., & Weller, A. (2020) · Apollo (University of Cambridge) · doi:10.17863/cam.48825 · Cited by 58
An Experimental Study of Quantitative Evaluations on Saliency Methods
Li, X., Shi, Y., Li, H., Bai, W., Cao, C. C., & Chen, L. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467148 · Cited by 65
Self-explaining AI as an Alternative to Interpretable AI
Elton, D. C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-52152-3_10 · Cited by 60
XAI Handbook: Towards a Unified Framework for Explainable AI
Palacio, S., Lucieri, A., Munir, M., Ahmed, S., Hees, J., & Dengel, A. R. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00420 · Cited by 60
Leveraging explanations in interactive machine learning: An overview
Teso, S., Alkan, Ö., Stammer, W., & Daly, E. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1066049 · Cited by 76
Towards robust explanations for deep neural networks
Dombrowski, A.-K., Anders, C. J., Müller, K., & Kessel, P. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108194 · Cited by 62
Self-citationBMB-LIME: LIME with modeling local nonlinearity and uncertainty in explainability
Hung, Y.-H., & Lee, C. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111732 · Cited by 51
Gradient-based Analysis of NLP Models is Manipulable
Wang, J., Tuyls, J., Wallace, E., & Singh, S. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.24 · Cited by 56
Adversarial Attacks and Defenses
Liu, N., Du, M., Guo, R., Liu, H., & Hu, X. (2021) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3468507.3468519 · Cited by 56
Lee, J., Kim, E., Mok, J., & Yoon, S. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3166916 · Cited by 46
Robust and Stable Black Box ExplanationsPreprint
Lakkaraju, H., Arsov, N., & Bastani, O. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.06169 · Cited by 45
Revisiting the Performance-Explainability Trade-Off in Explainable Artificial Intelligence (XAI)
Crook, B., Schlüter, M., & Speith, T. (2023) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew57809.2023.00060 · Cited by 47
Rajpura, P., Cecotti, H., & Meena, Y. K. (2024) · Journal of Neural Engineering · doi:10.1088/1741-2552/ad6593 · Cited by 45
IG 2 : Integrated Gradient on Iterative Gradient Path for Feature Attribution
Zhuo, Y., & Ge, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3388092 · Cited by 46
Model extraction from counterfactual explanationsPreprint
Aïvodji, U., Bolot, A., & Gambs, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.01884 · Cited by 61
Adversarial Attacks in Explainable Machine Learning: A Survey of Threats Against Models and Humans
Vadillo, J., Santana, R., & Lozano, J. A. (2024) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1567 · Cited by 42
A Survey on the Robustness of Feature Importance and Counterfactual ExplanationsPreprint
Mishra, S., Dutta, S., Long, J., & Magazzeni, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.00358 · Cited by 38
Liu, H., Wu, Y., Yu, Z., & Zhang, N. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00120 · Cited by 46
Robustifying models against adversarial attacks by Langevin dynamics
Srinivasan, V., Rohrer, C., Marbán, A., Müller, K., Samek, W., & Nakajima, S. (2021) · Neural Networks · doi:10.1016/j.neunet.2020.12.024 · Cited by 41
Self-citationAlgorithmic Fairness and Vertical Equity: Income Fairness with IRS Tax Audit Models
Black, E., Elzayn, H., Chouldechova, A., Goldin, J. S., & Ho, D. E. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533204 · Cited by 42
Vulnerabilities of Connectionist AI Applications: Evaluation and Defense
Berghoff, C., Neu, M., & von Twickel, A. (2020) · Frontiers in Big Data · doi:10.3389/fdata.2020.00023 · Cited by 35
SAFARI: Versatile and Efficient Evaluations for Robustness of Interpretability
Huang, W., Zhao, X., Jin, G., & Huang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00190 · Cited by 33
Sequential Integrated Gradients: a simple but effective method for explaining language models
Enguehard, J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.477 · Cited by 58
Faithfulness Tests for Natural Language Explanations
Atanasova, P., Camburu, O.-M., Lioma, C., Lukasiewicz, T., Simonsen, J. G., & Augenstein, I. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2023.acl-short.25 · Cited by 69
ML-Based QoE Estimation in 5G Networks Using Different Regression Techniques
Schwarzmann, S., Marquezan, C. C., Trivisonno, R., Nakajima, S., Barriac, V., & Zinner, T. (2022) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2022.3179924 · Cited by 31
Do gradient-based explanations tell anything about adversarial robustness to android malware?
Melis, M., Scalas, M., Demontis, A., Maiorca, D., Biggio, B., Giacinto, G., & Roli, F. (2021) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-021-01393-7 · Cited by 30
NoiseGrad — Enhancing Explanations by Introducing Stochasticity to Model Weights
Bykov, K., Hedström, A., Nakajima, S., & Höhne, M. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i6.20561 · Cited by 34
How Much Can I Trust You? -- Quantifying Uncertainties in Explaining Neural NetworksPreprint
Bykov, K., Höhne, M., Müller, K., Nakajima, S., & Kloft, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.09000 · Cited by 35
Self-citationDisguising Attacks with Explanation-Aware Backdoors
Noppel, M., Peter, L., & Wressnegger, C. (2023) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46215.2023.10179308 · Cited by 35
A simple defense against adversarial attacks on heatmap explanationsPreprint
Rieger, L., & Hansen, L. K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.06381 · Cited by 29
Sun, J., Shi, W., Giuste, F., Vaghani, Y. S., Tang, L., & Wang, M. D. (2023) · Scientific Reports · doi:10.1038/s41598-023-46493-2 · Cited by 30
Smoothed Geometry for Robust AttributionPreprint
Wang, Z., Wang, H., Ramkumar, S., Fredrikson, M., Mardziel, P., & Datta, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06643 · Cited by 28
Synthetic Benchmarks for Scientific Research in Explainable Machine LearningPreprint
Liu, Y., Khandagale, S., White, C., & Neiswanger, W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.12543 · Cited by 25
Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces
Chormai, P., Herrmann, J., Müller, K., & Montavon, G. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3388275 · Cited by 36
Self-citationSoK: Explainable Machine Learning in Adversarial Environments
Noppel, M., & Wressnegger, C. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00021 · Cited by 29
Best of both worlds: local and global explanations with human-understandable conceptsPreprint
Schrouff, J., Baur, S., Hou, S., Mincu, D., Loreaux, E., Blanes, R., Wexler, J., Karthikesalingam, A., & Kim, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.08641 · Cited by 36
Saliency Methods for Explaining Adversarial Attacks
Gu, J., & Tresp, V. (2019) · arXiv · doi:10.48550/arxiv.1908.08413 · Cited by 23
Perturbing Inputs for Fragile Interpretations in Deep Natural Language Processing
Sinha, S., Chen, H., Sekhon, A., Ji, Y., & Qi, Y. (2021) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2021.blackboxnlp-1.33 · Cited by 30
Spurious Features Everywhere - Large-Scale Detection of Harmful Spurious Features in ImageNet
Neuhaus, Y., Augustin, M., Boreiko, V., & Hein, M. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01851 · Cited by 30
Explaining Bayesian Neural NetworksPreprint
Bykov, K., Höhne, M., Creosteanu, A., Müller, K., Klauschen, F., Nakajima, S., & Kloft, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.10346 · Cited by 27
Self-citationTowards the Unification and Robustness of Perturbation and Gradient Based ExplanationsPreprint
Agarwal, S., Jabbari, S., Agarwal, C., Upadhyay, S., Wu, Z. S., & Lakkaraju, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.10618 · Cited by 20
Liang, L., Liu, M., Elefteriades, J. A., & Sun, W. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107616 · Cited by 21
Attributional Robustness Training Using Input-Gradient Spatial Alignment
Singh, M., Kumari, N., Mangla, P., Sinha, A., Balasubramanian, V. N., & Krishnamurthy, B. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58583-9_31 · Cited by 22
On the Lack of Robust Interpretability of Neural Text Classifiers
Zafar, M. B., Donini, M., Slack, D., Archambeau, C., Das, S. R., & Kenthapadi, K. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.327 · Cited by 23
Adversarial Infidelity Learning for Model Interpretation
Liang, J., Bai, B., Cao, Y., Bai, K., & Wang, F. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403071 · Cited by 18
Finding Experts in Transformer ModelsPreprint
Suau, X., Zappella, L., & Apostoloff, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.07647 · Cited by 29
XAI enhancing cyber defence against adversarial attacks in industrial applications
Makridis, G., Theodoropoulos, S., Dardanis, D., Makridis, I., Separdani, M. M., Fatouros, G., Kyriazis, D., & Koulouris, P. (2022) · IEEE International Conference on Image Processing, Applications and Systems (IPAS) · doi:10.1109/ipas55744.2022.10052858 · Cited by 17
FIMAP: Feature Importance by Minimal Adversarial Perturbation
Chapman-Rounds, M., Bhatt, U., Pazos, E., Schulz, M.-A., & Georgatzis, K. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17362 · Cited by 18
Do Input Gradients Highlight Discriminative Features?Preprint
Shah, H., Jain, P., & Netrapalli, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.12781 · Cited by 23
A Human-Centric Perspective on Model Monitoring
Shergadwala, M., Lakkaraju, H., & Kenthapadi, K. (2022) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v10i1.21997 · Cited by 19
Backdoor Attacks on the DNN Interpretation System
Fang, S., & Choromanska, A. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19935 · Cited by 16
Preventing deception with explanation methods using focused sampling
Vreš, D., & Robnik‐Šikonja, M. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00900-w · Cited by 16
The Impact of Adversarial Attacks on Interpretable Semantic Segmentation in Cyber–Physical Systems
Gipiškis, R., Chiaro, D., Preziosi, M., Prezioso, E., & Piccialli, F. (2023) · IEEE Systems Journal · doi:10.1109/jsyst.2023.3281079 · Cited by 18
Diffeomorphic Counterfactuals With Generative Models
Dombrowski, A.-K., Gerken, J. E., Müller, K., & Kessel, P. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3339980 · Cited by 26
Self-citationInterpretable Deep Neural Network to Predict Estrogen Receptor Status from Haematoxylin-Eosin Images
Seegerer, P., Binder, A., Saitenmacher, R., Bockmayr, M. L., Alber, M., Jurmeister, P. S., Klauschen, F., & Müller, K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50402-1_2 · Cited by 15
Self-citationPirch, L., Warnecke, A., Wressnegger, C., & Rieck, K. (2021) · European Workshop on System Security · doi:10.1145/3447852.3458719 · Cited by 15
Fooling Partial Dependence via Data Poisoning
Baniecki, H., Kretowicz, W., & Biecek, P. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26409-2_8 · Cited by 21
Machine Learning Explainability and Robustness
Datta, A., Fredrikson, M., Leino, K. R. M., Lu, K., Sen, S., & Wang, Z. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3470806 · Cited by 15
On Saliency Maps and Adversarial Robustness
Mangla, P., Singh, V., & Balasubramanian, V. N. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-67661-2_17 · Cited by 15
Debiased-CAM to mitigate image perturbations with faithful visual explanations of machine learning
Zhang, W., Dimiccoli, M., & Lim, B. Y. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517522 · Cited by 21
Consistent Counterfactuals for Deep ModelsPreprint
Black, E., Wang, Z., Fredrikson, M., & Datta, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.03109 · Cited by 15
Pixel-Grounded Prototypical Part Networks
Carmichael, Z., Lohit, S., Cherian, A., Jones, M., & Scheirer, W. J. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00470 · Cited by 15
NeuralSanitizer: Detecting Backdoors in Neural Networks
Zhu, H., Zhao, Y., Zhang, S., & Chen, K. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3390599 · Cited by 14
Nullius in Explanans: an ethical risk assessment for explainable AI
Nannini, L., Huyskes, D., Panai, E., Pistilli, G., & Tartaro, A. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09800-7 · Cited by 16
Gan, Y., Mao, Y., Zhang, X., Ji, S., Pu, Y., Han, M., Yin, J., & Wang, T. (2022) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3548606.3559392 · Cited by 14
Robust Models Are More Interpretable Because Attributions Look NormalPreprint
Wang, Z., Fredrikson, M., & Datta, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.11257 · Cited by 14
Metadata
- Authors
- Dombrowski, A.-K., Alber, M., Anders, C. J., Ackermann, M., Müller, K.-R., & Kessel, P.
- Affiliations
- Technische Universität Berlin (DE)
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1906.07983
- OpenAlex
- W2951306478
- Cited by
- 171
- References
- 27
- Citations by year
- 2025: 17 · 2024: 29 · 2023: 36 · 2022: 19 · 2021: 41 · 2020: 24 · 2019: 5
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- explanation methods, neural network interpretability, explanation robustness, input perturbation, explainable AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
