CoChat
Sign Up
PreprintCited by 310Explainable Artificial Intelligence (XAI)Software Testing and Debugging TechniquesAdversarial Robustness in Machine Learning

On the Robustness of Interpretability Methods

Open access
Alvarez-Melis, David, Jaakkola, Tommi S.·2018·arXiv
DOI 10.48550/arxiv.1806.08049arXiv 1806.08049
Open this paper in CoChat — free accountOpen PDF

Abstract

We argue that robustness of explanations---i.e., that similar inputs should give rise to similar explanations---is a key desideratum for interpretability. We introduce metrics to quantify robustness and demonstrate that current methods do not perform well according to these metrics. Finally, we propose ways that robustness can be enforced on existing interpretability approaches.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1806.08049

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Post hoc explanation methods often provide misleading information about model behavior, with local approximations achieving poor fidelity to actual decision boundaries.

    “Studies demonstrate that post hoc explanation methods often provide misleading information about model behavior, with local approximations achieving poor fidelity to actual decision boundaries [14,15,16,17,18,19].”

    From Beyond Post hoc Explanations: A Comprehensive Framework for Accountable AI in Medical Imaging Through Transparency, Interpretability, and Explainability (2025) · cites this paper as [14]

    The Robustness section (pp. 1–2) reports that LIME and SHAP explanations for a nonlinear classifier can vary substantially between neighboring inputs, and the Introduction (p. 1) warns that a single explanation can give a false sense of understanding. The paper does not establish that local approximations have poor fidelity to actual decision boundaries; it evaluates explanation stability rather than decision-boundary fidelity. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • Explanation fidelity was evaluated using the causal-based methodology of Alvarez-Melis & Jaakkola.

    “Explanation fidelity was evaluated using the established causal-based methodology by Alvarez-Melis & Jaakkola [14].”

    From Beyond Post hoc Explanations: A Comprehensive Framework for Accountable AI in Medical Imaging Through Transparency, Interpretability, and Explainability (2025) · cites this paper as [14]

    Section 2, p. 2 describes a local Lipschitz measure for evaluating the robustness of explanations, not a causal-based method for evaluating fidelity. Searches for causal-based methodology and explanation fidelity found no passage in our copy establishing the claimed method. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 15

  1. Alvarez-Melis, David and Jaakkola, Tommi S. A causal framework for explaining the predictions of black-box sequence-to-sequence models. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing , pp. 412-421, 2017. URL https: //www.aclweb.org/anthology/D17-1042 .
  2. Alvarez-Melis, David and Jaakkola, Tommi S. Towards Robust Interpretability with Self-explaining Neural Networks. arXiv preprint:1806.07538 , 2018. · arXiv
  3. Bach, Sebastian, Binder, Alexander, Montavon, Gr´ egoire, Klauschen, Frederick, M¨ uller, Klaus Robert, and Samek, Wojciech. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE , 10(7), 2015. ISSN 19326203. doi: 10.1371/ journal.pone.0130140.
  4. Hein, Matthias and Andriushchenko, Maksym. Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation. In Guyon, I, Luxburg, U V, Bengio, S, Wallach, H, Fergus, R, Vishwanathan, S, and Garnett, R (eds.), Advances in Neural Information Processing Systems 30 , pp. 2263-2273. Curran Associates, Inc., 2017.
  5. Kindermans, P.-J., Hooker, S, Adebayo, J, Alber, M, Sch¨ utt, K.˜T., D¨ ahne, S, Erhan, D, and Kim, B. The (Un)reliability of saliency methods. NIPS workshop on Explaining and Visualizing Deep Learning , 2017.
  6. Kolter, J Zico and Wong, Eric. Provable defenses against adversarial examples via the convex outer adversarial polytope. arXiv preprint arXiv:1711.00851 , 2017. · arXiv
  7. LeCun, Yann, Bottou, L´ eon, Bengio, Yoshua, and Haffner, Patrick. Gradient-based learning applied to document recognition. Proceedings of the IEEE , 86(11):22782323, 1998. ISSN 00189219. doi: 10.1109/5.726791. · DOI · Paper page
  8. Lundberg, Scott and Lee, Su-In. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems 30 , pp. 4768-4777, 2017. URL http://arxiv.org/abs/ 1705.07874 .
  9. Raghunathan, Aditi, Steinhardt, Jacob, and Liang, Percy. Certified defenses against adversarial examples. arXiv preprint arXiv:1801.09344 , 2018. · arXiv
  10. Ribeiro, Marco Tulio, Singh, Sameer, and Guestrin, Carlos. 'Why Should I Trust You?': Explaining the Predictions of Any Classifier. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pp. 1135-1144, New York, NY, USA, 2016. ACM. ISBN 978-14503-4232-2. doi: 10.1145/2939672.2939778. URL http://arxiv.org/abs/1602.04938http: //doi.acm.org/10.1145/2939672.2939778 . · DOI · Paper page
  11. Selvaraju, Ramprasaath R., Das, Abhishek, Vedantam, Ramakrishna, Cogswell, Michael, Parikh, Devi, and Batra, Dhruv. Grad-cam: Why did you say that? visual explanations from deep networks via gradient-based localization. In ICCV , 2017. URL http://arxiv.org/ abs/1610.02391 . · arXiv
  12. Shrikumar, Avanti, Greenside, Peyton, Shcherbina, Anna, and Kundaje, Anshul. Not just a black box: Learning important features through propagating activation differences. arXiv preprint arXiv:1605.01713 , 2016. · arXiv
  13. Sundararajan, Mukund, Taly, Ankur, and Yan, Qiqi. Axiomatic attribution for deep networks. arXiv preprint arXiv:1703.01365 , 2017. · arXiv
  14. Weng, Tsui-Wei, Zhang, Huan, Chen, Pin-Yu, Yi, Jinfeng, Su, Dong, Gao, Yupeng, Hsieh, Cho-Jui, and Daniel, Luca. Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach. arXiv preprint arXiv:1801.10578 , 2018. · arXiv
  15. Zeiler, Matthew D and Fergus, Rob. Visualizing and understanding convolutional networks. In European conference on computer vision , pp. 818-833. Springer, 2014.

Cited by

Sorted by citations

Authors
Alvarez-Melis, D., & Jaakkola, T. S.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2018
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1806.08049
arXiv
1806.08049
OpenAlex
W2809283485
Cited by
310
References
3
Citations by year
2026: 10 · 2025: 29 · 2024: 63 · 2023: 65 · 2022: 56 · 2021: 51 · 2020: 25 · 2019: 10 · 2018: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
interpretability, explainable AI, robustness metrics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review