Contrastive Explanations for Model Interpretability
Open accessAbstract
Contrastive explanations clarify why an event occurred in contrast to another.They are inherently intuitive to humans to both produce and comprehend.We propose a method to produce contrastive explanations in the latent space, via a projection of the input representation, such that only the features that differentiate two potential decisions are captured.Our modification allows model behavior to consider only contrastive reasoning, and uncover which aspects of the input are useful for and against particular decisions.Additionally, for a given input feature, our contrastive explanations can answer for which label, and against which alternative label, is the feature useful.We produce contrastive explanations via both highlevel abstract concept attribution and low-level input token/span attribution for two NLP classification benchmarks.Our findings demonstrate the ability of label-contrastive explanations to provide fine-grained interpretability of model decisions.1
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Model explanation and interpretability provide insights into decision-making.
“Provides insights into decision-making”
From AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications (2023) · cites this paper as [270]
Model explanation and interpretability facilitate understanding of model behavior and potential biases.
“Facilitates understanding of model behavior and potential biases”
From AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications (2023) · cites this paper as [270]
Model explanation and interpretability may not fully capture complex interactions in the model, leading to limited interpretability.
“May not fully capture interpretability complex interactions in the model, leading to limited”
From AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications (2023) · cites this paper as [270]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · see citing works
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302 · see citing works
Making things happen: a theory of causal explanation
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5840 · Cited by 3,383 · see citing works
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377 · see citing works
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342 · see citing works
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · see citing works
AllenNLP: A Deep Semantic Natural Language Processing Platform
Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N. F., Peters, M. E., Schmitz, M., & Zettlemoyer, L. (2018) · Workshop for NLP Open Source Software (NLP-OSS) · doi:10.18653/v1/w18-2501 · Cited by 1,273 · see citing works
Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., & Sayres, R. (2017) · arXiv · doi:10.48550/arxiv.1711.11279 · Cited by 1,240 · see citing works
(2013) · Cited by 1,126 · see citing works
Attention is not not Explanation
Wiegreffe, S., & Pinter, Y. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1002 · Cited by 966 · see citing works
Annotation Artifacts in Natural Language Inference Data
Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S., & Smith, N. A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2017 · Cited by 907 · see citing works
Self-citationRationalizing Neural Predictions
Lei, T., Barzilay, R., & Jaakkola, T. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1011 · Cited by 777 · see citing works
Hypothesis Only Baselines in Natural Language Inference
Poliak, A., Naradowsky, J., Haldar, A., Rudinger, R., & Durme, B. V. (2018) · International Workshop on Semantic Evaluation · doi:10.18653/v1/s18-2023 · Cited by 685 · see citing works
The pascal recognising textual entailment challenge
Dagan, I., & Glickman, O. (2005) · Cited by 579 · see citing works
Understanding Neural Networks through Representation ErasurePreprint
Li, J., Monroe, W., & Jurafsky, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.08220 · Cited by 493 · see citing works
Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
Zintgraf, L. M., Cohen, T. S., Adel, T., & Welling, M. (2017) · arXiv · doi:10.48550/arxiv.1702.04595 · Cited by 487 · see citing works
Designing and Interpreting Probes with Control Tasks
Hewitt, J., & Liang, P. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1275 · Cited by 393 · see citing works
Stepin, I., Alonso, J. M., Catala, A., & Pereira-Farina, M. (2021) · IEEE Access · doi:10.1109/access.2021.3051315 · Cited by 376 · see citing works
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics
Swayamdipta, S., Schwartz, R., Lourie, N., Wang, Y., Hajishirzi, H., Smith, N. A., & Choi, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.746 · Cited by 320 · see citing works
Self-citationNull It Out: Guarding Protected Attributes by Iterative Nullspace Projection
Ravfogel, S., Elazar, Y., Gonen, H., Twiton, M., & Goldberg, Y. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.647 · Cited by 287 · see citing works
Self-citationDomain-Adversarial Training of Neural NetworksPreprint
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.07818 · Cited by 286 · see citing works
Evaluating Models’ Local Decision Boundaries via Contrast Sets
Gardner, M., Artzi, Y., Basmov, V., Berant, J., Bogin, B., Chen, S., Dasigi, P., Dua, D., Elazar, Y., Gottumukkala, A., et al. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.117 · Cited by 265 · see citing works
Self-citationProblems with Shapley-value-based explanations as feature importance measures
Kumar, I. E., Venkatasubramanian, S., Scheidegger, C., & Friedler, S. (2020) · arXiv · doi:10.48550/arxiv.2002.11097 · Cited by 228 · cited 1× · see citing works
“These could include discrete features in the input ( textual highlights ; Lei et al., 2016; see Fig.”
BackgroundContrastive explanation: a structural-model approach
Miller, T. (2021) · The Knowledge Engineering Review · doi:10.1017/s0269888921000102 · Cited by 186 · see citing works
Learning The Difference That Makes A Difference With Counterfactually-Augmented Data
Kaushik, D., Hovy, E. H., & Lipton, Z. C. (2020) · International Conference on Learning Representations · Cited by 174 · see citing works
Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals
Elazar, Y., Ravfogel, S., Jacovi, A., & Goldberg, Y. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00359 · Cited by 126 · see citing works
Self-citationExplaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions
Han, X., Wallace, B., & Tsvetkov, Y. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.492 · Cited by 111 · see citing works
Explaining NLP Models via Minimal Contrastive Editing (MiCE)
Ross, A., Marasović, A., & Peters, M. E. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.336 · Cited by 93 · see citing works
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language InferencePreprint
McCoy, R. T., Pavlick, E., & Linzen, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.01007 · Cited by 86 · see citing works
The problem of causal selection.
Hesslow, G. (1988) · Cited by 86 · see citing works
Generating Counterfactual Explanations with Natural LanguagePreprint
Hendricks, L. A., Hu, R., Darrell, T., & Akata, Z. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.09809 · Cited by 81 · see citing works
Generating Counterfactual and Contrastive Explanations using SHAPPreprint
Rathi, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.09293 · Cited by 76 · see citing works
Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?
Ravichander, A., Belinkov, Y., & Hovy, E. H. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.295 · Cited by 75 · see citing works
Aligning Faithful Interpretations with their Social Attribution
Jacovi, A., & Goldberg, Y. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00367 · Cited by 74 · see citing works
Self-citationCausal Mediation Analysis for Interpreting Neural NLP: The Case of Gender BiasPreprint
Vig, J., Gehrmann, S., Belinkov, Y., Qian, S., Nevo, D., Sakenis, S., Huang, J., Singer, Y., & Shieber, S. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.12265 · Cited by 63 · see citing works
The Intriguing Relation Between Counterfactual Explanations and Adversarial Examples
Freiesleben, T. (2021) · Minds and Machines · doi:10.1007/s11023-021-09580-9 · Cited by 58 · see citing works
Investigating Transferability in Pretrained Language Models
Tamkin, A., Singh, T., Giovanardi, D., & Goodman, N. D. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.125 · Cited by 46 · see citing works
Interpretation of NLP models through input marginalization
Kim, S., Yi, J., Kim, E., & Yoon, S. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.255 · Cited by 45 · see citing works
What's in a Name? Reducing Bias in Bios without Access to Protected AttributesPreprint
Romanov, A., De‐Arteaga, M., Wallach, H., Chayes, J., Borgs, C., Chouldechova, A., Geyik, S. C., Kenthapadi, K., Rumshisky, A., & Kalai, A. T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.05233 · Cited by 34 · see citing works
Elazar, Y., & Goldberg, Y. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00280 · Cited by 20 · see citing works
Self-citationPolyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving ModelsPreprint
Wu, T., Ribeiro, M. T., Heer, J., & Weld, D. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.00288 · Cited by 19 · see citing works
Meyes, R., de Puiseau, C. W., Posada-Moreno, A. F., & Meisen, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.01254 · Cited by 13 · see citing works
Contrastive Examples for Addressing the Tyranny of the MajorityPreprint
Sharmanska, V., Hendricks, L. A., Darrell, T., & Quadrianto, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.06524 · Cited by 10 · see citing works
Flagged first, then by citations
Research works printed but not linked: 11
- Maria De-Arteaga, Alexey Romanov, Hanna M. Wal- lach, Jennifer T. Chayes, Christian Borgs, Alexan- dra Chouldechova, Sahin Cem Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. 2019. Bias in bios: A case study of semantic representation bias in a high-stakes setting. In Proceedings of the Confer- ence on Fairness, Accountability, and Transparency, FAT* 2019, Atlanta, GA, USA, January 29-31, 2019, pages 120-128. ACM. · DOI · Paper page
- Amir Feder, Nadav Oved, Uri Shalit, and Roi Re- ichart. 2021. CausaLM: Causal Model Explanation Through Counterfactual Language Models. Compu- tational Linguistics, 47(2):333-386. · DOI
- Mario Giulianelli, Jack Harding, Florian Mohnert, Dieuwke Hupkes, and Willem Zuidema. 2018. Un- der the hood: Using diagnostic classifiers to in- vestigate and improve how language models track agreement information. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and In- terpreting Neural Networks for NLP, pages 240- 248.
- Denis J Hilton. 1988. Logic and causal attribution. In Part of this chapter is the text of a paper read at the symposium," Attitudes and Attribution: A Sym- posium in Honour of Jos Jaspars," convened at the Annual Conference of the British Psychological So- ciety, Sheffield, England, Apr 3-5, 1986. New York University Press.
- Denis J Hilton and Ben R Slugoski. 1986. Knowledge- based causal attribution: The abnormal conditions focus model. Psychological review, 93(1):75.
- Alon Jacovi and Yoav Goldberg. 2020b. Towards faith- fully interpretable NLP systems: How should we de- fine and evaluate faithfulness? In Proceedings of the 58th Annual Meeting of the Association for Compu- tational Linguistics, pages 4198-4205, Online. As- sociation for Computational Linguistics. · DOI · Paper page
- PMLR.
- Peter Lipton. 1990. Contrastive explanation. Royal Institute of Philosophy Supplements, 27:247-266. · DOI
- Aakanksha Naik, Abhilasha Ravichander, Norman Sadeh, Carolyn Rose, and Graham Neubig. 2018. Stress test evaluation for natural language inference. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2340-2353, Santa Fe, New Mexico, USA. Association for Com- putational Linguistics.
- Jacob Sippy, Gagan Bansal, and Daniel S Weld. 2020. Data staining: A method for comparing faithfulness of explainers. In Proc. of ICML Workshop on Hu- man Interpretability in Machine Learning (WHI).
- PMLR.
Cited by
Sorted by citations
AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications
Chen, P., Wu, L., & Wang, L. (2023) · Applied Sciences · doi:10.3390/app131810258 · Cited by 155 · see citing works
EXplainable Artificial Intelligence (XAI)—From Theory to Methods and Applications
Ortigossa, E. S., Gonçalves, T., & Nonato, L. G. (2024) · IEEE Access · doi:10.1109/access.2024.3409843 · Cited by 124 · see citing works
Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations
Chen, Y., Zhong, R., Ri, N., Zhao, C., He, H., Steinhardt, J., Yu, Z., & McKeown, K. (2023) · arXiv · doi:10.48550/arxiv.2307.08678 · Cited by 5 · see citing works
“One possible solution is contrastive simulation (Jacovi et al., 2021; Miller, 2021; Yin and Neubig, 2022), where a human simulator is shown the model’s output mixed with fake outputs (distractors) and selects which output is from the model based on the explanation.”
Background
- Authors
- Jacovi, A., Swayamdipta, S., Ravfogel, S., Elazar, Y., Choi, Y., & Goldberg, Y.
- Affiliations
- Bar-Ilan University (IL)
- Allen Institute for Artificial Intelligence (US)
- University of Washington (US)
- Venue
- Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
- Published
- 2021
- Pages
- 1597-1611
- OpenAlex
- W3134100302
- Cited by
- 83
- References
- 45
- Citations by year
- 2026: 10 · 2025: 16 · 2024: 20 · 2023: 18 · 2022: 12 · 2021: 7
- FWCI
- 6.58
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- arXiv (Cornell University)
- Funders
- Allen Institute for Artificial Intelligence
- European Commission (HORIZON2020)
- H2020 European Research Council (802774)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- contrastive explanations, model interpretability, latent space projection, explainable AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
