e-SNLI: Natural language inference with natural language explanations
Open accessAbstract
In order for machine learning to garner widespread public adoption, models must be able to provide interpretable and robust explanations for their decisions, as well as learn from human-provided explanations at train time. In this work, we extend the Stanford Natural Language Inference dataset with an additional layer of human-annotated natural language explanations of the entailment relations. We further implement models that incorporate these explanations into their training process and output them at test time. We show how our corpus of explanations, which we call e-SNLI, can be used for various goals, such as obtaining full sentence justifications of a model's decisions, improving universal sentence representations and transferring to out-of-domain NLI datasets. Our dataset thus opens up a range of research directions for using natural language explanations, both for improving models and for asserting their trust.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Rationale supervision work such as e-SNLI and ERASER provides gold explanations for post hoc evaluation or training but does not embed quality criteria into the generation process itself.
“Rationale supervision work, such as e-SNLI (Camburu et al., 2018) and ERASER (DeYoung et al., 2020), provides gold explanations for post hoc evaluation or training but does not embed quality criteria into the generation process itself.”
From GIER: Gap-Driven Self-Refinement for Large Language Models (2025) · cites this paper as Camburu et al., 2018
The Introduction (p. 2) identifies e-SNLI as a corpus of human-annotated explanations intended for training and generation, and §4.3 (p. 6) reports manual evaluation of generated explanations. The paper does not establish the broader claim that quality criteria are absent from the generation process: §3 (p. 3) describes criteria used to collect and filter explanations, while §4.3 reports an attention-based model designed to improve explanation quality. (LLM, conf=0.80)
Experimental — accuracy not yet measuredResearchers increasingly draw on LLMs' own text generation capabilities to produce natural language explanations alongside predictions.
“researchers increasingly draw on an LLMs' own text generation capabilities to produce natural language explanations (NLEs) alongside predictions [22,23]”
From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [22]
Section 4.2 (p. 4) describes a model that predicts a label and generates a natural-language explanation for it. But Section 4 (p. 3) identifies its explanation decoder as an LSTM, not an LLM, and the paper does not establish that researchers are increasingly using LLMs for this purpose. (LLM, conf=0.88)
Experimental — accuracy not yet measuredAlthough LLMs can be prompted to self-explain, these explanations are unreliable.
“Although LLMs can be prompted to self-explain [2], these explanations are unreliable [8].”
From ProSLM: A Prolog Synergized Language Model for explainable Domain Specific Knowledge Based Question Answering (2024) · cites this paper as [2]
Section 4 (p. 3) describes models trained on e-SNLI to predict a label and generate an explanation, supporting the general point that a model can explain its own prediction. It does not describe prompting LLMs to self-explain. The separate claim that explanations are unreliable is attributed to [8], not to this paper. (LLM, conf=0.86)
Experimental — accuracy not yet measuredThe PredictAndExplain method requires access to a reliable explanation-augmented dataset for training.
“The PredictAndExplain method, introduced in [2], illustrates that models can be trained to generate explanations of their outputs without sacrificing performance, but this approach requires access to a reliable explanation-augmented dataset for training.”
From ProSLM: A Prolog Synergized Language Model for explainable Domain Specific Knowledge Based Question Answering (2024) · cites this paper as [2]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1× · see citing works
“Therefore, our encoders are 2048-bidirectional-LSTMs [15] with max-pooling, resulting in a sentence representation dimension of 4096.”
MethodImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1× · see citing works
“In computer vision, pretrained ImageNet-based encoders [10] provide standard image feature extractors for other downstream tasks.”
BackgroundNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 2× · see citing works
“Attention-based models, such as [2, 25], offer some degree of interpretability and have been shown to also improve performance on downstream tasks.”
BackgroundShow, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · see citing works
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377 · cited 2× · see citing works
“To address this deficiency, we have collected a large corpus of human-annotated explanations for the Stanford Natural Language Inference (SNLI) dataset [3].”
BackgroundMethodA Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342 · cited 1× · see citing works
“We test how our explanations affect the direct transfer in both label prediction and explanation generation by looking at SICK-E [20] and MultiNLI [26].”
BackgroundSupervised Learning of Universal Sentence Representations from Natural Language Inference Data
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1070 · Cited by 2,049 · cited 8× · see citing works
“Throughout our experiments, our models follow the architecture presented in Conneau et al. [7], as we build directly on top of their code 2 .”
BackgroundMethodInfluentialA Decomposable Attention Model for Natural Language Inference
Parikh, A. P., Täckström, O., Das, D., & Uszkoreit, J. (2016) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1244 · Cited by 1,453 · cited 3× · see citing works
“Glockner et al. show that models achieving high accuracies on SNLI, such as [21, 22, 6], show dramatically reduced performance on this simpler dataset, while the model of Chen et al. [5] is more robust due to incorporating exter-nal knowledge.”
BackgroundResultKiros, R., Zhu, Y., Salakhutdinov, R., Zemel, R. S., Torralba, A., Urtasun, R., & Fidler, S. (2015) · arXiv · doi:10.48550/arxiv.1506.06726 · Cited by 1,293 · cited 2× · see citing works
“Traditional approaches make use of very large unsupervised datasets [17, 14], taking weeks to train.”
BackgroundAnnotation Artifacts in Natural Language Inference Data
Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S., & Smith, N. A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2017 · Cited by 907 · cited 3× · see citing works
“Recently, Gururangan et al. [13] cast doubt on whether models trained on SNLI are learning to understand language, or are largely fixating on spurious correlations, also called artifacts.”
BackgroundInfluentialA SICK cure for the evaluation of compositional distributional semantic models
Marelli, M., Menini, S., Baroni, M., Bentivogli, L., Raffaella Bernardi, B., & Zamparelli, R. (2014) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/39qdhuevzbqa · Cited by 680 · cited 2× · see citing works
“7% when training on SNLI and evaluating on SICK-E [20].”
BackgroundReasoning about Entailment with Neural AttentionPreprint
Rocktäschel, T., Grefenstette, E., Hermann, K. M., Kočiský, T., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.06664 · Cited by 583 · cited 3× · see citing works
“Attention-based models, such as [2, 25], offer some degree of interpretability and have been shown to also improve performance on downstream tasks.”
BackgroundResultSelf-citationLearning Distributed Representations of Sentences from Unlabelled Data
Hill, F., Cho, K., & Korhonen, A. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1162 · Cited by 566 · cited 2× · see citing works
“Traditional approaches make use of very large unsupervised datasets [17, 14], taking weeks to train.”
BackgroundHuman Attention in Visual Question Answering: Do Humans and Deep Networks look at the same regions?
Das, A., Agrawal, H., Zitnick, C. L., Parikh, D., & Batra, D. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1092 · Cited by 431 · cited 1× · see citing works
“Similar to the evaluation performed for visual question answering in Das et al. [8], our highlighted words could provide a source of supervision and evaluation for attention models [25, 22] or post-hoc explanation models where the explanation consists of a subset of the input.”
ResultMultimodal Explanations: Justifying Decisions and Pointing to the Evidence
Park, D. H., Hendricks, L. A., Akata, Z., Rohrbach, A., Schiele, B., Darrell, T., & Rohrbach, M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00915 · Cited by 418 · cited 1× · see citing works
“In this direction, and very similar to our approach, is the recent work by Park et al. [23], who introduce two datasets of natural language explanations for the tasks of visual question-answering and activity recognition.”
ResultProgram Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems
Ling, W., Yogatama, D., Dyer, C., & Blunsom, P. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1015 · Cited by 325 · cited 1× · see citing works
“Another work in this direction is that of Ling et al. [18], who introduced a dataset of textual justifications for solving math problems and formulate the task in terms of program execution.”
BackgroundSelf-citationBreaking NLI Systems with Sentences that Require Simple Lexical Inferences
Glockner, M., Shwartz, V., & Goldberg, Y. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2103 · Cited by 319 · cited 1× · see citing works
“Glockner et al. [11] introduce a toy dataset, BreakingNLI, to test whether natural language inference models capture world knowledge and generalize beyond statistical regularities.”
BackgroundNatural Language Inference over Interaction SpacePreprint
Gong, Y., Luo, H., & Zhang, J. (2017) · International Conference on Learning Representations · doi:10.48550/arxiv.1709.04348 · Cited by 286 · cited 1× · see citing works
“Attention mechanisms in neural networks brought consistent improvements over the non-attention counter-parts in various areas, such as computer vision [27], speech [4], or natural language processing [12, 2].”
BackgroundA causal framework for explaining the predictions of black-box sequence-to-sequence models
Alvarez-Melis, D., & Jaakkola, T. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1042 · Cited by 185 · cited 1× · see citing works
“Alvarez-Melis and Jaakkola [1] introduces a similar approach but for structured prediction, where a variational autoencoder provides relevant perturbations of the inputs that are then used to infer pairs of input-output tokens that are causally related.”
BackgroundShortcut-Stacked Sentence Encoders for Multi-Domain Inference
Nie, Y., & Bansal, M. (2017) · RepEval@EMNLP · doi:10.18653/v1/w17-5308 · Cited by 129 · cited 2× · see citing works
“Glockner et al. show that models achieving high accuracies on SNLI, such as [21, 22, 6], show dramatically reduced performance on this simpler dataset, while the model of Chen et al. [5] is more robust due to incorporating exter-nal knowledge.”
BackgroundEvaluating Compositionality in Sentence EmbeddingsPreprint
Dasgupta, I., Guo, D., Stuhlmüller, A., Gershman, S. J., & Goodman, N. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04302 · Cited by 100 · cited 1× · see citing works
“For example, Dasgupta et al. [9] assembled a dataset to test whether inference models actually capture compositionality beyond word level.”
BackgroundJansen, P. A., Wainwright, E., Marmorstein, S., & Morrison, C. T. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/28283yn8kzpn · Cited by 56 · cited 1× · see citing works
“Jansen et al. [16] provided a dataset of natural language explanation graphs for elementary science questions.”
BackgroundModelling Interaction of Sentence Pair with Coupled-LSTMs
Liu, P., Qiu, X., Zhou, Y., Chen, J., & Huang, X. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1176 · Cited by 46 · cited 1× · see citing works
“The SNLI dataset [3], containing 570 K data points of human-generated triples (premise, hypothesis, label), has driven the development of a large number of neural network models [25, 21, 22, 6, 19, 5, 7].”
Background
Flagged first, then by citations
4 of the 27 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- [4][4] Chan, W., Jaitly, N., Le, Q. V., and Vinyals, O. (2015). Listen, attend and spell. CoRR , abs/1508.01211. - · arXiv
- [5][5] Chen, Q., Zhu, X., Ling, Z., Inkpen, D., and Wei, S. (2017). Natural language inference with external knowledge. CoRR , abs/1711.04289. - · arXiv
- [6][6] Chen, Q., Zhu, X., Ling, Z., Wei, S., and Jiang, H. (2016). Enhancing and combining sequential and tree LSTM for natural language inference. CoRR , abs/1609.06038. - · arXiv
- [10][10] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). ImageNet: A largescale hierarchical image database. In Proc. of CVPR . -
- [19][19] Liu, P., Qiu, X., and Huang, X. (2016). Modelling interaction of sentence pair with coupledlstms. CoRR , abs/1605.05573. - · arXiv
- [24][24] Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). "why should I trust you?": Explaining the predictions of any classifier. CoRR , abs/1602.04938. - · arXiv
Cited by
Sorted by citations
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505 · see citing works
Self-Consistency Improves Chain of Thought Reasoning in Language Models
Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2203.11171 · Cited by 829 · see citing works
“, 2018), and (2) Natural Language Inference: e-SNLI (Camburu et al., 2018), ANLI (Nie et al.”
BackgroundNauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y., Schlötterer, J., van Keulen, M., & Seifert, C. (2023) · ACM Computing Surveys · doi:10.1145/3583558 · Cited by 590 · see citing works
HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection
Mathew, B., Saha, P., Yimam, S. M., Biemann, C., Goyal, P., & Mukherjee, A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i17.17745 · Cited by 585 · see citing works
“Such rationales have been used in common-sense explanations (Rajani et al. 2019), e-SNLI (Camburu et al. 2018) and several other tasks (DeYoung et al. 2020).”
BackgroundExplainable Deep Learning: A Field Guide for the Uninitiated
Ras, G., Xie, N., van Gerven, M. A. J., & Doran, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13200 · Cited by 486 · see citing works
Explain Yourself! Leveraging Language Models for Commonsense Reasoning
Rajani, N. F., McCann, B., Xiong, C., & Socher, R. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1487 · Cited by 421 · see citing works
“Camburu et al. (2018) empirically show that transferring explanations on the natural language inference (NLI) problem from SNLI to MultiNLI performs very poorly and is still an open challenging problem.”
BackgroundMethodInfluentialHsieh, C.-Y., Li, C.-L., Yeh, C.-K., Nakhost, H., Fujii, Y., Ratner, A., Krishna, R., Lee, C.-Y., & Pfister, T. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.507 · Cited by 316 · see citing works
Abductive Commonsense ReasoningPreprint
Bhagavatula, C., Bras, R. J. L., Malaviya, C., Sakaguchi, K., Holtzman, A., Rashkin, H., Downey, D., Yih, S., & Choi, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.05739 · Cited by 331 · see citing works
“Related Datasets Our new resource ART complements ongoing efforts in building resources for natural language inference (Dagan et al., 2006; MacCartney & Manning, 2009; Bowman et al., 2015; Williams et al., 2018a; Camburu et al., 2018).”
BackgroundERASER: A Benchmark to Evaluate Rationalized NLP Models
DeYoung, J., Jain, S., Rajani, N. F., Lehman, E., Xiong, C., Socher, R., & Wallace, B. C. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.408 · Cited by 561 · see citing works
“Following Talmor et al. (2019), we repartition the training and validation sets to provide a canonical test split. e-SNLI (Camburu et al., 2018).”
MethodHuang, F., Kwak, H., & An, J. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543873.3587368 · Cited by 247 · see citing works
“Moreover, it has been a challenging problem to provide the explainability of those models by using natural language explanations (NLEs) [4, 8].”
BackgroundLarge Language Models Can Self-Improve
Huang, J., Gu, S. S., Hou, L., Wu, Y., Wang, X., Yu, H., & Han, J. (2022) · arXiv · doi:10.48550/arxiv.2210.11610 · Cited by 230 · see citing works
Measuring and improving consistency in pretrained language models
Elazar, Y., Kassner, N., Ravfogel, S., Ravichander, A., Hovy, E., Schütze, H., & Goldberg, Y. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00410 · Cited by 220 · see citing works
“Li et al. (2019) measure and mitigate inconsistency in natural language inference (NLI), and finally, Camburu et al. (2020) propose a method for measuring inconsistencies in NLI explanations (Camburu et al., 2018).”
MethodHolzinger, A., Dehmer, M., Emmert‐Streib, F., Cucchiara, R., Augenstein, I., Ser, J. D., Samek, W., Jurišica, I., & Díaz-Rodríguez, N. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.10.007 · Cited by 215 · see citing works
“Yet other works rely on hybrid approaches, where extractive summarization is followed by abstractive summarization [206,207].”
BackgroundPost-hoc Interpretability for Neural NLP: A Survey
Madsen, A., Reddy, S., & Chandar, S. (2022) · ACM Computing Surveys · doi:10.1145/3546577 · Cited by 229 · see citing works
“…for these While those methods are often quite general, they are not discussed in this survey which focuses on post-hoc methods. the year 's best and most unpredictable comedy The idea is that by enforcing the model to reason about its behavior, the model can generalize better [23, 63–65, 69, 92].”
BackgroundLarge Language Models Can Self-Improve
Huang, J., Gu, S., Hou, L., Wu, Y., Wang, X., Yu, H., & Han, J. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.67 · Cited by 230 · see citing works
A Diagnostic Study of Explainability Techniques for Text Classification
Atanasova, P., Simonsen, J. G., Lioma, C., & Augenstein, I. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.263 · Cited by 191 · see citing works
“Some studies propose the generation of text serving as an explanation, e.g., (Camburu et al., 2018; Lei et al., 2016; Atanasova et al., 2020a).”
BackgroundTowards Explainable NLP: A Generative Explanation Framework for Text Classification
Liu, H., Yin, Q., & Wang, W. Y. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1560 · Cited by 167 · see citing works
“In recent years, lots of works have been done to solve text classification problems, but just a few of them have explored the explainability of their systems (Camburu et al., 2018; Ouyang et al., 2018).”
BackgroundCan language models learn from explanations in context?
Lampinen, A., Dasgupta, I., Chan, S., Mathewson, K., Tessler, M., Creswell, A., McClelland, J., Wang, J., & Hill, F. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.38 · Cited by 142 · see citing works
Gaur, M., Faldu, K., & Sheth, A. (2021) · IEEE Internet Computing · doi:10.1109/mic.2020.3031769 · Cited by 140 · see citing works
“In such scenarios, it is exceptionally challenging to probe the model’s mechanism without the support of background knowledge [1].”
BackgroundGenerating Fact Checking Explanations
Atanasova, P., Simonsen, J. G., Lioma, C., & Augenstein, I. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.656 · Cited by 143 · see citing works
DATA-GRU: Dual-Attention Time-Aware Gated Recurrent Unit for Irregular Multivariate Time Series
Tan, Q., Ye, M., Yang, B., Liu, S., Ma, A. J., Yip, T. C., Wong, G. L., & Yuen, P. C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5440 · Cited by 128 · see citing works
“Machine learning methods have been successfully applied in many areas, e.g., speech recognition (Afouras et al. 2018), computer version (Ye et al. 2019a; 2019b; 2019c; Fu et al. 2019), natural language processing (Camburu et al. 2018), energy prediction (Yuan et al. 2017; 2015; Kang et al. 2017).”
BackgroundMethodGet Your Vitamin C! Robust Fact Verification with Contrastive Evidence
Schuster, T., Fisch, A., & Barzilay, R. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.52 · Cited by 126 · see citing works
“Moreover, we use the revision history as distant supervision for word-level rationales, allowing for finer-grained explanations (Camburu et al., 2018; Lei et al., 2016; Portelli et al., 2020; Thorne et al., 2019b).”
MethodTowards Faithful Model Explanation in NLP: A Survey
Lyu, Q., Apidianaki, M., & Callison-Burch, C. (2024) · Computational Linguistics · doi:10.1162/coli_a_00511 · Cited by 143 · see citing works
“2016), language (Camburu et al. 2018), multimodal tasks (Park et al.”
BackgroundMethodProofWriter: Generating implications, proofs, and abductive statements over natural language
Tafjord, O., Dalvi, B., & Clark, P. (2021) · Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 · doi:10.18653/v1/2021.findings-acl.317 · Cited by 122 · see citing works
“Similarly, work on explanations has sought to gen-Proof erate human-style justifications, which again are typically supporting evidence rather than a fully-formed line of reasoning, and without explicit reasoning rules (Camburu et al., 2018; Jhamtani and Clark, 2020; Inoue et al., 2020).”
BackgroundSmith-Renner, A., Fan, R., Birchfield, M., Wu, T., Boyd-Graber, J., Weld, D. S., & Findlater, L. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376624 · Cited by 112 · see citing works
NILE : Natural Language Inference with Faithful Natural Language Explanations
Kumar, S., & Talukdar, P. P. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.771 · Cited by 114 · see citing works
“Camburu et al. (2018) propose a model for NLI to first generate a free-form natural language explanation and then infer the label from the explanation.”
BackgroundMethodResultInfluentialLearning What Makes a Difference from Counterfactual Examples and Gradient Supervision
Teney, D., Abbasnedjad, E., & van den Hengel, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58607-2_34 · Cited by 106 · see citing works
“In some datasets, such pairs are provided explicitly [7,12,47,48].”
BackgroundMethodInfluentialMeasuring Association Between Labels and Free-Text Rationales
Wiegreffe, S., Marasović, A., & Smith, N. A. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.804 · Cited by 119 · see citing works
“A smaller improvement for E-SNLI could be explained by the fact that E-SNLI has substantially more training examples for each label than CoS-E, which helps a pipeline model learn features predictive of each label.”
BackgroundMethodInfluentialExplaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions
Han, X., Wallace, B., & Tsvetkov, Y. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.492 · Cited by 111 · see citing works
“…of models for certain tasks and properties (Liu et al., 2019b; Hewitt and Liang, 2019), and generative approaches that make the model jointly extract or generate natural language explanations to support predictions (Lei et al., 2016; Camburu et al., 2018; Liu et al., 2019a; Rajani et al., 2019).”
BackgroundInfluentialExplainable Automated Fact-Checking: A Survey
Kotonya, N., & Toni, F. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.474 · Cited by 114 · see citing works
“As a certain level of expertise is required in order to produce competent fact-checking explanations, it would be difficult to use crowd-sourcing techniques, as has been done for acquiring human explanations for other NLP tasks (Camburu et al., 2018).”
BackgroundMethodInfluentialWT5?! Training Text-to-Text Models to Explain their PredictionsPreprint
Narang, S., Raffel, C., Lee, K., Roberts, A., Fiedel, N., & Malkan, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.14546 · Cited by 141 · see citing works
“The primary differences between Camburu et al. (2018) and this work are that our approach requires no special model architecture and that we take advantage of a pre-trained model that is already quite capable of generating natural text.”
BackgroundMethodInfluentialMaieutic Prompting: Logically Consistent Reasoning with Recursive Explanations
Jung, J., Qin, L., Welleck, S., Brahman, F., Bhagavatula, C., Bras, R. J. L., & Choi, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.82 · Cited by 117 · see citing works
Language Models as Agent Models
Andreas, J. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.423 · Cited by 92 · see citing works
Explaining Answers with Entailment Trees
Dalvi, B. B., Jansen, P. A., Tafjord, O., Xie, Z., Smith, H., Pipatanangkura, L., & Clark, P. E. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.585 · Cited by 96 · see citing works
Reframing Human-AI Collaboration for Generating Free-Text Explanations
Wiegreffe, S., Hessel, J., Swayamdipta, S., Riedl, M. O., & Choi, Y. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.47 · Cited by 97 · see citing works
“Crowdsourced free-text explanations are provided in the e-SNLI dataset (Camburu et al., 2018).”
BackgroundMethodInfluentialAn Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction
Paranjape, B., Joshi, M., Thickstun, J., Hajishirzi, H., & Zettlemoyer, L. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.153 · Cited by 94 · see citing works
“We do not consider e-SNLI (Camburu et al., 2018) and CoSE (Rajani et al.”
BackgroundExplaining NLP Models via Minimal Contrastive Editing (MiCE)
Ross, A., Marasović, A., & Peters, M. E. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.336 · Cited by 93 · see citing works
“2 1 Free-text rationales (Narang et al., 2020) can be contrastive if human justifications are collected by asking “why... instead of...” which is not the case with current benchmarks (Camburu et al., 2018; Rajani et al., 2019; Zellers et al., 2019).”
BackgroundAre Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition
Jeretič, P., Warstadt, A., Bhooshan, S., & Williams, A. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.768 · Cited by 83 · see citing works
“NLI itself has been steadily gaining in popularity; many datasets for training and/or testing systems are now available including: FraCaS (Cooper et al., 1994), RTE (Dagan et al., 2006; Mirkin et al., 2009; Dagan et al., 2013), Sentences In-volving Compositional Knowledge (Marelli et al., 2014, SICK), large scale imag…”
BackgroundSurvey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application
Yang, C., Zhu, Y. T., Wang, L., Wang, Y., Chen, Q., Gao, C., Yan, B., & Chen, Y. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3699518 · Cited by 93 · see citing works
On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning
Shaikh, O., Zhang, H., Held, W., Bernstein, M., & Yang, D. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.244 · Cited by 76 · see citing works
On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning
Shaikh, O., Zhang, H., Held, W., Bernstein, M., & Yang, D. (2022) · arXiv · doi:10.48550/arxiv.2212.08061 · Cited by 76 · see citing works
Going Beyond XAI: A Systematic Survey for Explanation-Guided Learning
Gao, Y., Gu, S., Jiang, J., Hong, S.-S., Yu, D., & Zhao, L. (2024) · ACM Computing Surveys · doi:10.1145/3644073 · Cited by 71 · see citing works
e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks
Kayser, M., Camburu, O.-M., Salewski, L., Emde, C., Do, V., Akata, Z., & Lukasiewicz, T. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00128 · Cited by 68 · see citing works
“For this reason, current automatic NLG metrics, such as the BLEU score [36], do not perform well in evaluating NLEs [11].”
BackgroundMethodInfluentialSelf-citationAutomatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data
Shum, K., Diao, S., & Zhang, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.811 · Cited by 65 · see citing works
Rationalization for explainable NLP: a survey
Gurrapu, S., Kulkarni, A., Huang, L., Lourentzou, I., & Batarseh, F. A. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1225093 · Cited by 69 · see citing works
Natural Language Reasoning, A Survey
Yu, F., Zhang, H., Tiwari, P., & Wang, B. (2024) · ACM Computing Surveys · doi:10.1145/3664194 · Cited by 78 · see citing works
“SNLI [12]/e-SNLI† [18] generic 570k realistic human-authored the first large-scale NLI dataset MultiNLI [161] generic 433k realistic human-authored cover more styles and topics XNLI [26] generic 7.”
NLX-GPT: A Model for Natural Language Explanations in Vision and Vision-Language Tasks
Sammani, F., Mukherjee, T., & Deligiannis, N. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00814 · Cited by 59 · see citing works
How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking
Cao, N. D., Schlichtkrull, M., Aziz, W. F., & Titov, I. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.262 · Cited by 56 · see citing works
“Human-provided labels will not help us in demonstrating this (e.g., human rationales (Camburu et al., 2018; DeYoung et al., 2019)), as humans cannot judge if an interpretation is faithful (Jacovi and Goldberg, 2020).”
BackgroundFastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging
Guo, H., Rajani, N. F., Hase, P., Bansal, M., & Xiong, C. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.808 · Cited by 63 · see citing works
“, 2020), natural language generation (Camburu et al., 2018; Rajani et al., 2019; Hase et al., 2020; Wiegreffe et al., 2020), and recently, kNN-based methods (Papernot and McDaniel, 2018; Rajani et al.”
MethodTeach Me to Explain: A Review of Datasets for Explainable NLP.Preprint
Wiegreffe, S., & Marasović, A. (2021) · arXiv (Cornell University) · Cited by 57 · see citing works
Few-Shot Self-Rationalization with Natural Language Prompts
Marasović, A., Beltagy, I., Downey, D., & Peters, M. E. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.31 · Cited by 64 · see citing works
“Note that we use T5 with the ≈ T5 prompt for E-SNLI, and U NI - FIED QA with QA SIMPLE (§3) for other datasets to establish the best possible performance for each dataset.”
BackgroundInfluentialFine-tuned Language Models are Continual Learners
Scialom, T., Chakrabarty, T., & Muresan, S. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.410 · Cited by 55 · see citing works
“…emotional situations using the Empathetic Dialogue data Rashkin et al. (2019); 7) Explanation Generation (Exp) where we train a model on the eSNLI (Camburu et al., 2018) benchmark to generate natural language explanations given a premise, hypothesis and a label (entail, contradict, neutral); 8)…”
MethodPrompting Contrastive Explanations for Commonsense Reasoning Tasks
Paranjape, B., Michael, J., Ghazvininejad, M., Hajishirzi, H., & Zettlemoyer, L. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.366 · Cited by 58 · see citing works
“Recent work use free-form textual explanations to generate explanations for commonsense reasoning tasks like SNLI (Camburu et al., 2018),”
BackgroundMethodInfluentialKim, S., Joo, S., Kim, D., Jang, J., Ye, S., Shin, J., & Seo, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.782 · Cited by 69 · see citing works
Marasović, A., Bhagavatula, C., Park, J. S., Bras, R. J. L., Smith, N. A., & Choi, Y. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.253 · Cited by 52 · see citing works
“Human Evaluation For evaluating our models, we follow Camburu et al. (2018) who show that BLEU (Papineni et al., 2002) is not reliable for evaluation of rationale generation, and hence use human evaluation.9 We believe that other automatic sentence similarity measures are also likely not suitable…”
BackgroundMethodInfluentialPRover: Proof Generation for Interpretable Reasoning over Rules
Saha, S., Ghosh, S., Srivastava, S., & Bansal, M. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.9 · Cited by 50 · see citing works
“PR OVER answers all the questions correctly and also generates all the corresponding proofs accurately in the above scenarios. tions for multiple NLP tasks, including NLI (Cam-buru et al., 2018), commonsense reasoning (Rajani et al., 2019; Zhang et al., 2020) and generic text classification tasks (Liu et al., 2019a), o…”
BackgroundLeveraging explanations in interactive machine learning: An overview
Teso, S., Alkan, Ö., Stammer, W., & Daly, E. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1066049 · Cited by 55 · see citing works
“Recent works have extended this setup to complex natural language inference tasks [31].”
BackgroundInfluentialLocal Interpretations for Explainable Natural Language Processing: A Survey
Luo, S., Ivison, H., Han, S. C., & Poon, J. (2024) · ACM Computing Surveys · doi:10.1145/3649450 · Cited by 60 · see citing works
“[15] are made up of two parts: a predictor module and an explanation module.”
BackgroundMethodResultInfluentialExplanation-Based Human Debugging of NLP Models: A Survey
Lertvittayakumjorn, P., & Toni, F. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00440 · Cited by 51 · see citing works
“, natural language feedback (Camburu et al., 2018), ratings of the provided explanations, new training examples (Fiebrink et al.”
BackgroundSCOTT: Self-Consistent Chain-of-Thought Distillation
Wang, P., Wang, Z., Li, Z., Gao, Y., Yin, B., & Ren, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.304 · Cited by 68 · see citing works
Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers
Chen, H., & Ji, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.347 · Cited by 52 · see citing works
“Some work focuses on improving model’s inter-pretability by aligning explanations with human-judgements (Camburu et al., 2018; Du et al., 2019b; Chen and Ji, 2019; Erion et al., 2019; Plumb et al., 2019).”
BackgroundRecent Advances of Foundation Language Models-based Continual Learning: A Survey
Yang, Y., Zhou, J., Ding, X., Huai, T., Liu, S., Chen, Q., Xie, Y., & He, L. (2024) · ACM Computing Surveys · doi:10.1145/3705725 · Cited by 57 · see citing works
“By leverages a VLM to enhance the”
Complementary Explanations for Effective In-Context Learning
Ye, X., Iyer, S. V., Çelikyılmaz, A., Stoyanov, V., Durrett, G., & Pasunuru, R. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.273 · Cited by 46 · see citing works
QED: A Framework and Dataset for Explanations in Question Answering
Lamm, M., Palomaki, J., Alberti, C., Andor, D., Choi, E., Soares, L. B., & Collins, M. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00398 · Cited by 47 · see citing works
“…the one hand, such as attention distributions (Wiegreffe and Pinter, 2019; Jain and Wallace, 2019; Mohankumar et al., 2020) or sequential outputs (Camburu et al., 2018, 2019; Narang et al., 2020; Kumar and Talukdar, 2020) and more elaborate, discrete semantic representations that can in theory…”
BackgroundExplaining Question Answering Models through Text GenerationPreprint
Latcinnik, V., & Berant, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.05569 · Cited by 49 · see citing works
“This has sparked interest in explainable models , where intermediate parts of the model can be inspected and interpreted by humans (Lipton, 2016; Ribeiro et al., 2016; Lundberg and Lee, 2017; Camburu et al., 2018; Thorne et al., 2019; Rajani et al., 2019; Jain and Wallace, 2019).”
BackgroundProcess Knowledge-Infused AI: Toward User-Level Explainability, Interpretability, and Safety
Sheth, A., Gaur, M., Roy, K., Venkataramanan, R., & Khandelwal, V. (2022) · IEEE Internet Computing · doi:10.1109/mic.2022.3182349 · Cited by 43 · see citing works
“…would require human evaluation, which is a mandate; these metrics are also equally important as they either involve: (a) annotators’ agreements/disagreements, (b) knowledge source, and (c) train deep language models on datasets that have data samples ordered by some relationships [12], [13].”
BackgroundBridging the Transparency Gap: What Can Explainable AI Learn from the AI Act?
Gyevnár, B., Ferguson, N., & Schäfer, B. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230367 · Cited by 43 · see citing works
“, natural language inference [7]) by the definition of an AI system in the Act which raises further legal questions, discussed further in Section 4.”
BackgroundExplainable Artificial Intelligence for Resilient Security Applications in the Internet of Things
Masud, M. T., Keshk, M., Moustafa, N., Linkov, I., & Emge, D. K. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3413790 · Cited by 43 · see citing works
MARTA: Leveraging Human Rationales for Explainable Text Classification
Arous, I., Dolamic, L., Yang, J., Bhardwaj, A., Cuccu, G., & Cudré-Mauroux, P. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i7.16734 · Cited by 43 · see citing works
Symbolic Chain-of-Thought Distillation: Small Models Can Also “Think” Step-by-Step
Li, L. H., Hessel, J., Yu, Y., Ren, X., Chang, K.-W., & Choi, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.150 · Cited by 69 · see citing works
Supervising Model Attention with Human Explanations for Robust Natural Language Inference
Stacey, J., Belinkov, Y., & Rei, M. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21386 · Cited by 42 · see citing works
“As previous work using e-InferSent (Camburu et al. 2018), TextCat (Hase and Bansal 2021) and NILE (Kumar and Talukdar 2020) found no significant improvements using the explanations, we do not recreate these baselines.”
BackgroundMethodInfluentialGenerating Token-Level Explanations for Natural Language Inference
Thorne, J. P., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1101 · Cited by 41 · see citing works
R4C: A Benchmark for Evaluating RC Systems to Get the Right Answer for the Right Reason
Inoue, N., Stenetorp, P., & Inui, K. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.602 · Cited by 43 · see citing works
“Another direction, which we follow, includes evaluating a systems’ reasoning (Jansen, 2018; Yang et al., 2018; Thorne and Vlachos, 2018; Camburu et al., 2018; Fan et al., 2019; Rajani et al., 2019).”
BackgroundInfluentialSELFEXPLAIN: A Self-Explaining Architecture for Neural Text Classifiers
Rajagopal, D., Balachandran, V., Hovy, E. H., & Tsvetkov, Y. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.64 · Cited by 44 · see citing works
“, 2019), e-SNLI (Camburu et al., 2018) and recently, Eraser benchmark (DeYoung et al.”
Jhamtani, H., & Clark, P. E. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.10 · Cited by 43 · see citing works
“Finally there are a few human-authored explanation datasets. e-SNLI (Camburu et al., 2018) adds crowdsourced explanations to SNLI entailment problems (Bowman et al., 2015), and CoS-E (Rajani et al., 2019) adds explanations for Com-monsenseQA questions (Talmor et al., 2019).”
BackgroundKnowledge-Guided Sentiment Analysis Via Learning From Natural Language Explanations
Ke, Z., Sheng, J., Li, Z., Silamu, W., & Guo, Q. (2021) · IEEE Access · doi:10.1109/access.2020.3048088 · Cited by 40 · see citing works
“Reference [4] extended the Stanford Natural Language Inference dataset with an extra layer of nat-ural language explanations based on human-annotated implication relations.”
BackgroundHase, P., & Bansal, M. (2022) · LNLS · doi:10.18653/v1/2022.lnls-1.4 · Cited by 40 · see citing works
“Meanwhile, natural language explanations have been treated as targets in a multitask, sequence-to-sequence framework, using datasets with free-form textual annotations for each data point (Camburu et al., 2018; Narang et al., 2020; Hase et al., 2020; Wiegreffe et al., 2020).”
BackgroundInfluentiale-CARE: a New Dataset for Exploring Explainable Causal Reasoning
Du, L., Ding, X. H., Xiong, K., Liu, T., & Qin, B. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.33 · Cited by 47 · see citing works
Evaluating and Characterizing Human Rationales
Carton, S., Rathore, A., & Tan, C. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.747 · Cited by 39 · see citing works
“• E-SNLI (Camburu et al., 2018) A textual entailment dataset of 568,939 short snippets and claims for which each snippet either refutes, supports, or is neutral toward.”
BackgroundYang, L., Zhang, S., Qin, L., Li, Y., Wang, Y., Liu, H., Wang, J., Xie, X., & Zhang, Y. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.806 · Cited by 43 · see citing works
ExpBERT: Representation Engineering with Natural Language Explanations
Murty, S., Koh, P. W., & Liang, P. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.190 · Cited by 35 · see citing works
“Rather than using language to define a global collection of features, Rajani et al. (2019) and Camburu et al. (2018) use instance-level explanations to train models that generate their own explanations.”
BackgroundFilighera, A., Parihar, S., Steuer, T., Meuser, T., & Ochs, S. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.587 · Cited by 33 · see citing works
“Camburu et al. (2018) extended the Stanford Natural Language Inference dataset (SNLI) (Bow-man et al., 2015) using Amazon Mechanical Turk.”
MethodCan I Trust the Explainer? Verifying Post-hoc Explanatory MethodsPreprint
Camburu, O.-M., Giunchiglia, E., Foerster, J., Lukasiewicz, T., & Blunsom, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.02065 · Cited by 33 · see citing works
“Other types of explainers are (iii) example-based [12]: identify the most relevant instances in the training set that influenced the model’s prediction on the current input, and (iv) natural language explainers [4, 18, 11]: design models capable of generating full-sentence justifications for their own predictions.”
BackgroundSelf-citationExplainability of vision-based autonomous driving systems: Review and challenges.Preprint
Zablocki, É., Ben-younes, H., Pérez, P., & Cord, M. (2021) · arXiv (Cornell University) · Cited by 32 · see citing works
Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport
Swanson, K., Yu, L., & Lei, T. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.496 · Cited by 31 · see citing works
“e-SNLI (Camburu et al., 2018) is an extended version of the SNLI dataset (Bowman et al.”
BackgroundMethodInfluentialExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning
Saha, S., Yadav, P., Bauer, L., & Bansal, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.609 · Cited by 39 · see citing works
“…extractive rationales (Zaidan et al., 2007; Lei et al., 2016; Yu et al., 2019; DeYoung et al., 2020), (2) free-form or natural language explanations (Camburu et al., 2018; Rajani et al., 2019; Narang et al., 2020; Brahman et al., 2021; Zhang et al., 2020), and (3) structured explanations…”
BackgroundInfluentialExplainable Machine Learning with Prior Knowledge: An Overview
Beckh, K., Müller, S., Jakobs, M., Toborek, V., Tan, H., Fischer, R., Welke, P., Houben, S., & von Rueden, L. (2021) · Lancaster EPrints (Lancaster University) · doi:10.48550/arxiv.2105.10172 · Cited by 40 · see citing works
“Human Feedback [6] [65] [77] [13] [31] [60] [58] [66] [13] [37] [31] [5] [67] [72] [29] [82] [32] [70] [64]”
Shaping Visual Representations with Language for Few-Shot Classification
Mu, J., Liang, P., & Goodman, N. D. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.436 · Cited by 30 · see citing works
“…studies models which generate natural language explanations of decisions for interpretability for both textual (e.g. natural language inference; [3]) and visual [17, 18] tasks, but here we examine whether this act of predicting language can actually improve downstream task performance; similar…”
A Survey on Explainability in Machine Reading ComprehensionPreprint
Thayaparan, M., Valentino, M., & Freitas, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.00389 · Cited by 37 · see citing works
“Language generation models: Recent developments in language modeling along with the creation of explanation-supporting benchmarks, such as e-SNLI (Camburu et al., 2018) and Cos-E (Rajani et al., 2019), have opened up the possibility to automatically generate semantically plausible and coherent…”
BackgroundInfluentialA Survey on Efficient Vision‐Language Models
Shinde, G., Ravi, A., Dey, E., Sakib, S., Rampure, M., & Roy, N. (2025) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70036 · Cited by 33 · see citing works
“Explainable AI techniques such as visual attention maps Selvaraju et al. (2017), natural language rationales Camburu et al. (2018), and counterfactual reasoning Goyal et al. (2019b) Confidence Calibration and Uncertainty Estimation: Human-in-the-loop systems depend on the model’s ability to…”
A Diagnostic Study of Explainability Techniques for Text Classification
Atanasova, P. (2020) · Springer eBooks · doi:10.1007/978-3-031-51518-7_7 · Cited by 31 · see citing works
Is Sparse Attention more Interpretable?
Meister, C., Lazov, S., Augenstein, I., & Cotterell, R. (2021) · doi:10.18653/v1/2021.acl-short.17 · Cited by 28 · see citing works
“Such tasks can be framed as generative, where models generate free text explanations (Camburu et al., 2018; Kotonya and Toni, 2020; Atanasova et al., 2020b), or as post-hoc interpretability methods, where salient portions of the input are highlighted (Lipton, 2018; DeYoung et al., 2020; Atanasova…”
BackgroundDistilling ChatGPT for Explainable Automated Student Answer Assessment
Li, J., Gui, L., Zhou, Y., West, D., Aloisi, C., & He, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.399 · Cited by 35 · see citing works
Interpreting Deep Learning Models in Natural Language Processing: A ReviewPreprint
Sun, X., Yang, D., Li, X., Zhang, T., Meng, Y., Qiu, H., Wang, G., Hovy, E. H., & Li, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.10470 · Cited by 40 · see citing works
“Figure 17 shows the model architectures proposed in [31] and [149].”
BackgroundMethodThe Explanation Game: Towards Prediction Explainability through Sparse Communication
Treviso, M. V., & Martins, A. F. T. (2020) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2020.blackboxnlp-1.10 · Cited by 29 · see citing works
“To this end, we used the e-SNLI corpus (Camburu et al., 2018), which extends the SNLI with human rationales.”
MethodExCAR: Event Graph Knowledge Enhanced Explainable Causal Reasoning
Du, L., Ding, X., Xiong, K., Liu, T., & Qin, B. (2021) · doi:10.18653/v1/2021.acl-long.183 · Cited by 26 · see citing works
Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge
Chen, J., Shi, W., Fu, Z., Cheng, S., Li, L., & Xiao, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.550 · Cited by 30 · see citing works
Faithfulness Tests for Natural Language Explanations
Atanasova, P., Camburu, O.-M., Lioma, C., Lukasiewicz, T., Simonsen, J. G., & Augenstein, I. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.25 · Cited by 31 · see citing works
Self-citationEvaluating the effectiveness of XAI techniques for encoder-based language models
Mersha, M. A., Yigezu, M. G., & Kalita, J. K. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113042 · Cited by 23 · see citing works
e-SNLI-VE: Corrected Visual-Textual Entailment with Natural Language ExplanationsPreprint
Do, V., Camburu, O.-M., Akata, Z., & Lukasiewicz, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.03744 · Cited by 23 · see citing works
“These explanations were collected in e-SNLI [3] to support textual entailment for SNLI.”
BackgroundMethodInfluentialSelf-citation
- Authors
- Camburu, O.-M., Rocktäschel, T., Lukasiewicz, T., & Blunsom, P.
- Affiliations
- University of Oxford (GB)
- University College London (GB)
- Google (United States) (US)
- ORCID
- Lukasiewicz, Thomas
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- Volume
- 31
- Pages
- 9539-9549
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1812.01193
- OpenAlex
- W2891012317
- Semantic Scholar
- c242438dac5aa4d9b13766c14240bb8426690d58
- Cited by
- 367
- References
- 27
- Citations by year
- 2025: 19 · 2024: 33 · 2023: 94 · 2022: 73 · 2021: 84 · 2020: 55 · 2019: 9
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- arXiv (Cornell University)
- neural information processing systems
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- natural language inference, natural language explanations, entailment relations, explainable AI, out-of-domain transfer, interpretability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
