The DOI matches its DataCite record; 23 of the 27 references it cites were found in a registry or an index.
e-SNLI: Natural Language Inference with Natural Language Explanations
Open accessAbstract
In order for machine learning to garner widespread public adoption, models must be able to provide interpretable and robust explanations for their decisions, as well as learn from human-provided explanations at train time. In this work, we extend the Stanford Natural Language Inference dataset with an additional layer of human-annotated natural language explanations of the entailment relations. We further implement models that incorporate these explanations into their training process and output them at test time. We show how our corpus of explanations, which we call e-SNLI, can be used for various goals, such as obtaining full sentence justifications of a model's decisions, improving universal sentence representations and transferring to out-of-domain NLI datasets. Our dataset thus opens up a range of research directions for using natural language explanations, both for improving models and for asserting their trust.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Rationale supervision work, such as e-SNLI (Camburu et al., 2018) and ERASER (DeYoung et al., 2020), provides gold explanations for post hoc evaluation or training but does not embed quality criteria into the generation process itself.”
From GIER: Gap-Driven Self-Refinement for Large Language Models (2025) · cites this paper as Camburu et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“researchers increasingly draw on an LLMs' own text generation capabilities to produce natural language explanations (NLEs) alongside predictions [22,23]”
From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [22]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Although LLMs can be prompted to self-explain [2], these explanations are unreliable [8].”
From ProSLM: A Prolog Synergized Language Model for explainable Domain Specific Knowledge Based Question Answering (2024) · cites this paper as [2]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The PredictAndExplain method, introduced in [2], illustrates that models can be trained to generate explanations of their outputs without sacrificing performance, but this approach requires access to a reliable explanation-augmented dataset for training.”
From ProSLM: A Prolog Synergized Language Model for explainable Domain Specific Knowledge Based Question Answering (2024) · cites this paper as [2]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“Therefore, our encoders are 2048-bidirectional-LSTMs [15] with max-pooling, resulting in a sentence representation dimension of 4096.”
MethodImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1×
“In computer vision, pretrained ImageNet-based encoders [10] provide standard image feature extractors for other downstream tasks.”
BackgroundNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 2×
“Attention-based models, such as [2, 25], offer some degree of interpretability and have been shown to also improve performance on downstream tasks.”
BackgroundShow, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377 · cited 2×
“To address this deficiency, we have collected a large corpus of human-annotated explanations for the Stanford Natural Language Inference (SNLI) dataset [3].”
BackgroundMethodA Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342 · cited 1×
“We test how our explanations affect the direct transfer in both label prediction and explanation generation by looking at SICK-E [20] and MultiNLI [26].”
BackgroundSupervised Learning of Universal Sentence Representations from Natural Language Inference Data
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1070 · Cited by 2,049 · cited 8×
“Throughout our experiments, our models follow the architecture presented in Conneau et al. [7], as we build directly on top of their code 2 .”
BackgroundMethodInfluentialA Decomposable Attention Model for Natural Language Inference
Parikh, A. P., Täckström, O., Das, D., & Uszkoreit, J. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1244 · Cited by 1,351 · cited 3×
“Glockner et al. show that models achieving high accuracies on SNLI, such as [21, 22, 6], show dramatically reduced performance on this simpler dataset, while the model of Chen et al. [5] is more robust due to incorporating exter-nal knowledge.”
BackgroundResultKiros, R., Zhu, Y., Salakhutdinov, R., Zemel, R. S., Torralba, A., Urtasun, R., & Fidler, S. (2015) · arXiv · doi:10.48550/arxiv.1506.06726 · Cited by 1,293 · cited 2×
“Traditional approaches make use of very large unsupervised datasets [17, 14], taking weeks to train.”
BackgroundAnnotation Artifacts in Natural Language Inference Data
Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S., & Smith, N. A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2017 · Cited by 907 · cited 3×
“Recently, Gururangan et al. [13] cast doubt on whether models trained on SNLI are learning to understand language, or are largely fixating on spurious correlations, also called artifacts.”
BackgroundInfluential
Research works printed but not linked: 6
- [4][4] Chan, W., Jaitly, N., Le, Q. V., and Vinyals, O. (2015). Listen, attend and spell. CoRR , abs/1508.01211. - · arXiv
- [5][5] Chen, Q., Zhu, X., Ling, Z., Inkpen, D., and Wei, S. (2017). Natural language inference with external knowledge. CoRR , abs/1711.04289. - · arXiv
- [6][6] Chen, Q., Zhu, X., Ling, Z., Wei, S., and Jiang, H. (2016). Enhancing and combining sequential and tree LSTM for natural language inference. CoRR , abs/1609.06038. - · arXiv
- [10][10] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). ImageNet: A largescale hierarchical image database. In Proc. of CVPR . -
- [19][19] Liu, P., Qiu, X., and Huang, X. (2016). Modelling interaction of sentence pair with coupledlstms. CoRR , abs/1605.05573. - · arXiv
- [24][24] Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). "why should I trust you?": Explaining the predictions of any classifier. CoRR , abs/1602.04938. - · arXiv
Cited by
Sorted by citations
Items 201–300 of 369
Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations in a Label-Abundant Setup
Yordanov, Y., Kocijan, V., Lukasiewicz, T., & Camburu, O.-M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.255 · Cited by 12
“We instantiate our few-shot learning approaches on e-SNLI (Camburu et al., 2018) as parent task, and WinoGrande (Sakaguchi et al., 2020) and ComVE (Wang et al., 2020) as child tasks.”
BackgroundMethodResultInfluentialSelf-citationCLUES: A Benchmark for Learning Classifiers using Natural Language Explanations
Menon, R. R., Ghosh, S., & Srivastava, S. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.451 · Cited by 11
CH-Wang, S., Saakyan, A., Li, O., Zhou, Y., & Muresan, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.215 · Cited by 15
How Fast can BERT Learn Simple Natural Language Inference?
Lin, Y., & Su, K. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.51 · Cited by 7
“Entailment judgment ( Dagan et al., 2006; Marelli et al., 2014a) is a common test for natural language inference (NLI) (Camburu et al., 2018; Conneau et al., 2018) as it possesses the simplest form in related tasks such as question and answering (Bowman and Zhu, 2019).”
BackgroundD-REX: Dialogue Relation Extraction with Explanations
Albalak, A., Embar, V. R., Tuan, Y.-L., Getoor, L. C., & Wang, W. Y. (2022) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2022.nlp4convai-1.4 · Cited by 10
“One approach to encourage a model to learn the context surrounding a relation is by requiring the model to generate an explanation along with the relation (Camburu et al., 2018).”
BackgroundMethodLogical Satisfiability of Counterfactuals for Faithful Explanations in NLI
Sia, S., Belyy, A., Almahairi, A., Khabsa, M., Zettlemoyer, L., & Mathias, L. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i8.26174 · Cited by 11
Elaboration-Generating Commonsense Question Answering at Scale
Wang, W., Srikumar, V., Hajishirzi, H., & Smith, N. A. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.90 · Cited by 10
Explanation Selection Using Unlabeled Data for Chain-of-Thought Prompting
Ye, X., & Durrett, G. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.41 · Cited by 11
“• E -SNLI (Camburu et al., 2018) studies the task of natural language inference which is to classify the relation between a premise and a hypothesis.”
BackgroundYao, B., Jindal, I., Popa, L. C., Katsis, Y., Ghosh, S., He, L., Lu, Y., Srivastava, S., Li, Y., Hendler, J., et al. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.778 · Cited by 10
“As researchers looked into the quality (Car-ton et al., 2020; Yao et al., 2023) of human-annotated natural language explanations (Camburu et al., 2018; Rajani et al., 2019; Aggarwal et al., 2021), they discovered numerous issues in existing datasets (Geva et al., 2019; Chmielewski and Kucker, 2020;…”
BackgroundMethodAghahadi, Z., & Talebpour, A. (2022) · Journal of Applied Non-Classical Logics · doi:10.1080/11663081.2022.2041352 · Cited by 9
“To generate a natural language explanations for the entailment relations (NLEs) (Camburu et al., 2018) on the SNLI dataset, the output is limited to specific templates (e.”
BackgroundMethodInfluentialHo, T. K., Luo, Y.-F., & Guido, R. C. (2022) · IEEE Signal Processing Magazine · doi:10.1109/msp.2022.3155906 · Cited by 8
Evaluating Explanations for Reading Comprehension with Realistic Counterfactuals.Preprint
Ye, X., Nair, R., & Durrett, G. (2021) · arXiv (Cornell University) · Cited by 6
Prompting with Pseudo-Code Instructions
Mishra, M., Kumar, P., Bhat, R. A., Murthy, R., Contractor, D., & Tamilselvam, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.939 · Cited by 18
From outputs to insights: a survey of rationalization approaches for explainable text classification
Guzman, E. M., Schlegel, V., & Batista-Navarro, R. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1363531 · Cited by 8
Improving Interpretability via Explicit Word Interaction Graph Layer
Sekhon, A., Chen, H., Shrivastava, A., Wang, Z., Ji, Y., & Qi, Y. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26586 · Cited by 7
Language Modeling with Latent Situations
Li, B. Z., Nye, M., & Andreas, J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.795 · Cited by 12
Breakpoint Transformers for Modeling and Tracking Intermediate Beliefs
Richardson, K., Tamari, R., Sultan, O., Shahaf, D., Tsarfaty, R., & Sabharwal, A. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.658 · Cited by 7
Learning Robust Rationales for Model Explainability: A Guidance-Based Approach
Hu, S., & Yu, K. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29783 · Cited by 6
“…and Jaakkola 2016; Chen et al. 2018), which select or generate a subset of features that can justify the output; and natu-ral language explanations (Camburu et al. 2018; Kumar and Talukdar 2020; Rajani et al. 2019), which produce textual pieces that can explain the result in a human-readable way.”
BackgroundKunz, J. (2024) · Linköping studies in science and technology. Dissertations · doi:10.3384/9789180754712 · Cited by 7
Zhou, Y., & Tan, C. (2021) · Workshop on Insights from Negative Results in NLP (Insights) · doi:10.18653/v1/2021.insights-1.17 · Cited by 8
“We use BLEU to evaluate the quality of generated explanations with regard to groundtruth explanations because it is a commonly used metric to evaluate natural language explanations (Camburu et al., 2018; Rajani et al., 2019).”
InfluentialA Study of the Plausibility of Attention between RNN Encoders in Natural Language Inference
Nguyen, D. H., Gravier, G., & Sébillot, P. (2021) · International Conference on Machine Learning and Applications · doi:10.1109/icmla52953.2021.00259 · Cited by 7
Joshi, B., Liu, Z., Ramnath, S., Chan, A., Tong, Z., Nie, S., Wang, Q., Choi, Y., & Ren, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.392 · Cited by 20
AlKhamissi, B., Verma, S. S., Yu, P., Jin, Z., Çelikyılmaz, A., & Diab, M. (2023) · doi:10.18653/v1/2023.nlrse-1.10 · Cited by 8
“ESNLI The Stanford Natural Language Inference dataset with Explanations Camburu et al. (2018) to train models to provide interpretable and robust explanations for their decisions.”
MethodInfluentialER-Test: Evaluating Explanation Regularization Methods for Language Models
Joshi, B., Chan, A., Liu, Z., Nie, S., Sanjabi, M., Firooz, H., & Ren, X. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.242 · Cited by 12
“Second, for NLI, we use eSNLI (Camburu et al., 2018; DeYoung et al., 2019) and MNLI (Williams et al.”
BackgroundMethodInfluentialAre Multilingual Sentiment Models Equally Right for the Right Reasons?
Jørgensen, R. L., Caccavale, F., Igel, C., & Søgaard, A. (2022) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2022.blackboxnlp-1.11 · Cited by 6
S3C: Semi-Supervised VQA Natural Language Explanation via Self-Critical Learning
Suo, W., Sun, M., Liu, W., Gao, Y., Wang, P., Zhang, Y., & Wu, Q. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00260 · Cited by 8
Cross-Modal Conceptualization in Bottleneck Models
Alukaev, D., Kiselev, S., Pershin, I., Ibragimov, B., Ivanov, V. V., Kornaev, A. V., & Titov, I. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.318 · Cited by 11
“…ML has become a major field, with various methods developed over the year to offer different types of explanations, such as input attribution (Sundararajan et al., 2017; Si-monyan et al., 2014), free-text rationales (Camburu et al., 2018) or training data attribution (Koh and Liang, 2017).”
BackgroundHow rationals boost textual entailment modeling: Insights from large language models
Pham, D.-H., Le, T., & Nguyen, H. T. (2024) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2024.109517 · Cited by 7
Generating Hypothetical Events for Abductive Inference
Paul, D., & Frank, A. (2021) · Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/2021.starsem-1.6 · Cited by 6
“Recently, there has been substantial effort to build datasets with natural language explanations (Camburu et al., 2018; Park et al., 2018; Thayaparan et al., 2020).”
BackgroundHe, J., Peng, B., Liao, Y., Liu, Q., & Xiong, D. (2021) · doi:10.18653/v1/2021.acl-long.469 · Cited by 7
Cross-Domain Transfer of Generative Explanations Using Text-to-Text Models
Erliksson, K. F., Arpteg, A., Matskin, M., & Payberah, A. H. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-80599-9_8 · Cited by 7
“Generative natural language explanations were studied in [4], who proposed an extended version of the SNLI dataset [2] with annotated abstractive explanations, and considered different seq2seq models for learning to generate such explanations.”
BackgroundMethodInfluentialSystematic Human Learning and Generalization From a Brief Tutorial With Explanatory Feedback
Nam, A., & McClelland, J. L. (2024) · Open Mind · doi:10.1162/opmi_a_00123 · Cited by 12
“Previous efforts to train models to use explanations (Mishra et al., 2022), whether from scratch (Camburu et al., 2018, Lampinen, Roy, et al., 2022), through fine-tuning (Lampinen, Dasgupta, et al., 2022), or through conditioning with in-context prompts at evaluation time (Lu et al., 2022; Wei et…”
BackgroundGenerating Token-Level Explanations for Natural Language InferencePreprint
Thorne, J. P., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2019) · arXiv (Cornell University) · doi:10.17863/cam.53800 · Cited by 14
Vakharia, P., Kufeldt, A., Meyers, M., Lane, I., & Gilpin, L. H. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-71170-1_23 · Cited by 5
“The PredictAndExplain method, introduced in [2], illustrates that models can be trained to generate explanations of their outputs without sacrificing performance, but this approach requires access to a reliable explanation-augmented dataset for training.”
BackgroundMethodPriyadarshana, Y. H. P. P., Liang, Z., & Piumarta, I. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1564828 · Cited by 5
Zero-shot Sequence Labeling for Transformer-based Sentence Classifiers
Bujel, K., Yannakoudakis, H., & Rei, M. (2021) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2021.repl4nlp-1.20 · Cited by 5
“The token-level predictions were evaluated against human explanations of the entailment relation using the e-SNLI dataset (Camburu et al., 2018).”
MethodHuman-Centered Evaluation of Explanations
Boyd-Graber, J. L., Carton, S., Shi, F., Liao, Q. V., Lombrozo, T., Smith, A., & Tan, C. (2022) · doi:10.18653/v1/2022.naacl-tutorials.4 · Cited by 5
“These come mostly in the form of rationales indicating which tokens within a text are important or causal for the true label, e.g., (Zaidan et al., 2007; Khashabi et al., 2018; Thorne et al., 2018), but sometimes consist of natural language e.g., (Camburu et al., 2018).”
BackgroundZhang, H., Korikov, A., Farinneya, P., Pour, M. M. A., Bharadwaj, M., Pesaranghader, A., Huang, X. Y., Lok, Y. X., Wang, Z., Jones, N., et al. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591880 · Cited by 6
“Also included is the explanation-augmented Stanford Natural Language Inference (e-SNLI) corpus [4], where rationales are labeled for the task of inferring one of three principal NL inference (NLI) relations: entailment, contradiction, or neutral.”
BackgroundChen, Q., Xu, G., Yan, M., Zhang, J., Huang, F., Si, L., & Zhang, Y. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.835 · Cited by 8
ALERT: Adapt Language Models to Reasoning Tasks
Yu, P., Wang, T., Golovneva, O., AlKhamissi, B., Verma, S. S., Jin, Z., Ghosh, G., Diab, M., & Çelikyılmaz, A. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.60 · Cited by 25
Explanation-based Finetuning Makes Models More Robust to Spurious Cues
Ludan, J. M., Meng, Y., Nguyen, T. D., Shah, S. A., Lyu, Q., Apidianaki, M., & Callison-Burch, C. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.242 · Cited by 10
Uni-NLX: Unifying Textual Explanations for Vision and Vision-Language Tasks
Sammani, F., & Deligiannis, N. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00498 · Cited by 5
“Moving away from general and high-level explanations such as heatmaps [30, 33, 3, 31], Natural Language Explanations (NLE)2 [6, 19] offer a detailed, human-friendly textual format explanation.”
BackgroundTo BERT or Not to BERT Dealing with Possible BERT Failures in an Entailment Task
Fialho, P., Coheur, L., & Quaresma, P. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-50146-4_54 · Cited by 5
“SNLI inspired the creation of other corpora on NLI, for instance the e-SNLI corpus [6] that augments SNLI with natural language explanations for the annotations, or the MultiNLI corpus [21], that follows the same design procedure and size of SNLI, but instead of captions includes sentence pairs…”
MethodEfficient Explanations from Empirical Explainers
Schwarzenberg, R., Feldhus, N., & Möller, S. (2021) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2021.blackboxnlp-1.17 · Cited by 13
Detecting High-Quality Comments in Written Feedback with a Zero Shot Classifier
Hart-Davidson, W., Omizo, R. M., & Meeks, M. (2021) · ACM International Conference on Design of Communication (SIGDOC) · doi:10.1145/3472714.3473659 · Cited by 6
E-BART: Jointly Predicting and Explaining Truthfulness.
Brand, E., Roitero, K., Soprano, M., & Demartini, G. (2021) · Institutional Research Information System (University of Udine) · Cited by 5
Explainable Natural Language Inference via Identifying Important Rationales
Yang, Z., Dong, S., & Hu, J. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3178391 · Cited by 5
“However,extractingrationalesfrom bothpremisesandhypotheseswouldbemorerealistic[24].”
BackgroundMethodInfluentialZero-shot Code-Mixed Offensive Span Identification through Rationale Extraction
Ravikiran, M., & Chakravarthi, B. R. (2022) · Workshop on Speech, Vision, and Language Technologies for Dravidian Languages (DravidianLangTech) · doi:10.18653/v1/2022.dravidianlangtech-1.37 · Cited by 5
Yao, B., Sen, P., Popa, L. C., Hendler, J., & Wang, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.821 · Cited by 7
Logical Reasoning with Span-Level Predictions for Interpretable and Robust NLI Models
Stacey, J., Minervini, P., Dubossarsky, H., & Rei, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.251 · Cited by 7
“Training models with the e-SNLI explanations can improve both model performance and robustness (Stacey et al., 2022; Zhao and Vydis-waran, 2021), although not all prior work has found these improvements (Kumar and Talukdar, 2020; Camburu et al., 2018; Carton et al., 2022).”
BackgroundMethodInfluentialSBU Figures It Out: Models Explain Figurative Language
Lal, Y. K., & Bastan, M. (2022) · Workshop on Figurative Language Processing (FigLang) · doi:10.18653/v1/2022.flp-1.20 · Cited by 5
“Both e-SNLI (Camburu et al., 2018) and SuMe (Bastan et al., 2022) are tasks where models have to predict labels as well as explain it.”
BackgroundMethodInfluentialZARA: Improving Few-Shot Self-Rationalization for Small Language Models
Chen, W., Yen, A.-Z., Wu, C.-K., Huang, H., & Chen, H. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.310 · Cited by 9
Explainable Natural Language Reasoning via Conceptual Unification.Preprint
Valentino, M., Thayaparan, M., & Freitas, A. (2020) · arXiv (Cornell University) · Cited by 5
Towards Explainable Fact CheckingPreprint
Augenstein, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.10274 · Cited by 15
Weakly Supervised Explainable Phrasal Reasoning with Neural Fuzzy LogicPreprint
Wu, Z., Zhang, Z. X., Naik, A., Mei, Z., Firdaus, M., & Mou, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.08927 · Cited by 8
“We evaluated our textual explanation approach on the e-SNLI dataset (Camburu et al. 2018), which extends the SNLI dataset with one reference explanation for each training sample, and three reference explanations for each validation or test samples.”
InfluentialDo Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations
Chen, Y., Zhong, R., Ri, N., Zhao, C., He, H., Steinhardt, J., Yu, Z., & McKeown, K. (2023) · arXiv · doi:10.48550/arxiv.2307.08678 · Cited by 5
“In Post-Hoc, given an input x, the model first predicts the output ox, and then generates an explanation ex conditioned on x and ox (Camburu et al., 2018; Park et al., 2018).”
MethodSiegel, N. Y., Camburu, O.-M., Heess, N., & Perez-Ortiz, M. (2024) · arXiv · doi:10.48550/arxiv.2404.03189 · Cited by 5
“Following Atanasova et al. (2023), we evaluate on three popular classification datasets including human-written NLEs: e-SNLI (Camburu et al., 2018): Sentence pairs labeled with entailment, contradiction, or neutral.”
MethodSelf-citationCan Language Models Laugh at YouTube Short-form Videos?
Ko, D., Lee, S., & Kim, G. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.176 · Cited by 10
Explanation Regeneration via Information Bottleneck
Li, Q., Wu, Z., Kong, L., & Bi, W. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.765 · Cited by 12
Towards an Exhaustive Evaluation of Vision-Language Foundation Models
Salin, E., Ayache, S., & Favre, B. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00041 · Cited by 4
“Reasoning We identify a few reasoning tasks necessary for vision-language models, using as inspiration existing monomodal tasks [60, 56, 10, 105, 13].”
MethodRather a Nurse than a Physician - Contrastive Explanations under Investigation
Eberle, O., Chalkidis, I., Piqueras, L. C., & Brandl, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.427 · Cited by 4
Knowledge Graph Compression Enhances Diverse Commonsense Generation
Hwang, E., Thost, V., Shwartz, V., & Ma, T. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.37 · Cited by 5
“In a related line of work, researchers collected or generated commonsense explanations for existing tasks (e.g., Camburu et al., 2018; Rajani et al., 2019; Brahman et al., 2021).”
BackgroundUnderstanding hate speech: the HateInsights dataset and model interpretability
Arshad, M. U., & Shahzad, W. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2372 · Cited by 5
“Rationales of this nature have found application in various domains, including common sense explanations ( Fatema Rajani et al., 2019 ), e-SNLI ( Camburu et al., 2018 ), and numerous other tasks ( DeYoung et al., 2020 ).”
A Generative Approach for Mitigating Structural Biases in Natural Language Inference
Asael, D., Ziegler, Z. M., & Belinkov, Y. (2022) · Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/2022.starsem-1.16 · Cited by 6
Perturbation-Based Self-Supervised Attention for Attention Bias in Text Classification
Feng, H., Lin, Z., & Ma, Q. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3302230 · Cited by 7
“In general, the annotators need to explicitly highlight the important words or rationales [10], [14], [15] for the”
BackgroundDoes Self-Rationalization Improve Robustness to Spurious Correlations?
Ross, A., Peters, M. E., & Marasović, A. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.501 · Cited by 9
Explaining Interactions Between Text Spans
Choudhury, S., Atanasova, P., & Augenstein, I. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.783 · Cited by 4
“Other explanation types, such as free-text explanations (Camburu et al., 2018; Wang et al., 2020; Rajani et al., 2019) can offer complementary information, but their faithfulness could be hard to estimate (Atanasova et al., 2023).”
Backgrounde-FEVER: Explanations and Summaries forAutomated Fact Checking.
Stammbach, D., & Ash, E. (2020) · Cited by 4
Chinese Semantic Matching with Multi-granularity Alignment and Feature Fusion
Zhao, P., Lü, W., Li, Y., Yu, J., Jian, P., & Zhang, X. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534130 · Cited by 4
“…and candidate answers in question answering system [1] and dialogue system [2], to judge whether a hypothesis sentence could be inferred by another premise one in natural language inference [3], and to assess whether two sentences convey the same intention in semantic equivalence identification [4].”
BackgroundUnsupervised Explanation Generation via Correct Instantiations
Cheng, S., Wu, Z., Chen, J., Li, Z., Liu, Y., & Kong, L. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26494 · Cited by 5
XplaiNLI: Explainable Natural Language Inference through Visual Analytics
Kalouli, A.-L., Sevastjanova, R., de Paiva, V., Crouch, R., & El‐Assady, M. (2020) · doi:10.18653/v1/2020.coling-demos.9 · Cited by 4
“Another strand of research has approached the task by directly learning natural language explanations along with the inference decision (Camburu et al., 2018) or creating distributional representations of syntactic and semantic inference rules (Zanzotto and Ferrone, 2017) and training…”
BackgroundNatural Language Inference with a Human Touch: Using Human Explanations to Guide Model Attention.Preprint
Stacey, J., Belinkov, Y., & Rei, M. (2021) · arXiv (Cornell University) · Cited by 4
Are Training Resources Insufficient? Predict First Then Explain!Preprint
Jang, M., & Lukasiewicz, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.02056 · Cited by 11
“For the experiments, we used two datasets for commonsense-knowledge-requiring tasks: e-SNLI (Camburu et al., 2018) for the NLI task and CosE (Rajani et al., 2019) for the CQA task.”
InfluentialSelf-citationLearning to Rationalize for Nonmonotonic Reasoning with Distant Supervision
Brahman, F., Shwartz, V., Rudinger, R., & Choi, Y. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i14.17492 · Cited by 8
“Recent work by Camburu et al. (2018) and Ra-jani et al. (2019) collected human-written explanations for the natural language inference (NLI; Bowman et al. 2015) and commonsense question answering (CommonSenseQA; Talmor et al. 2019) tasks and trained models to predict explanations for new instances.”
BackgroundMethodInfluentialASPIRE: A Multi-Agent Framework for Execution-Free Code Analysis and Repair
Ramanan, B., Khan, M. A., & Rao, A. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825553 · Cited by 3
Metaphor and Large Language Models: When Surface Features Matter More than Deep Understanding
Sanchez-Bayona, E., & Agerri, R. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.898 · Cited by 14
Self-Rationalization in the Wild: A Large-scale Out-of-Distribution Evaluation on NLI-related tasks
Yang, J., Glockner, M., Rocha, A., & Gurevych, I. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00741 · Cited by 4
Exploring Reasoning Schemes: A Dataset for Syllogism Figure Identification
Peng, S., Liu, L., Liu, C., & Yu, D. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-81197-6_37 · Cited by 3
Single-Turn Debate Does Not Help Humans Answer Hard Reading-Comprehension Questions
Parrish, A., Trivedi, H. K., Perez, E., Chen, A., Nangia, N., Phang, J., & Bowman, S. R. (2022) · doi:10.18653/v1/2022.lnls-1.3 · Cited by 25
Kunz, J., Jirenius, M., Holmström, O., & Kuhlmann, M. (2022) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2022.blackboxnlp-1.14 · Cited by 4
“The template-like explanations in e-SNLI have also been noted by Camburu et al. (2018) and Brahman et al. (2021).”
BackgroundMethodInfluentialKavumba, P., Brassard, A., Heinzerling, B., & Inui, K. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.162 · Cited by 4
Rao, K., Jiang, L., Pyatkin, V., Gu, Y., Tandon, N., Dziri, N., Brahman, F., & Choi, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.812 · Cited by 6
Ferraz, T. P., Duarte, C. H. D., Ribeiro, M. F., Takayanagi, G. G. B., Alcoforado, A., Lopes, R. D. D., & Susi, M. (2024) · Estudos Avançados · doi:10.1590/s0103-4014.202438111.020 · Cited by 4
Effective Cross-Task Transfer Learning for Explainable Natural Language Inference with T5
Bigoulaeva, I., Sachdeva, R. S., Madabushi, H. T., Villavicencio, A., & Gurevych, I. (2022) · Workshop on Figurative Language Processing (FigLang) · doi:10.18653/v1/2022.flp-1.8 · Cited by 5
Jang, M., Majumder, B. P., McAuley, J., Lukasiewicz, T., & Camburu, O.-M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.47 · Cited by 4
Self-citationPham, T. M., Bui, T., Mai, L., & Nguyen, A. (2022) · Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (AACL-IJCNLP) · doi:10.18653/v1/2022.aacl-main.2 · Cited by 3
Valentino, M., Pratt-Hartmann, I. E., & Freitas, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.01974 · Cited by 3
“Despite the popularity of these datasets, and their application for measuring explainability on tasks such as Textual Entailment (Camburu et al., 2018), Multiple-choice Question Answering (Xie et al., 2020; Jhamtani and Clark, 2020; Khot et al., 2020; Jansen et al., 2018), and other inference tasks…”
BackgroundMethodInfluentialOn Measuring Faithfulness or Self-consistency of Natural Language Explanations
Parcalabescu, L., & Frank, A. (2023) · arXiv · doi:10.48550/arxiv.2311.07466 · Cited by 27
“Shortcoming of this test are: i) The hand-crafted rules to construct inputs from model explanations are specific for the e-SNLI (Camburu et al., 2018) and ComVE (Wang et al., 2020) datasets, but are not applicable, e.g., for CoS-E (Rajani et al., 2019).”
YNU-oxz at SemEval-2020 Task 4: Commonsense Validation Using BERT with Bidirectional GRU
Ou, X., & Li, H. (2020) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2020.semeval-1.80 · Cited by 2
“(Camburu et al., 2018) generates interpretation and prediction of natural language inference problems.”
BackgroundScene-text Oriented Visual Entailment: Task, Dataset and Solution
Li, N., Li, P., Xu, D., Zhao, W., Cai, Y., & Huang, Q. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612593 · Cited by 2
“Virginie et al. [26] propose e-SNLI-VE corpus, which append human-written natural language explanations to e-SNLI [12].”
BackgroundREFER: An End-to-end Rationale Extraction Framework for Explanation Regularization
Madani, M. R. G., & Minervini, P. (2023) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2023.conll-1.40 · Cited by 5
Exploring Explainability in Arabic Language Models: An Empirical Analysis of Techniques
Azzem, Y. C. H., Harrag, F., & Bellatreche, L. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.10.194 · Cited by 3
Annotation Difficulties in Natural Language Inference
Kalouli, A.-L., Real, L., Buis, A., Palmer, M. S., & de Paiva, V. (2021) · Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL) · doi:10.5753/stil.2021.17804 · Cited by 2
“However, these tend to be very expensive (if at all possible) during and after the corpus construction [Camburu et al. 2018].”
BackgroundMPII: Multi-Level Mutual Promotion for Inference and Interpretation
Liu, Y., Chen, S., Yang, Y., & Dai, Q. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.488 · Cited by 2
“We use six datasets as our testbeds: SNLI (Bow-man et al., 2015), e-SNLI (Camburu et al., 2018), CQA (Talmor et al., 2019), CoS-E (Rajani et al., 2019), MultiNLI (Williams et al., 2018), and SICK-E (Marelli et al., 2014).”
MethodAn Investigation of Language Model Interpretability via Sentence Editing
Stevens, S., & Su, Y. (2021) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2021.blackboxnlp-1.34 · Cited by 2
“In other cases, rationales are collected for an existing dataset by different annotators (Zaidan et al., 2008; Camburu et al., 2018; Rajani et al., 2019).”
BackgroundTowards Unifying the Explainability Evaluation Methods for NLP
Lucaci, D., & Inkpen, D. Z. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-88483-3_23 · Cited by 2
Natural Language Explanatory Arguments for Correct and Incorrect Diagnoses of Clinical Cases
Marro, S., Molinet, B., Cabrio, E., & Villata, S. (2023) · International Conference on Agents and Artificial Intelligence (ICAART) · doi:10.5220/0011927000003393 · Cited by 4
“(Kumar and Talukdar, 2020) propose to generate short explanations with GPT-2 (Radford et al., 2019), learned together with the input by a classifier to improve the final label prediction, using e-SNLI (Camburu et al., 2018).”
Metadata
- Authors
- Camburu, O.-M., Rocktäschel, T., Lukasiewicz, T., & Blunsom, P.
- Affiliations
- University of Oxford (GB)
- University College London (GB)
- Google (United States) (US)
- ORCID
- Lukasiewicz, Thomas
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- Volume
- 31
- Pages
- 9539-9549
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1812.01193
- OpenAlex
- W2891012317
- Semantic Scholar
- c242438dac5aa4d9b13766c14240bb8426690d58
- Cited by
- 367
- References
- 27
- Citations by year
- 2025: 19 · 2024: 33 · 2023: 94 · 2022: 73 · 2021: 84 · 2020: 55 · 2019: 9
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- arXiv (Cornell University)
- neural information processing systems
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- natural language inference, natural language explanations, entailment relations, explainable AI, out-of-domain transfer, interpretability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
