Rationalizing Neural Predictions
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Prediction without justification has limited applicability.As a remedy, we learn to extract pieces of input text as justifications -rationales -that are tailored to be short and coherent, yet sufficient for making the same prediction.Our approach combines two modular components, generator and encoder, which are trained to operate well together.The generator specifies a distribution over text fragments as candidate rationales and these are passed through the encoder for prediction.Rationales are never given during training.Instead, the model is regularized by desiderata for rationales.We evaluate the approach on multi-aspect sentiment analysis against manually annotated test cases.Our approach outperforms attention-based baseline by a significant margin.We also successfully illustrate the method on the question retrieval task. 1
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“A lot of work has also been done on argumentation and dialog games [6, 11, 51, 53] but the focus in these areas is generally the logical structure of the framework to express and to relate arguments or the protocol to exchange arguments.”
From Beyond explainability: justifiability and contestability of algorithmic decision systems (2022) · cites this paper as [53]
Experimental — accuracy not yet measuredIn the citing paper
“This model can achieve competitive accuracy while having built-in interpretability because the prediction is made exclusively based on rationales.”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [52]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 2×
“Attention based models offer another lens to the inner workings of neural models (Bahdanau et al., 2015; Cheng et al., 2016; Martins and Astudillo, 2016; Chen et al., 2015; Xu and Saenko, 2015; Yang et al., 2015).”
BackgroundConvolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2016) · arXiv · doi:10.48550/arxiv.1607.01759 · Cited by 4,542
A Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1062 · Cited by 3,601 · cited 1×
“For instance, we could use a convolutional network (Kim, 2014; Kalchbrenner et al., 2014), deep averaging network (Iyyer et al.”
MethodA Neural Attention Model for Abstractive Sentence Summarization
Rush, A. M., Chopra, S., & Weston, J. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1044 · Cited by 2,656
Recurrent Models of Visual Attention
Mnih, V., Heess, N., Graves, A., & Kavukcuoglu, K. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.6247 · Cited by 2,568 · cited 1×
“space of possible rationales in terms of their interaction with the encoder. We could also apply variance reduction techniques to increase stability of stochastic training (cf. (Weaver and Tao, 2001; Mnih et al., 2014; Ba et al., 2015; Xu et al., 2015)). 7 Acknowledgments We thank Prof. Julian McAuley for sharing the…”
MethodTeaching Machines to Read and Comprehend
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,521
Research works printed but not linked: 6
- [6]6. Cicero dos Santos, Luciano Barbosa, Dasha Bogdanova, and Bianca Zadrozny. 2015. Learning hybrid rep- resentations to retrieve semantically equivalent ques- tions. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Lan- guage Processing (Volume 2: Short Papers), pages 694-699, Beijing, China, July. Association for Com- putational Linguistics.
- [7]7. Manaal Faruqui, Jesse Dodge, Sujay K. Jauhar, Chris Dyer, Eduard Hovy, and Noah A. Smith. 2015a. Retrofitting word vectors to semantic lexicons. In Pro- ceedings of NAACL.
- [9]9. Aurélie Herbelot and Eva Maria Vecchi. 2015. Build- ing a shared world: mapping distributional to model- theoretic semantic spaces. In Proceedings of the 2015 Conference on Empirical Methods in Natural Lan- guage Processing. Association for Computational Lin- guistics.
- [12]12. Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III. 2015. Deep unordered compo- sition rivals syntactic methods for text classification. In Proceedings of the 53rd Annual Meeting of the As- sociation for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers).
- [27]27. Tim Rocktäschel, Edward Grefenstette, Karl Moritz Her- mann, Tomáš Kočiskỳ, and Phil Blunsom. 2016. Rea- soning about entailment with neural attention. In In- ternational Conference on Learning Representations.
- [29]29. Sebastian Thrun. 1995. Extracting rules from artifi- cial neural networks with distributed representations. In Advances in neural information processing systems (NIPS).
Cited by
Items 201–300 of 900
Interpretable Rationale Augmented Charge Prediction System
Jiang, X., Ye, H., Luo, Z., Chao, W., & Ma, W. (2018) · International Conference on Computational Linguistics · Cited by 50
Evaluation of Similarity-based ExplanationsPreprint
Hanawa, K., Yokoi, S., Hara, S., & Inui, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.04528 · Cited by 55
Synthesizing configuration file specifications with association rule learning
Santolucito, M., Zhai, E., Dhodapkar, R. M., Shim, A., & Piskač, R. (2017) · Proceedings of the ACM on Programming Languages · doi:10.1145/3133888 · Cited by 50
“While it is possible to analyze the models from other machine learning techniques, such as neural networks [Lei et al. 2016] and conditional random fields [Raychev et al.”
MethodExplainable AI in HealthcarePreprint
Khedkar, S., Subramanian, V., Shinde, G., & Gandhi, P. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3367686 · Cited by 53
SELFEXPLAIN: A Self-Explaining Architecture for Neural Text Classifiers
Rajagopal, D., Balachandran, V., Hovy, E. H., & Tsvetkov, Y. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.64 · Cited by 40
“Another recent line of work explores collecting rationales (Lei et al., 2016) through expert annotations (Zaidan and Eisner, 2008).”
BackgroundMethodThirdEye: Attention Maps for Safe Autonomous Driving Systems
Stocco, A., Nunes, P. J., d’Amorim, M., & Tonella, P. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3556968 · Cited by 64
“Explaining the predictions of DNNs has been largely studied using several interpretation methods [37, 40, 50, 51, 56, 63, 76].”
BackgroundMethodImproving BERT Model Using Contrastive Learning for Biomedical Relation Extraction
Su, P., Peng, Y., & Vijay‐Shanker, K. (2021) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2021.bionlp-1.1 · Cited by 53
CXAI: Explaining Convolutional Neural Networks for Medical Imaging Diagnostic
Rguibi, Z., Hajami, A., Zitouni, D., Elqaraoui, A., & Bedraoui, A. (2022) · Electronics · doi:10.3390/electronics11111775 · Cited by 57
“For example, the models introduced by Lei et al. [22,23] do not have supervision on the explanations but only at the final prediction.”
BackgroundMethodIcoRating: A Deep-Learning System for Scam ICO IdentificationPreprint
Bian, S., Deng, Z., Li, F., Monroe, W., Shi, P. J., Sun, Z., Wu, W., Wang, S., Wang, W. Y., Yuan, A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.03670 · Cited by 55
“Deep learning models are hard to be rationalized directly (Lei et al., 2016; Mahendran and Vedaldi, 2015; Weinzaepfel et al., 2011; Vondrick et al., 2013).”
BackgroundAssessing Composition in Sentence Vector RepresentationsPreprint
Ettinger, A., Elgohary, A., Phillips, C., & Resnik, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.03992 · Cited by 86
“…(Li et al., 2015; K´ad´ar et al., 2016), efforts to increase interpretability by generating explanations of model predictions (Ribeiro et al., 2016; Lei et al., 2016; Li et al., 2016), and work submitting adversarial examples to systems in order to identify weaknesses (Zhao et al., 2017; Jia…”
BackgroundA Game Theoretic Approach to Class-wise Selective Rationalization
Chang, S., Zhang, Y., Yu, M., & Jaakkola, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12853 · Cited by 38
“The first class of method is to explicitly introduce a generator that learns to select important subsets of inputs as explanations [12, 19, 21, 30, 31], which often comes with some information-theoretic properties.”
BackgroundMethodInfluentialSelf-citationDescription Based Text Classification with Reinforcement LearningPreprint
Chai, D., Wu, W., Han, Q., Wu, F. Q., & Li, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.03067 · Cited by 56
“Take the task of aspect sentiment classification (Lei et al., 2016) as an example, the goal of which is to classify the sentiment of a specific aspect of a review.”
BackgroundMethodSelf-Explaining Structures Improve NLP ModelsPreprint
Sun, Z., Fan, C. C., Han, Q., Sun, X., Meng, Y., Wu, F. Q., & Li, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.01786 · Cited by 62
“In NLP, approaches to interpret neural models include extracting pieces of input text, called “rationales”, as justifications to model predictions (Lei et al., 2016; Chang et al., 2019; DeYoung et al., 2020; Jain et al., 2020), studying the efficacy and dynamics of hidden states in recurrent networks (Karpathy et al.”
BackgroundMethodInfluentialExplicit Document Modeling through Weighted Multiple-Instance Learning
Pappas, N., & Popescu-Belis, A. (2017) · Journal of Artificial Intelligence Research · doi:10.1613/jair.5240 · Cited by 57
“In terms of non-neural methods, the first two lines show that our MIR model with unigram features largely outperforms SVM with bigram features as evaluated by Lei et al. (2016), while using far fewer parameters ( | θ | ).”
BackgroundMethodInfluentialGenerating Token-Level Explanations for Natural Language Inference
Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2019) · Proceedings of the 2019 Conference of the North · doi:10.18653/v1/n19-1101 · Cited by 41
“Bansal, 2016), augment models to generate explanations accompanying their predictions (Lei et al., 2016; Camburu et al., 2018), or expose model internals such as magnitude of hidden states (Linzen et al.”
BackgroundMARTA: Leveraging Human Rationales for Explainable Text Classification
Arous, I., Dolamic, L., Yang, J., Bhardwaj, A., Cuccu, G., & Cudré-Mauroux, P. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i7.16734 · Cited by 41
“The idea of finding a small subset of input units capable of generating the same output has resulted in various selective rationalization techniques (Lei, Barzilay, and Jaakkola 2016; Li, Monroe, and Jurafsky 2016; Chen et al. 2018; Chang et al. 2019; Yu et al. 2019).”
MethodExplaining and Improving Model Behavior with k Nearest Neighbor RepresentationsPreprint
Rajani, N. F., Krause, B., Yin, W., Niu, T., Socher, R., & Xiong, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.09030 · Cited by 54
“…techniques for NLP models have focused on explaining individual predictions by using gradient-based saliency maps over the input text (Lei, Barzilay, and Jaakkola 2016; Ribeiro, Singh, and Guestrin 2018; Bastings, Aziz, and Titov 2019) or interpreting attention (Brunner et al. 2020;…”
Method(2019) · doi:10.18653/v1/w19-4807 · Cited by 43
“Lei et al. (2016) present a model that extracts rationales for predictions without training on rationales.”
BackgroundA Framework for Explainable Text Classification in Legal Document Review
Mahoney, C. J., Zhang, J., Huber-Fliflet, N., Gronvall, P., & Zhao, H. (2019) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata47090.2019.9005659 · Cited by 43
“[9] proposed a neural network approach to generate rationales for text classifications.”
MethodChrysostomou, G., & Αλέτρας, Ν. (2021) · doi:10.18653/v1/2021.acl-long.40 · Cited by 68
“Finally, extracting a short snippet from the original input text (rationale) and using it to make a prediction has been recently proposed (Lei et al., 2016; Bastings et al., 2019; Tre-viso and Martins, 2020; Jain et al., 2020; Chalkidis et al., 2021).”
Model agnostic interpretability of rankers via intent modelling
Singh, J., & Anand, A. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3375234 · Cited by 54
“Recently, in the machine learning (ML) and NLP communities there has been growing interest in the interpretability of ML models [5, 24, 27] but there has been limited work on interpreting retrieval/ranking models considered central to information retrieval (IR).”
BackgroundXFlag: Explainable Fake News Detection Model on Social Media
Chien, S.-Y. J., Yang, C., & Yu, F. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2062113 · Cited by 50
“…part of daily life, for AI-based recommendations to be trusted, it is crucial that users are able to understand the logic behind the system (Barredo Arrieta et al., 2020; Lei et al., 2016), that is, we must be able to show them how the predictions came about and how the decisions were generated.”
BackgroundDistilling ChatGPT for Explainable Automated Student Answer Assessment
Li, J., Gui, L., Zhou, Y., West, D., Aloisi, C., & He, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.399 · Cited by 67
Deep Weighted Averaging Classifiers
Card, D., Zhang, M., & Smith, N. A. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287595 · Cited by 37
“…more complex models, in a way that provides some utility to the user or developer of a model beyond what is attainable through more direct means [2, 27, 30, 42–44 In this paper, we propose a method which, like those in the former category, offers an explanation that is transparent (in that the…”
MethodInterpreting Deep Models for Text Analysis via Optimization and Regularization Methods
Yuan, H., Chen, Y., Hu, X., & Ji, S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33015717 · Cited by 47
“Most existing approaches for interpreting NLP models only investigate the relationships between input sentences and output decisions to explore which input words are more important to make decisions (Lei, Barzilay, and Jaakkola 2016; Li et al. 2015).”
BackgroundWang, J., Sun, C., Li, S., Liu, X., Si, L., Zhang, M., & Zhou, G. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1345 · Cited by 48
“In order to differentiate whether a word is selected or discarded, inspired by Lei et al. (2016), we incorporate the action result o i into the input ˆ v i of LSTM p at time-step i and compute hidden state h i ∈ R d of word x i as: (1) Aspect Vector (cid:89) where v i ∈ R d is word embedding of…”
BackgroundDetecting and Explaining Crisis
Kshirsagar, R., Morris, R., & Bowman, S. (2017) · Proceedings of the Fourth Workshop on Computational Linguistics and Clinical Psychology –- From Linguistic Signal to Clinical Reality · doi:10.18653/v1/w17-3108 · Cited by 34
“In the future experiments, we hope to explore human evaluation of the generated explanations as an indicator of trust in the model, to investigate compression-based approaches to explanation (Lei et al., 2016), and to consider richer architectures for text classification.”
BackgroundMethodInfluentialTowards Interactivity and Interpretability: A Rationale-based Legal Judgment Prediction Framework
Wu, Y., Liu, Y., Lü, W., Zhang, Y., Feng, J., Sun, C., Wu, F. Q., & Kuang, K. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.316 · Cited by 41
“Lei et al. (2016) defines the rationale as some key sentences extracted from the input text and uses these extracted sentences to make predictions.”
BackgroundHollingshead, K., Ireland, M. E., & Loveys, K. (2017) · doi:10.18653/v1/w17-31 · Cited by 44
CHEF: A Pilot Chinese Dataset for Evidence-Based Fact-Checking
Hu, X., Guo, Z., Wu, G., Liu, A., Wen, L., & Yu, P. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.246 · Cited by 86
A Visual Interaction Framework for Dimensionality Reduction Based Data Exploration
Cavallo, M., & Demiralp, Ç. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174209 · Cited by 49
“,[32, 37, 39, 47]), but much still must be done in this direction, particularly through close collaboration of HCI and machine-learning researchers.”
BackgroundIslam, S. R., Eberle, W., Bundy, S. C., & Ghafoor, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11474 · Cited by 47
“Lei et al. [52] propose an approach that generates the rationale for a prediction.”
BackgroundCarton, S., Mei, Q., & Resnick, P. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1386 · Cited by 48
“Following (Lei et al., 2016), we refer to the maskproducing layer g as the generator, but for clarity we call the predictive layer f the predictor rather than the encoder.”
BackgroundMethodInfluentialChhatwal, R., Gronvall, P., Huber-Fliflet, N., Keeling, R., Zhang, J., & Zhao, H. (2018) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2018.8622073 · Cited by 39
“[9] proposed an approach to generate rationales for text classifications.”
MethodLearning Credible Deep Neural Networks with Rationale Regularization
Du, M., Liu, N., Yang, F., & Hu, X. (2019) · Proceedings · doi:10.1109/icdm.2019.00025 · Cited by 51
“Rationales are provided by [16], which are also sub-sentential snippets indicating higher relevance for prediction (see Fig.”
BackgroundFIND: Human-in-the-Loop Debugging Deep Text Classifiers
Lertvittayakumjorn, P., Specia, L., & Toni, F. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.24 · Cited by 55
A Survey of Computer Vision Technologies in Urban and Controlled-environment Agriculture
Luo, J., Li, B., & Leung, C. (2023) · ACM Computing Surveys · doi:10.1145/3626186 · Cited by 50
“Moreover, inherently explainable models [33, 59, 178, 267, 354] incorporate explainable”
A Roadmap of Explainable Artificial Intelligence: Explain to Whom, When, What and How?
Wang, Z., Huang, C., & Yao, X. (2024) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3702004 · Cited by 51
“AE aims to establish associations between inputs [5], internal model structures (e.g., neurons) [51], or outputs [117] with human-understandable conceptual objects (e.g., natural language, images) by modifying the ML model’s architecture or incorporating regularization terms [175].”
A Comprehensive Survey on Self-Interpretable Neural Networks
Ji, Y., Sun, Y., Zhang, Y., Wang, Z., Zhuang, Y., Gong, Z., Shen, D., Qin, C., Zhu, H., & Xiong, H. (2025) · Proceedings of the IEEE · doi:10.1109/jproc.2025.3635153 · Cited by 43
“A common approach is to directly impose ℓ 1 -norm on the input features [113], [134], [135], which can be formulated as: C lasso = { α ( x ; θ ) ∈ R N : ∥ α ( x ; θ ) ∥ 1 ≤ λ } , where λ is a hyperparameter.”
A Visual Interaction Framework for Dimensionality Reduction Based Data Exploration
Cavallo, M., & Demiralp, Ç. (2018) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3170427.3186508 · Cited by 45
Explainable AI for Text Classification: Lessons from a Comprehensive Evaluation of Post Hoc Methods
Cesarini, M., Malandri, L., Pallucchini, F., Seveso, A., & Xing, F. (2024) · Cognitive Computation · doi:10.1007/s12559-024-10325-w · Cited by 53
“These include techniques such as the one proposed by Lei et al. [26], which extracts concise and coherent pieces of input text as jus-tifications; TCAV [27], utilising directional derivatives to measure the importance of user-defined concepts; Check-List [28], which evaluates explainers’ capabilities…”
Code Failure Prediction and Pattern Extraction Using LSTM Networks
Hajiaghayi, M., & Vahedi, E. (2019) · IEEE International Conference on Big Data Computing Service and Applications (BigDataService) · doi:10.1109/bigdataservice.2019.00014 · Cited by 38
“Recently there have been some efforts by researchers to better understand the decision making process in recurrent neural networks with applications in NLP [21] [22] [23] [24] and Genomic Sequencing [25].”
BackgroundReinforced Extractive Summarization with Question-Focused Rewards
Arumae, K., & Liu, F. (2018) · Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW) · doi:10.18653/v1/p18-3015 · Cited by 37
“Following (Lei et al., 2016), we introduce a single-direction LSTM encoder whose hidden state s t tracks the sampling decisions up to time step t (Eq.”
BackgroundMethodInfluentialRationalizing Text Matching: Learning Sparse Alignments via Optimal Transport
Swanson, K., Yu, L., & Lei, T. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.496 · Cited by 43
Self-citationChalkidis, I., Fergadiotis, M., Manginas, N., Katakalou, E., & Malakasiotis, P. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.305 · Cited by 50
Assessing the Ability of Self-Attention Networks to Learn Word Order
Yang, B., Wang, L., Wong, D. F., Chao, L. S., & Tu, Z. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1354 · Cited by 36
“Recent studies also pointed out that abandoning irrelevant features in source sentence benefits to some downstream NLP tasks (Lei et al., 2016; Yu et al., 2017; Shen et al., 2018b).”
Annotator Rationales for Labeling Tasks in Crowdsourcing
Kutlu, M., McDonnell, T., Lease, M., & Elsayed, T. (2020) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12012 · Cited by 42
“Recent work on neural models has continued to investigate use of such constrained rationales to both supervise models (Zhang et al., 2016; Bao et al., 2018) and evaluate unsupervised models (Lei et al., 2016; Bastings et al., 2019).”
BackgroundThe Explanation Game: Towards Prediction Explainability through Sparse Communication
Treviso, M. V., & Martins, A. F. T. (2020) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2020.blackboxnlp-1.10 · Cited by 46
“Both our proposed models (sparsemax and 1.5-entmax) seem generally more accurate than the Bernoulli model of Lei et al. (2016) and comparable to the HardKuma model of Bastings et al. (2019), with a much simpler training procedure, not requiring gradient estimation over stochastic computation graphs.”
BackgroundMethodInfluentialHuman Interpretation of Saliency-based Explanation Over Text
Schuff, H., Jacovi, A., Adel, H., Goldberg, Y., & Vu, N. T. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533127 · Cited by 59
“To demonstrate, consider the case of binary feature-attribution—marking parts of the input as “important” and “not important”, also known as highlighting or extractive rationalization [33].”
BackgroundExplaining Deep Neural NetworksPreprint
Camburu, O.-M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01496 · Cited by 49
“Similarly to Lei et al. (2016), I will treat the task as a regression, with the score being a real number linearly reflecting the intensity of the sentiment.”
BackgroundMethodInfluentialToward Practical Usage of the Attention Mechanism as a Tool for Interpretability
Tutek, M., & Šnajder, J. (2022) · IEEE Access · doi:10.1109/access.2022.3169772 · Cited by 46
Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Prediction
Chu, E., Roy, D., & Andreas, J. (2020) · arXiv · doi:10.48550/arxiv.2007.12248 · Cited by 34
Guiding Extractive Summarization with Question-Answering Rewards
Arumae, K., & Liu, F. (2019) · Proceedings of the 2019 Conference of the North · doi:10.18653/v1/n19-1264 · Cited by 36
“Our proposed method is inspired by the work of Lei et al. (2016) who seek to identify rationales from textual input to support sentiment classification and question retrieval.”
MethodInterpretable-by-Design Text Understanding with Iteratively Generated Concept BottleneckPreprint
Ludan, J. M., Lyu, Q., Yang, Y., Dugan, L. D., Yatskar, M., & Callison-Burch, C. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.19660 · Cited by 47
“Token-based explanations, such as rationales (Zaidan et al., 2007; Lei et al., 2016; Bastings et al., 2019), provide a span of important tokens that are minimally sufficient for the prediction.”
BackgroundSenseMate: An Accessible and Beginner-Friendly Human-AI Platform for Qualitative Data Analysis
Overney, C., Saldías-Fuentes, B., Dimitrakopoulou, D., & Roy, D. K. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645194 · Cited by 46
MUSE: Modularizing Unsupervised Sense Embeddings
Lee, G.-H., & Chen, Y.-N. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1034 · Cited by 32
“supports stochastic optimization (Lei et al., 2016), which uses a stochastic sample zik for optimization.”
BackgroundVector space explorations of literary language
van Cranenburgh, A., van Dalen-Oskam, K., & van Zundert, J. (2019) · Language Resources and Evaluation · doi:10.1007/s10579-018-09442-4 · Cited by 32
“Therefore we can expect that the classification of a review can be readily attributed to a limited set of surface features (words or phrases) that explain why a review is positive or negative (called a rationale in Lei et al. 2016).”
BackgroundA Diagnostic Study of Explainability Techniques for Text Classification
Atanasova, P. (2020) · Springer eBooks · doi:10.1007/978-3-031-51518-7_7 · Cited by 51
Optimizing for Interpretability in Deep Neural Networks with Tree Regularization
Wu, M., Parbhoo, S., Hughes, M. C., Röth, V., & Doshi‐Velez, F. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12558 · Cited by 40
Query Understanding in the Age of Large Language ModelsPreprint
Anand, A., Venktesh, V., Anand, A., & Setty, V. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.16004 · Cited by 39
“are either interpretable-by-design or posthoc methods [36, 38, 89].”
BackgroundCan Rationalization Improve Robustness?
Chen, H., He, J., Narasimhan, K., & Chen, D. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.278 · Cited by 31
Rethinking Cooperative Rationalization: Introspective Extraction and Complement ControlPreprint
Yu, M., Chang, S., Zhang, Y., & Jaakkola, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.13294 · Cited by 31
“From the left part of Table 4, we observe that the original model of (Lei et al., 2016) has a hard time maintaining the accuracy compared to the classi-fier trained with full texts.”
BackgroundMethodResultInfluentialSelf-citationCountering Online Hate Speech: An NLP PerspectivePreprint
Chaudhary, M., Saxena, C., & Meng, H. M. L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.02941 · Cited by 34
“They use RCNN recurrent cells (Lei et al., 2016b) in their classifier.”
MethodFrom ”Explainable AI” to ”Graspable AI”
Ghajargar, M., Bardzell, J., Smith-Renner, A., Krogh, P. G., Höök, K., Cuartielles, D. J., Boer, L., & Wiberg, M. (2021) · International Conference on Tangible, Embedded, and Embodied Interaction (TEI) · doi:10.1145/3430524.3442704 · Cited by 38
“, explaining—or rationalizing—individual predictions) [23].”
REV: Information-Theoretic Evaluation of Free-Text Rationales
Chen, H., Brahman, F., Ren, X., Ji, Y., Choi, Y., & Swayamdipta, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.112 · Cited by 55
Finding decision jumps in text classification
Liu, X., Mou, L., Cui, H., Lu, Z., & Song, S. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.08.082 · Cited by 28
Finding Generalizable Evidence by Learning to Convince Q&A Models
Perez, E., Karamcheti, S., Fergus, R., Weston, J., Kiela, D., & Cho, K. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1244 · Cited by 27
“Inspired by work in interpreting neural networks (Lei et al., 2016), we have agents find a subset of the passage (i.”
MethodA Puzzle concerning Compositionality in Machines
Nefdt, R. M. (2020) · Minds and Machines · doi:10.1007/s11023-020-09519-6 · Cited by 41
“Promising research in this vein includes Lei et al. (2016) using the extraction of unsupervised input text as justification (or “rationales”) for prediction in neural networks based on a multi-aspect sentiment analysis, and building on this the work of Yu et al. (2019) aimed at identifying…”
MethodLearning to Explain: Generating Stable Explanations Fast
Situ, X., Zukerman, I., Paris, C. L., Maruf, S., & Haffari, G. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.415 · Cited by 46
“Other works train a classifier and an explainer jointly in order to incorporate explainability directly into the classifier (Lei et al., 2016; Cam-buru et al., 2018).”
BackgroundExplainable Prediction of Text Complexity: The Missing Preliminaries for Text Simplification
Gârbacea, C., Guo, M., Carton, S., & Mei, Q. (2021) · doi:10.18653/v1/2021.acl-long.88 · Cited by 27
“Explainable classification can also be solved simultaneously through a neural network, using hard attentions to select individual words into the “rationale” behind a classification decision (Lei et al., 2016).”
BackgroundInterpreting BERT-based Text Similarity via Activation and Saliency Maps
Malkiel, I., Ginzburg, D., Barkan, O., Caciularu, A., Weill, J., & Koenigstein, N. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512045 · Cited by 38
“The first works that proposed interpretation of textual-based networks include [32], and [40].”
BackgroundAssuring Safe and Efficient Operation of UAV Using Explainable Machine Learning
Alharbi, A., Petrunin, I. V., & Panagiotakopoulos, D. (2023) · Drones · doi:10.3390/drones7050327 · Cited by 32
A Causality Inspired Framework for Model Interpretation
Wu, C., Wang, X., Lian, D., Xie, X., & Chen, E. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599240 · Cited by 39
“We compare the proposed CIMI with nine base-lines, including Gradient [41], Attention [3], LIME [34], Ker-nelSHAP [23], Rationale [20], Probing [1], CXPlain [37], AXAI [33], Smask [21].”
MethodInfluentialPathogenomics for accurate diagnosis, treatment, prognosis of oncology: a cutting edge overview
Feng, X., Shu, W., Li, M., Li, J., Xu, J., & He, M. (2024) · Journal of Translational Medicine · doi:10.1186/s12967-024-04915-3 · Cited by 35
“…[115] are currently the most widespread form of explanatory techniques; while self-explanatory methods with feature-based explanations include RCNN [116], CAR [117], INVASE [118], as well as the influential class of attention models [119] are commonly used for common features like super-pixels for…”
MethodEvaluating Explanation Methods for Neural Machine Translation
Li, J., Liu, L., Li, H., Li, G., Huang, G., & Shi, S. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.35 · Cited by 31
Interpreting Image Classifiers by Generating Discrete Masks
Yuan, H., Cai, L., Hu, X., Wang, J., & Ji, S. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3028783 · Cited by 39
SEAL: Segment-wise Extractive-Abstractive Long-form Text SummarizationPreprint
Zhao, Y., Saleh, M., & Liu, P. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10213 · Cited by 27
“[30].”
Wang, J., Sun, C., Li, S., Wang, J., Si, L., Zhang, M., Liu, X., & Zhou, G. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1560 · Cited by 26
“We conduct our experiments on three public datasets on DASC, i.e., TripUser (Li et al., 2018), TripAdvisor (Wang et al., 2010) and BeerAdvo-cate (McAuley et al., 2012; Lei et al., 2016).”
MethodBenefits of Intermediate Annotations in Reading Comprehension
Dua, D., Singh, S., & Gardner, M. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.497 · Cited by 35
“Especially relevant is work on supervising models using explanations (Ross et al., 2017), which, similar to our annotations, identify parts of the input that are important for prediction (Lei et al., 2016; Ribeiro et al., 2016).”
BackgroundWhy do you think that? Exploring Faithful Sentence-Level Rationales Without Supervision
Glockner, M., Habernal, I., & Gurevych, I. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.97 · Cited by 34
FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation
Lakhotia, K., Paranjape, B., Ghoshal, A., Yih, S., Mehdad, Y., & Iyer, S. V. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.301 · Cited by 38
“Recent works have proposed the use of textual rationales (Lei et al., 2016; DeYoung et al., 2020; Latcinnik and Berant, 2020) as a means to either obtain an understanding of the reasoning process of models, or as a human-∗ Q: Is Sanskrit the first language of the world ?”
BackgroundThe Eval4NLP Shared Task on Explainable Quality Estimation: Overview and Results
Fomicheva, M., Lertvittayakumjorn, P., Zhao, W., Eger, S., & Gao, Y. (2021) · Workshop on Evaluation and Comparison of NLP Systems (Eval4NLP) · doi:10.18653/v1/2021.eval4nlp-1.17 · Cited by 46
UNIREX: A Unified Learning Framework for Language Model Rationale Extraction
Chan, A., Sanjabi, M., Mathias, L., Tan, L., Nie, S., Peng, X., Ren, X., & Firooz, H. (2022) · doi:10.18653/v1/2022.bigscience-1.5 · Cited by 26
Véstias, M. P. (2020) · Advances in information quality and management · doi:10.4018/978-1-7998-3479-3.ch002 · Cited by 28
Grounded Recurrent Neural NetworksPreprint
Vani, A., Jernite, Y., & Sontag, D. A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08557 · Cited by 28
“Thus, the model is able to combine the data efficiency of simple Bag-of-Word text classification methods with an RNN’s ability to model linguistic structures by tying labels of interest to specific dimensions of its hidden state.”
MethodRegular Inference on Artificial Neural Networks
Mayr, F., & Yovine, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99740-7_25 · Cited by 25
“However, ANN are considered to be a rather obscure model [21], meaning that understanding the specifics that were taken into consideration by the model to make a decision is not a trivial task.”
Recent Studies of XAI - Review
Hu, Z. F., Kuflik, T., Mocanu, I. G., Najafian, S., & Tal, A. S. (2021) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3450614.3463354 · Cited by 29
“[33] proposed a method of generating rationales, which are small chunks of sentences extracted from input sentences used to provide justifications.”
MethodExtractive Explanations for Interpretable Text Ranking
Leonhardt, J., Rudra, K., & Anand, A. (2022) · ACM Transactions on Information Systems · doi:10.1145/3576924 · Cited by 35
“Existing approaches rely on sampling from a stochastic distribution using the REINFORCE algo-rithm, resulting in a boolean mask over the sentences.”
MethodInfluentialSOLD: Sinhala offensive language dataset
Ranasinghe, T., Anuradha, I., Premasiri, D., Silva, K., Hettiarachchi, H., Uyangodage, L., & Zampieri, M. (2024) · Language Resources and Evaluation · doi:10.1007/s10579-024-09723-1 · Cited by 35
“These tokens can be used to train explainable models, as is shown in recent works [34, 35, 59].”
Bridging CNNs, RNNs, and Weighted Finite-State Machines
Schwartz, R., Thomson, S., & Smith, N. A. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1028 · Cited by 36
“Other works jointly learned to encode text and extract the span which best explains the model’s prediction (Yessenalina et al., 2010; Lei et al., 2016).”
BackgroundLearning Saliency Maps to Explain Deep Time Series Classifiers
Parvatharaju, P. S., Doddaiah, R., Hartvigsen, T., & Rundensteiner, E. A. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482446 · Cited by 43
“However, most of these XAI methods are designed for images [8, 9, 21, 29, 32, 39, 42, 47] and text [1, 20, 22, 36]. while very little has been done for time series.”
Hoppe, D. B., Hendriks, P., Ramscar, M., & van Rij, J. (2022) · Behavior Research Methods · doi:10.3758/s13428-021-01711-5 · Cited by 30
“…difficult to pinpoint which part of a model contributes to its behavior, and interestingly, there is currently a trend towards developing methods for analyzing the workings of complex neural networks (e.g., Adi, Kermany, Belinkov, Lavi, & Goldberg, 2016; Lei, Barzilay, & Jaakkola, 2016).”
MethodGraspable AI: Physical Forms as Explanation Modality for Explainable AI
Ghajargar, M., Bardzell, J., Smith, A., Höök, K., & Krogh, P. G. (2022) · International Conference on Tangible, Embedded, and Embodied Interaction (TEI) · doi:10.1145/3490149.3503666 · Cited by 32
“, explaining—or rationalizing—individual predictions) [22].”
A survey on improving NLP models with human explanations
Hartmann, M., & Sonntag, D. (2022) · doi:10.18653/v1/2022.lnls-1.5 · Cited by 38
From the Detection of Toxic Spans in Online Discussions to the Analysis of Toxic-to-Civil Transfer
Pavlopoulos, J., Laugier, L., Xenos, A., Sorensen, J. S., & Androutsopoulos, I. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.259 · Cited by 46
“Toxic spans detection can be seen as a case of attribution or rationale extraction (Li et al., 2016; Ribeiro et al., 2016; Lei et al., 2016; Zhang et al., 2021; Jain et al., 2020; DeYoung et al., 2020), but specifically for toxic posts, a task that has never been considered in general toxicity…”
BackgroundFeldhus, N., Hennig, L., Nasert, M. D., Ebert, C., Schwarzenberg, R., & Mller, S. (2023) · doi:10.18653/v1/2023.nlrse-1.4 · Cited by 36
“Most previous approaches in generating NLEs depend on datasets of human-annotated text highlights (Zaidan et al., 2007; Lei et al., 2016; Wiegreffe and Marasovi´c, 2021) or carefully constructed gold rationales for supervised training (Camburu et al., 2020; Wiegreffe et al., 2022), which are costly…”
BackgroundInfluentialJumper: Learning When to Make Classification Decision in Reading
Liu, X., Mou, L., Cui, H., Lu, Z., & Song, S. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/589 · Cited by 24
“Lei et al. [2016] build a neural text clas-sifier on key phrases in a paragraph, where key phrase extraction is learned by reinforcement learning with real-valued reward.”
BackgroundMethodA Machine Learning-based Approach for Automated Vulnerability Remediation Analysis
Zhang, F., Huff, P., McClanahan, K., & Li, Q. (2020) · IEEE Conference on Communications and Network Security · doi:10.1109/cns48642.2020.9162309 · Cited by 24
Coupling Distributed and Symbolic Execution for Natural Language QueriesPreprint
Mou, L., Lu, Z., Li, H., & Jin, Z. G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.02741 · Cited by 25
Metadata
- Authors
- Lei, T., Barzilay, R., & Jaakkola, T.
- Affiliations
- MIT Computer Science and Artificial Intelligence Laboratory (US)
- Venue
- Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing
- Published
- 2016
- Pages
- 107-117
- OpenAlex
- W2440159969
- Cited by
- 777
- References
- 27
- Citations by year
- 2026: 27 · 2025: 33 · 2024: 39 · 2023: 88 · 2022: 92 · 2021: 138 · 2020: 126 · 2019: 101 · 2018: 78 · 2017: 39 · 2016: 4
- FWCI
- 39.87
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- explanation generation, question retrieval, model regularization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.