Rationalizing Neural Predictions
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Prediction without justification has limited applicability.As a remedy, we learn to extract pieces of input text as justifications -rationales -that are tailored to be short and coherent, yet sufficient for making the same prediction.Our approach combines two modular components, generator and encoder, which are trained to operate well together.The generator specifies a distribution over text fragments as candidate rationales and these are passed through the encoder for prediction.Rationales are never given during training.Instead, the model is regularized by desiderata for rationales.We evaluate the approach on multi-aspect sentiment analysis against manually annotated test cases.Our approach outperforms attention-based baseline by a significant margin.We also successfully illustrate the method on the question retrieval task. 1
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“A lot of work has also been done on argumentation and dialog games [6, 11, 51, 53] but the focus in these areas is generally the logical structure of the framework to express and to relate arguments or the protocol to exchange arguments.”
From Beyond explainability: justifiability and contestability of algorithmic decision systems (2022) · cites this paper as [53]
Experimental — accuracy not yet measuredIn the citing paper
“This model can achieve competitive accuracy while having built-in interpretability because the prediction is made exclusively based on rationales.”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [52]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 2×
“Attention based models offer another lens to the inner workings of neural models (Bahdanau et al., 2015; Cheng et al., 2016; Martins and Astudillo, 2016; Chen et al., 2015; Xu and Saenko, 2015; Yang et al., 2015).”
BackgroundConvolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2016) · arXiv · doi:10.48550/arxiv.1607.01759 · Cited by 4,542
A Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1062 · Cited by 3,601 · cited 1×
“For instance, we could use a convolutional network (Kim, 2014; Kalchbrenner et al., 2014), deep averaging network (Iyyer et al.”
MethodA Neural Attention Model for Abstractive Sentence Summarization
Rush, A. M., Chopra, S., & Weston, J. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1044 · Cited by 2,656
Recurrent Models of Visual Attention
Mnih, V., Heess, N., Graves, A., & Kavukcuoglu, K. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.6247 · Cited by 2,568 · cited 1×
“space of possible rationales in terms of their interaction with the encoder. We could also apply variance reduction techniques to increase stability of stochastic training (cf. (Weaver and Tao, 2001; Mnih et al., 2014; Ba et al., 2015; Xu et al., 2015)). 7 Acknowledgments We thank Prof. Julian McAuley for sharing the…”
MethodTeaching Machines to Read and Comprehend
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,521
Research works printed but not linked: 6
- [6]6. Cicero dos Santos, Luciano Barbosa, Dasha Bogdanova, and Bianca Zadrozny. 2015. Learning hybrid rep- resentations to retrieve semantically equivalent ques- tions. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Lan- guage Processing (Volume 2: Short Papers), pages 694-699, Beijing, China, July. Association for Com- putational Linguistics.
- [7]7. Manaal Faruqui, Jesse Dodge, Sujay K. Jauhar, Chris Dyer, Eduard Hovy, and Noah A. Smith. 2015a. Retrofitting word vectors to semantic lexicons. In Pro- ceedings of NAACL.
- [9]9. Aurélie Herbelot and Eva Maria Vecchi. 2015. Build- ing a shared world: mapping distributional to model- theoretic semantic spaces. In Proceedings of the 2015 Conference on Empirical Methods in Natural Lan- guage Processing. Association for Computational Lin- guistics.
- [12]12. Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III. 2015. Deep unordered compo- sition rivals syntactic methods for text classification. In Proceedings of the 53rd Annual Meeting of the As- sociation for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers).
- [27]27. Tim Rocktäschel, Edward Grefenstette, Karl Moritz Her- mann, Tomáš Kočiskỳ, and Phil Blunsom. 2016. Rea- soning about entailment with neural attention. In In- ternational Conference on Learning Representations.
- [29]29. Sebastian Thrun. 1995. Extracting rules from artifi- cial neural networks with distributed representations. In Advances in neural information processing systems (NIPS).
Cited by
Items 301–400 of 900
Belinkov, Y. (2018) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 34
EDUCE: Explaining model Decisions through Unsupervised Concepts ExtractionPreprint
Bouchacourt, D., & Denoyer, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11852 · Cited by 25
“Such methods are specific to text data select and rationales, i.e. parts of text, on which the model bases its prediction, see [15, 27] and very recently [5].”
MethodNourani, M., Roy, C., Rahman, T., Ragan, E. D., Ruozzi, N., & Gogate, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.02335 · Cited by 29
“To date, a major focus of XAI research has been on introducing novel explanation types and then developing and evaluating models and algorithms for these new types [27, 48, 67].”
BackgroundDo Large Language Models Perform the Way People Expect? Measuring the Human Generalization FunctionPreprint
Vafa, K., Rambachan, A., & Mullainathan, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.01382 · Cited by 31
“A common method is to form instance-level explanations, e.g. in the form of rationales (Lei et al., 2016; Chen et al., 2018; Yoon et al., 2018; Bastings et al., 2019; Jain et al., 2020).”
MethodExplainability in Practice: A Survey of Explainable NLP Across Various DomainsPreprint
Mohammadi, H., Bagheri, R. A., Giachanou, A., & Oberski, D. L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.00837 · Cited by 35
“Techniques like LIME, Grad-CAM, or SHAP highlight the parts of the input most influential for a model’s prediction [16, 154], and information-theoretic variants select the smallest set of tokens sufficient to preserve the prediction [233].”
RankingSHAP - Faithful Listwise Feature Attribution Explanations for Ranking Models
Heuss, M., de Rijke, M., & Anand, A. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3729971 · Cited by 27
Tangible Explainable AI - an Initial Conceptual Framework
Colley, A., Väänänen, K., & Häkkilä, J. (2022) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3568444.3568456 · Cited by 27
“users providing feedback on the correctness of individual predictions [26] or on the model as a whole [11], the performance of the AI system can be improved [25].”
BackgroundInvestigating the Benefits of Free-Form Rationales
Sun, J., Swayamdipta, S., May, J., & Ma, X. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.432 · Cited by 29
Diversified Multiple Instance Learning for Document-Level Multi-Aspect Sentiment Classification
Ji, Y., Liu, H., He, B., Xiao, X., Wu, H. R., & Yan-hua, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.570 · Cited by 30
“Document-level multi-aspect sentiment classifi-cation In previous studies, DMSC is usually done by supervised learning methods (Lei et al., 2016; Yin et al., 2017; Li et al., 2018; Wang et al., 2019), where aspect-level annotations should be provided.”
MethodExplainable Clinical Decision Support from Text
Feng, J., Shaib, C., & Rudzicz, F. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.115 · Cited by 37
“Lei et al. (2016) published a subset of 90k reviews selected to minimize correlation between appearance and other aspects.”
BackgroundMethodAutomating autism assessment: What AI can bring to the diagnostic process
Erden, Y. J., Hummerstone, H., & Rainey, S. (2020) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13527 · Cited by 31
“23 Masking could involve making eye contact, imitating others' behaviour/gestures, and following social scripts to hide social difficulties that the individual experiences – as Lei et al 24 observe, ‘ If the diagnostician further misses signs of camouflaging, superficially “ typical ” non-verbal skills and social mann…”
BackgroundInfluentialSaliency Learning: Teaching the Model Where to Pay AttentionPreprint
Ghaeini, R., Fern, X. Z., Shahbazi, H., & Tadepalli, P. V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.08649 · Cited by 26
“Recent work on explanation and interpretation has introduced a few approaches (Simonyan et al., 2013; Ribeiro et al., 2016; Lei et al., 2016; Li et al., 2016, 2017; Ghaeini et al., 2018b; Ribeiro et al., 2018) for explanation.”
BackgroundMatthews, J. (2020) · AI Magazine · doi:10.1609/aimag.v41i1.5204 · Cited by 22
Look Harder: A Neural Machine Translation Model with Hard Attention
Indurthi, S. R., Chung, I., & Kim, S. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1290 · Cited by 25
“Our approach is also related to the method proposed by Lei et al. (2016) to explain the decision of text classifier.”
MethodSelect and Attend: Towards Controllable Content Selection in Text Generation
Shen, X., Suzuki, J., Inui, K., Su, H., Klakow, D., & Sekine, S. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1054 · Cited by 32
“2Devlin et al. (2019) have shown that excellent performance can be obtained by assuming such conditionally independence given a sufficiently expressive representation of x, though modelling a richer inter-label dependency is for sure beneficial (Lei et al., 2016; Nallapati et al., 2017).”
BackgroundRationalization through Concepts
Antognini, D., & Faltings, B. V. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.68 · Cited by 35
Trusting deep learning natural-language models via local and global explanations
Ventura, F., Greco, S., Apiletti, D., & Cerquitelli, T. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01690-9 · Cited by 29
“The authors in [25] propose to learn how to explain a predictive model jointly with the training of the predictor.”
BackgroundLogic Traps in Evaluating Attribution Scores
Ju, Y., Zhang, Y., Zhao, Y., Jiang, Z., Liu, K., & Zhao, J. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.407 · Cited by 32
“Thus, many works (Lei et al., 2016; Li et al., 2016; Tsang et al., 2020; Kim et al., 2020) perform quantitative evaluation of attribution scores by comparing them with the word-level tags in SST-2.”
Interpretable Neural Architectures for Attributing an Ad’s Performance to its Writing Style
Pryzant, R., Basu, S., & Sone, K. (2018) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w18-5415 · Cited by 28
“Ribeiro et al. (2016), Biran and McKeown (2017), and Lei et al. (2016) also use “importance scores” to explain the predictions of neural network-based clas-sifiers.”
BackgroundE3: Entailment-driven Extracting and Editing for Conversational Machine Reading
Zhong, V., & Zettlemoyer, L. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1223 · Cited by 22
“These extractions from text are subsequently used for inference over a knowledge base (Bordes et al., 2013; Dettmers et al., 2018; Lin et al., 2018) and rationalizing model predictions (Lei et al., 2016).”
MethodA Variational Approach to Weakly Supervised Document-Level Multi-Aspect Sentiment Classification
Zeng, Z., Zhou, W., Liu, X., & Song, Y. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1036 · Cited by 23
“We evaluate our model on TripAdvisor (Wang et al., 2010) and BeerAdvocate (McAuley et al., 2012; Lei et al., 2016; Yin et al., 2017) datasets, which contain seven aspects (value, room, location, cleanliness, check in/front desk, service, and business) and four aspects (feel, look, smell, and taste)…”
Raimondi, D., Passemiers, A., Fariselli, P., & Moreau, Y. (2021) · BMC Biology · doi:10.1186/s12915-020-00930-0 · Cited by 26
“Methods such as linear models, Random Forests [41], and Neural Networks [42, 43] are nowadays interpretable to a variable extent, and also model-independent approaches [18, 44] exist.”
BackgroundLearning from the Best: Rationalizing Predictions by Adversarial Information Calibration
Sha, L., Camburu, O.-M., & Lukasiewicz, T. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17623 · Cited by 35
Hierarchical Interpretation of Neural Text Classification
Yan, H., Gui, L., & He, Y. (2022) · Computational Linguistics · doi:10.1162/coli_a_00459 · Cited by 29
“It has been widely used in evaluating rationale extraction models (Bastings, Aziz, and Titov 2019; Lei, Barzilay, and Jaakkola 2016; Li and Eisner 2019; Yu et al. 2021) by calculating the agreement between the annotated sentence-level rationales and model identified text spans.”
MethodAccurate and Explainable Recommendation via Review Rationalization
Pan, S., Li, D., Gu, H., Lu, T., Luo, X., & Gu, N. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512029 · Cited by 29
“[20] first proposed a generator-predictor framework for rationalization, following which several recent works [6, 9, 10, 45] applied different techniques to deal with the challenges in generating high-quality rationales.”
BackgroundMethodA Fine-grained Interpretability Evaluation Benchmark for Neural NLP
Wang, L., Shen, Y., Peng, S., Zhang, S., Xiao, X., Liu, H., Tang, H., Chen, Y., Wu, H., & Wang, H. (2022) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2022.conll-1.6 · Cited by 30
Cognitive computing for coalition situational understanding
Preece, A. D., Cerutti, F., Braines, D., Chakraborty, S., & Srivastava, M. B. (2017) · IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) · doi:10.1109/uic-atc.2017.8397426 · Cited by 21
“, [21]), it is an open question as to how to integrate these.”
BackgroundChalkidis, I., Fergadiotis, M., Tsarapatsanis, D., Αλέτρας, Ν., Androutsopoulos, I., & Malakasiotis, P. (2021) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · doi:10.48550/arxiv.2103.13084 · Cited by 34
“In their work on word-level rationales, Lei et al. (2016) also required the selected words to be contiguous , to obtain more coherent rationales.”
BackgroundMethodResultInfluentialSPECTRA: Sparse Structured Text Rationalization
Guerreiro, N. M., & Martins, A. F. T. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.525 · Cited by 35
“Similarly to Jain et al. (2020), we found that the stochastic rationalizers of Lei et al. (2016) and its variants (SFE, SFE w/ Baseline and Gumbel) require cumbersome hyperparameter search and tend to degenerate in such a way that the generated rationales are either the whole input text or empty…”
BackgroundMethodResultInfluentialProtoTEx: Explaining Model Decisions with Prototype Tensors
Das, A., Gupta, C., Kovatchev, V., Lease, M., & Li, J. J. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.213 · Cited by 35
Towards Safe Artificial General Intelligence
Everitt, T. (2018) · ANU Open Research (Australian National University) · doi:10.25911/5d134a2f8a7d3 · Cited by 39
“Another line of work tries to explain what text and speech networks have learned (Alvarez-Melis and Jaakkola, 2017; Belinkov and Glass, 2017; Lei et al., 2016).”
BackgroundGonzález, A. V., Bansal, G., Fan, A., Mehdad, Y., Jia, R., & Iyer, S. V. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.95 · Cited by 20
“For example, rationales can explain a text classi-fier using phrases in the input text that are relevant to the prediction (Lei et al., 2016).”
BackgroundSelf-training with Few-shot Rationalization
Bhat, M. M., Sordoni, A., & Mukherjee, S. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.836 · Cited by 21
“We build over insights from prior work (Yu et al., 2019; Lei et al., 2016) focusing on low-resource settings with access to limited labels via multi-task self-training.”
BackgroundMethodWhat to Learn, and How: Toward Effective Learning from Rationales
Carton, S., Kanoria, S., & Tan, C. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.86 · Cited by 39
Cognition-aware Knowledge Graph Reasoning for Explainable Recommendation
Bing, Q., Zhu, Q., & Dou, Z. (2023) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3539597.3570391 · Cited by 23
“Previous work indicates that concise explanations with high quality are sufficient to justify recommendation results and are preferable for users [22, 38, 45], hence we limit the maximum length of reasoning paths to 3.”
BackgroundData Augmentation for Improving Explainability of Hate Speech Detection
Ansari, G., Kaur, P., & Saxena, C. (2023) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-023-08100-4 · Cited by 21
“The work in [28] is an encoder generator framework to provide rationales to outputs.”
Read it Twice: Towards Faithfully Interpretable Fact Verification by Revisiting Evidence
Hu, X., Hong, Z., Guo, Z., Wen, L., & Yu, P. S. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3592049 · Cited by 44
“We adopt (5) Reinforcementbased Method [16].”
InfluentialImproving Moderation of Online Discussions via Interpretable Neural Models
Švec, A., Pikuliak, M., Šimko, M., & Bieliková, M. (2018) · Workshop on Abusive Language Online (ALW) · doi:10.18653/v1/w18-5108 · Cited by 20
“Our work uses the method by Lei et al. (2016), which selects coherent subset of words responsible for neural network decision.”
MethodControl and Ownership of Neuroprosthetic Speech
Maslen, H., & Rainey, S. (2020) · Philosophy & Technology · doi:10.1007/s13347-019-00389-0 · Cited by 23
“The statistical language models and algorithmic prediction employed in the processing to some extent transform the information to impose meaning where it is lacking, or correct where indeterminate or highly improbable (Bocquelet et al. 2016b; Lei et al. 2016; Song and Sepulveda 2014).”
MethodLooking in the Right Place for Anomalies: Explainable Ai Through Automatic Location Learning
Kashyap, S., Karargyris, A., Wu, J. T.-Y., Gur, Y., Sharma, A., Wong, K. C. L., Moradi, M., & Syeda-Mahmood, T. F. (2020) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi45749.2020.9098370 · Cited by 22
“The relevancy of regions is also built into the object detection formalisms in computer vision where current networks such as RetinaNet solve for both the object category and its predicted image location [4].”
MethodZhang, H., & Litman, D. J. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.759 · Cited by 20
“In the NLP area, Lei et al. (2016) proposed to use a generator to find candidate rationale and these are passed through the encoder for prediction.”
MethodText Classification for Azerbaijani Language Using Machine Learning
Suleymanov, U., Kalejahi, B. K., Amrahov, E., & Badirkhanli, R. (2020) · Computer Systems Science and Engineering · doi:10.32604/csse.2020.35.467 · Cited by 20
Translation Error Detection as Rationale Extraction
Fomicheva, M., Specia, L., & Αλέτρας, Ν. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.327 · Cited by 25
Shickel, B., & Rashidi, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.12524 · Cited by 28
“In an effort to better explain the prediction of multi-aspect sentiment analysis and retrieval of semantically similar questions, Lei et al. 63 augmented their RCNN model 64 with a jointly trained probabilistic generator network that learned a binary vector over input words most influential for a…”
MethodImplicit MLE: Backpropagating Through Discrete Exponential Family DistributionsPreprint
Niepert, M., Minervini, P., & Franceschi, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.01798 · Cited by 79
“We use pre-trained word embeddings from Lei et al. [2016].”
MethodUnderstanding Interlocking Dynamics of Cooperative RationalizationPreprint
Yu, M., Zhang, Y., Chang, S., & Jaakkola, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.13880 · Cited by 29
“We follow the standard setting [6, 27] to use the sparsity and continuity constraints to regularize the selected rationales for all methods.”
BackgroundMethodResultInfluentialSelf-citationDiagnosing AI Explanation Methods with Folk Concepts of Behavior
Jacovi, A., Bastings, J., Gehrmann, S., Goldberg, Y., & Filippova, K. (2023) · Journal of Artificial Intelligence Research · doi:10.1145/3593013.3593993 · Cited by 30
Tailoring Self-Rationalizers with Multi-Reward Distillation
Ramnath, S., Joshi, B., Hallinan, S., Lu, X., Li, L. H., Chan, A., Hessel, J., Choi, Y., & Ren, X. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2311.02805 · Cited by 23
multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoning
Saha, S., Yadav, P., & Bansal, M. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.287 · Cited by 31
Bagel: A Benchmark for Assessing Graph Neural Network Explanations
Rathee, M., Funke, T., Anand, A., & Khosla, M. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-25305-7_3 · Cited by 21
MGR: Multi-generator Based Rationalization
Liu, W., Wang, H., Wang, J., Li, R., Li, X., Zhang, Y., & Qiu, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.715 · Cited by 31
Explainable AI: A Neurally-Inspired Decision Stack Framework
Khan, M. S., Nayebpour, M., Li, M., El‐Amine, H., Koizumi, N., & Olds, J. L. (2022) · Biomimetics · doi:10.3390/biomimetics7030127 · Cited by 21
“In rationale methods, pieces of input texts are extracted to determine a possible justification (Rationale) for prediction [62,63].”
BackgroundMethodInterpretable Visual Reasoning via Induced Symbolic Space
Wang, Z., Wang, K., Yu, M., Xiong, J., Hwu, W., Hasegawa‐Johnson, M., & Shi, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00189 · Cited by 26
“Our work is closely related to the self-explaining systems via rationalization [31, 7, 48].”
BackgroundDistilling the Evidence to Augment Fact Verification Models
Portelli, B., Zhao, J., Schuster, T., Serra, G., & Santus, E. (2020) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/2020.fever-1.7 · Cited by 19
“This approach was shown to perform better than simply relying on the internal attention weights of a classifier (Lei et al., 2016; Jain and Wallace, 2019).”
MethodEvaluating neural network explanation methods using hybrid documents and morphological agreementPreprint
Poerner, N., Roth, B., & Schütze, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.06422 · Cited by 27
“Note that we do not address self-explanatory models, such as attention-based (Bahdanau et al., 2014) or rationale models (Lei et al., 2016).”
Representation Learning for Electronic Health RecordsPreprint
Weng, W., & Szolovits, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.09248 · Cited by 27
“For example, using an attention mechanism or encoder-generator framework to highlight where is the most consequential input for a model’s prediction (Bahdanau et al., 2015; Lei et al., 2016).”
MethodWhy Is It Hate Speech? Masked Rationale Prediction for Explainable Hate Speech DetectionPreprint
Kim, J., Lee, B., & Sohn, K.-A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.00243 · Cited by 31
“Some Natural Language Processing (NLP) studies define rationale as snippets of an input text on which the model’s prediction is supported (Han-cock et al., 2018; Lei et al., 2016).”
Abductive Commonsense Reasoning Exploiting Mutually Exclusive Explanations
Zhao, W., Chiu, J., Cardie, C., & Rush, A. M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.831 · Cited by 37
A Deep Learning Approaches for Natural Language Processing and Sentiment Analysis in Social Media
Ashiq, V. M., Fredrik, E. J. T., Kumar, N. K., Torres‐Cruz, F., Colque, J. P. B., & Manoharan, G. (2023) · International Conference on Contemporary Computing and Informatics (IC3I) · doi:10.1109/ic3i59117.2023.10398079 · Cited by 18
“To extract text from the input that acts as the reasons for review ratings, Lei recommended utilising a neural network approach [12].”
MethodAn Imitation Learning Approach to Unsupervised Parsing
Li, B., Mou, L., & Keller, F. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1338 · Cited by 18
“Training involves doubly stochastic gradient descent (Lei et al., 2016): the first stochasticity comes from sampling input from the data distribution, and the second one from sampling actions for each input.”
BackgroundExplainable process trace classification: An application to stroke
Leonardi, G., Montani, S., & Striani, M. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2021.103981 · Cited by 18
ExSum: From Local Explanations to Model Understanding
Zhou, Y., Ribeiro, M. T., & Shah, J. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.392 · Cited by 28
Exploring Faithful Rationale for Multi-Hop Fact Verification via Salience-Aware Graph Learning
Si, J., Zhu, Y., & Zhou, D. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26591 · Cited by 40
Learning What's Easy: Fully Differentiable Neural Easy-First Taggers
Martins, A. F. T., & Kreutzer, J. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1036 · Cited by 28
“Finally, the visualization provided in Figures 2– 4 brings up the question how to understand and rationalize predictions by neural network systems, addressed by Lei et al. (2016).”
MethodDeveloping user-centered system design guidelines for explainable AI: a systematic literature review
Hong, S., & Park, W. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11363-y · Cited by 22
“Most XAI research has focused on developing various explanation methods, ranging from traditional approaches like decision trees (Breiman et al. 1984) to modern techniques such as input gradients and sensitivity analysis (Selvaraju et al. 2016; Ribeiro et al. 2016; Lei et al. 2016 ).”
Rational LAMOL: A Rationale-based Lifelong Learning Framework
Kanwatchara, K., Horsuwan, T., Lertvittayakumjorn, P., Kijsirikul, B., & Vateekul, P. (2021) · doi:10.18653/v1/2021.acl-long.229 · Cited by 27
“Existing rationalization techniques commonly use the maximum mutual information (MMI) criterion to select rationales, which is prone to choosing spurious correlation between input features and outputs as rationales (Lei et al., 2016; Yu et al., 2019).”
MethodFaxPlainAC: A Fact-Checking Tool Based on EXPLAINable Models with HumAn Correction in the Loop
Zhang, Z., Rudra, K., & Anand, A. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3481985 · Cited by 17
“Towards this, there have been some recent proposals that create models that are interpretable-by-design approach that generate an explanation alongwith the actual fact-checking prediction [1, 7, 14].”
BackgroundAn Empirical Study on Explanations in Out-of-Domain Settings
Chrysostomou, G., & Αλέτρας, Ν. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.477 · Cited by 30
A Physically Interpretable Rice Field Extraction Model for PolSAR Imagery
Ge, J., Zhang, H., Xu, L., Sun, C., Duan, H., Guo, Z., & Wang, C. (2023) · Remote Sensing · doi:10.3390/rs15040974 · Cited by 17
CREST: A Joint Framework for Rationalization and Counterfactual Text Generation
Treviso, M., Ross, A., Guerreiro, N. M., & Martins, A. F. T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.842 · Cited by 40
Exploring Explainable AI in Large Language Models: Enhancing Transparency and Trust
H., I., Jo, A. A., & Raj, E. D. (2024) · International Conference on Advances in Computing and Communications (ICACC) · doi:10.1109/icacc63692.2024.10845370 · Cited by 13
Ethical Behaviour Aspects of Autonomous Intelligent Cyber-Physical Systems
Trentesaux, D., & Karnouskos, S. (2019) · Studies in computational intelligence · doi:10.1007/978-3-030-27477-1_5 · Cited by 17
Contrastive Relevance Propagation for Interpreting Predictions by a Single-Shot Object Detector
Tsunakawa, H., Kameya, Y., Lee, H., Shinya, Y., & Mitsumoto, N. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8851770 · Cited by 21
“There have been several methods where an interpretable model is embedded into a prediction model [14], [15].”
MethodInterpretable Charge Prediction for Criminal Cases with Dynamic Rationale Attention
Chao, W., Jiang, X., Luo, Z., Hu, Y., & Ma, W. (2019) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11377 · Cited by 17
“Besides, we adopt a reinforcement style neural method (Jiang, Ye, Luo, Chao, & Ma, 2018) which stems from Lei et al. (2016) to extract rationales using the only supervision of charge label.”
BackgroundMethodInfluentialDoT: An efficient Double Transformer for NLP tasks with tables
Krichene, S., Müller, T., & Eisenschlos, J. M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.289 · Cited by 20
Alignment Rationale for Natural Language Inference
Jiang, Z., Zhang, Y., Zhao, Y., Zhao, J., & Liu, K. (2021) · doi:10.18653/v1/2021.acl-long.417 · Cited by 23
“…et al., 2016; Kim et al., 2020), extracting predictive related input pieces, referred as sufficient input subset (Carter et al., 2019) or rationale (Lei et al., 2016; Bastings et al., 2019), building hierarchical explanations (Chen et al., 2020; Zhang et al., 2020), and generating natural language…”
BackgroundInfluentialDeep learning for situational understanding
Chakraborty, S., Preece, A. D., Alzantot, M., Xing, T., Braines, D., & Srivastava, M. B. (2017) · International Conference on Information Fusion (FUSION) · doi:10.23919/icif.2017.8009785 · Cited by 18
“The authors in [29], combine two modular components – a generator and encoder – to operate together and learn candidate rationales for a prediction.”
BackgroundLearning How to Self-Learn: Enhancing Self-Training Using Neural Reinforcement Learning
Chen, C., Zhang, Y., & Gao, Y. (2018) · International Conference on Asian Language Processing · doi:10.1109/ialp.2018.8629107 · Cited by 21
Learning credible DNNs via incorporating prior knowledge and model local explanation
Du, M., Liu, N., Yang, F., & Hu, X. (2020) · Knowledge and Information Systems · doi:10.1007/s10115-020-01517-5 · Cited by 17
“A rationale is a subset of features highlighted by annotators and regarded to be more important in predicting an instance [28,57], with illustrative examples shown in Fig.”
BackgroundThe Irrationality of Neural Rationale Models
Zheng, Y., Booth, S., Shah, J., & Zhou, Y. (2022) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2022.trustnlp-1.6 · Cited by 23
BEExAI: Benchmark to Evaluate Explainable AI
Sithakoul, S., Meftah, S., & Feutry, C. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-63787-2_23 · Cited by 26
Comparative Study of XAI Perception Between Eastern and Western Cultures
Chien, S.-Y. J., Wang, Y.-F., Cheng, K., & Chen, Y. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2441015 · Cited by 20
“Prior research also emphasizes the importance of comprehensive explanations, which not only enhance system transparency and strengthen the usefulness of recommendations but also allow users to develop trust and improve their decision-making processes (Adadi & Berrada, 2018; Lei et al., 2016).”
Rationale-Inspired Natural Language Explanations with Commonsense.Preprint
Majumder, B. P., Camburu, O.-M., Lukasiewicz, T., & McAuley, J. (2021) · arXiv (Cornell University) · Cited by 20
Group Cognition and Collaborative AI
Koch, J., & Oulasvirta, A. (2018) · Human–Computer Interaction Series · doi:10.1007/978-3-319-90403-0_15 · Cited by 15
A Multi-Criteria Decision-Making Approach for the Selection of Explainable AI Methods
Matejová, M., & Paralič, J. (2025) · Machine Learning and Knowledge Extraction · doi:10.3390/make7040158 · Cited by 17
Machine Learning Verification and Safety for Unmanned Aircraft - A Literature Study
Torens, C., Jünger, F., Schirmer, S., Schopferer, S., Maienschein, T., & Dauer, J. C. (2022) · AIAA SciTech Forum · doi:10.2514/6.2022-1133 · Cited by 20
Huo, Y., Wong, D. F., Ni, L. M.-S., Chao, L. S., & Zhang, J. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106389 · Cited by 17
Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks
Chen, H., Feng, S., Ganhotra, J., Wan, H., Gunasekara, C., Joshi, S., & Ji, Y. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.306 · Cited by 20
Structurizing Misinformation Stories via Rationalizing Fact-Checks
Jiang, S., & Wilson, C. (2021) · doi:10.18653/v1/2021.acl-long.51 · Cited by 19
Han, Y., Chen, C., Tang, L., Lin, M., Jaiswal, A. K., Wang, S., Tewfik, A. H., Shih, G., Ding, Y., & Peng, Y. (2020) · PubMed · doi:10.48550/arxiv.2011.12506 · Cited by 18
“Therefore, it provides a way to introduce prior information about anticipated explanations, a technique that is widely used in the “Rationale model” 19 (Section 2).”
Diagnostics-Guided Explanation Generation
Atanasova, P., Simonsen, J. G., Lioma, C., & Augenstein, I. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21287 · Cited by 35
Connecting Attributions and QA Model Behavior on Realistic Counterfactuals
Ye, X., Nair, R., & Durrett, G. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.447 · Cited by 25
“Past work has evaluated along plausibility criteria (Lei et al., 2016; Strout et al., 2019; Thorne et al., 2019), but as we see from this example, faithful explanations (Subramanian et al.”
BackgroundGenerating knowledge aware explanation for natural language inference
Yang, Z., Xu, Y., Hu, J., & Dong, S. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103245 · Cited by 22
ExASAG: Explainable Framework for Automatic Short Answer Grading
Tornqvist, M., Mahamud, M., Guzman, E. M., & Farazouli, A. (2023) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/2023.bea-1.29 · Cited by 29
“Such XAI methods can usually be divided into feature importance-based explanations (Si-monyan et al., 2013), perturbation-based explanations (Zeiler and Fergus, 2014), explanations by simplification (Ribeiro et al., 2016) and language explanations (Lei et al., 2016).”
BackgroundMethodInfluentialStrategies to Exploit XAI to Improve Classification Systems
Apicella, A., Lorenzo, L. D., Isgrò, F., Pollastro, A., & Prevete, R. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44064-9_9 · Cited by 15
ESPRIT: Explaining Solutions to Physical Reasoning Tasks
Rajani, N. F., Zhang, R., Tan, Y. C., Zheng, S., Weiss, J., Vyas, A., Gupta, A., Xiong, C., Socher, R., & Radev, D. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.706 · Cited by 26
Towards Explainable Fact CheckingPreprint
Augenstein, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.10274 · Cited by 17
SelfExplain: A Self-Explaining Architecture for Neural Text ClassifiersPreprint
Rajagopal, D., Balachandran, V., Hovy, E. H., & Tsvetkov, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.12279 · Cited by 29
“Another recent line of work explores collecting rationales (Lei et al., 2016) through expert annotations (Zaidan and Eisner, 2008).”
BackgroundMethodImproving the Explainability of Neural Sentiment Classifiers via Data AugmentationPreprint
Chen, H., & Ji, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.04225 · Cited by 18
“The most common way of generating local explanations in sentiment classification is identifying the important part of a text associated with predicted sentiment polarity (Lei et al., 2016; Chen et al., 2018; Nguyen, 2018).”
BackgroundInfluentialRationales for Sequential Predictions
Vafa, K., Deng, Y., Blei, D., & Rush, A. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.807 · Cited by 14
“Various rationale methods have been proposed for sequence classification, where each sequence has a single rationale (Lei et al., 2016; Chen et al., 2018; Jain et al., 2020).”
BackgroundMethod
Metadata
- Authors
- Lei, T., Barzilay, R., & Jaakkola, T.
- Affiliations
- MIT Computer Science and Artificial Intelligence Laboratory (US)
- Venue
- Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing
- Published
- 2016
- Pages
- 107-117
- OpenAlex
- W2440159969
- Cited by
- 777
- References
- 27
- Citations by year
- 2026: 27 · 2025: 33 · 2024: 39 · 2023: 88 · 2022: 92 · 2021: 138 · 2020: 126 · 2019: 101 · 2018: 78 · 2017: 39 · 2016: 4
- FWCI
- 39.87
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- explanation generation, question retrieval, model regularization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.