Rationalizing Neural Predictions
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Prediction without justification has limited applicability.As a remedy, we learn to extract pieces of input text as justifications -rationales -that are tailored to be short and coherent, yet sufficient for making the same prediction.Our approach combines two modular components, generator and encoder, which are trained to operate well together.The generator specifies a distribution over text fragments as candidate rationales and these are passed through the encoder for prediction.Rationales are never given during training.Instead, the model is regularized by desiderata for rationales.We evaluate the approach on multi-aspect sentiment analysis against manually annotated test cases.Our approach outperforms attention-based baseline by a significant margin.We also successfully illustrate the method on the question retrieval task. 1
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“A lot of work has also been done on argumentation and dialog games [6, 11, 51, 53] but the focus in these areas is generally the logical structure of the framework to express and to relate arguments or the protocol to exchange arguments.”
From Beyond explainability: justifiability and contestability of algorithmic decision systems (2022) · cites this paper as [53]
Experimental — accuracy not yet measuredIn the citing paper
“This model can achieve competitive accuracy while having built-in interpretability because the prediction is made exclusively based on rationales.”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [52]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 2×
“Attention based models offer another lens to the inner workings of neural models (Bahdanau et al., 2015; Cheng et al., 2016; Martins and Astudillo, 2016; Chen et al., 2015; Xu and Saenko, 2015; Yang et al., 2015).”
BackgroundConvolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2016) · arXiv · doi:10.48550/arxiv.1607.01759 · Cited by 4,542
A Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1062 · Cited by 3,601 · cited 1×
“For instance, we could use a convolutional network (Kim, 2014; Kalchbrenner et al., 2014), deep averaging network (Iyyer et al.”
MethodA Neural Attention Model for Abstractive Sentence Summarization
Rush, A. M., Chopra, S., & Weston, J. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1044 · Cited by 2,656
Recurrent Models of Visual Attention
Mnih, V., Heess, N., Graves, A., & Kavukcuoglu, K. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.6247 · Cited by 2,568 · cited 1×
“space of possible rationales in terms of their interaction with the encoder. We could also apply variance reduction techniques to increase stability of stochastic training (cf. (Weaver and Tao, 2001; Mnih et al., 2014; Ba et al., 2015; Xu et al., 2015)). 7 Acknowledgments We thank Prof. Julian McAuley for sharing the…”
MethodTeaching Machines to Read and Comprehend
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,521
Research works printed but not linked: 6
- [6]6. Cicero dos Santos, Luciano Barbosa, Dasha Bogdanova, and Bianca Zadrozny. 2015. Learning hybrid rep- resentations to retrieve semantically equivalent ques- tions. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Lan- guage Processing (Volume 2: Short Papers), pages 694-699, Beijing, China, July. Association for Com- putational Linguistics.
- [7]7. Manaal Faruqui, Jesse Dodge, Sujay K. Jauhar, Chris Dyer, Eduard Hovy, and Noah A. Smith. 2015a. Retrofitting word vectors to semantic lexicons. In Pro- ceedings of NAACL.
- [9]9. Aurélie Herbelot and Eva Maria Vecchi. 2015. Build- ing a shared world: mapping distributional to model- theoretic semantic spaces. In Proceedings of the 2015 Conference on Empirical Methods in Natural Lan- guage Processing. Association for Computational Lin- guistics.
- [12]12. Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III. 2015. Deep unordered compo- sition rivals syntactic methods for text classification. In Proceedings of the 53rd Annual Meeting of the As- sociation for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers).
- [27]27. Tim Rocktäschel, Edward Grefenstette, Karl Moritz Her- mann, Tomáš Kočiskỳ, and Phil Blunsom. 2016. Rea- soning about entailment with neural attention. In In- ternational Conference on Learning Representations.
- [29]29. Sebastian Thrun. 1995. Extracting rules from artifi- cial neural networks with distributed representations. In Advances in neural information processing systems (NIPS).
Cited by
Items 401–500 of 900
A framework to contest and justify algorithmic decisions
Henin, C., & Le Métayer, D. (2021) · AI and Ethics · doi:10.1007/s43681-021-00054-3 · Cited by 14
“In some papers [20], justifications are seen as ways to make understandable the inner operations of a complex system, which would be called white-box explanations according to our definitions.”
BackgroundLS-Tree: Model Interpretation When the Data Are Linguistic
Chen, J., & Jordan, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5749 · Cited by 14
Interventional Rationalization
Yue, L., Liu, Q., Wang, L., An, Y., Du, Y., & Huang, Z. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.700 · Cited by 14
“Among them, the selective rationalization task (Lei et al., 2016; Yu et al., 2019, 2021) has received increasing attention, answering the question “What feature has a significant impact on the prediction results of the model?”.”
InfluentialELAD: Explanation-Guided Large Language Models Active Distillation
Zhang, Y., Pan, B., Chen, L., Hu, Y., & Zhao, L. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.264 · Cited by 23
“Extractive explanations (Lei et al., 2016; Yu et al., 2019) focus on identifying key elements within the input that justify a prediction.”
Yeo, W. J., Satapathy, R., & Cambria, E. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.529 · Cited by 30
Distribution Matching for Rationalization
Huang, Y., Chen, Y., Du, Y., & Yang, Z. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i14.17547 · Cited by 26
“Our above derivation is based on the original rationalization framework (Lei, Barzilay, and Jaakkola 2016).”
BackgroundMethodResultInfluentialThe Role of Deep Learning in Improving Healthcare
Thaler, S., & Menkovski, V. (2019) · Springer eBooks · doi:10.1007/978-3-030-05249-2_3 · Cited by 14
Luo, G. (2019) · Global Transitions · doi:10.1016/j.glt.2018.11.001 · Cited by 14
“For an LSTM network built on text data, researchers have automatically explained its classification result on a text sequence by showing which words [145], pieces of text [146], or phrases [147] in the sequence are responsible for the classification result.”
Relation Extraction with Explanation
Shahbazi, H., Fern, X. Z., Ghaeini, R., & Tadepalli, P. V. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.579 · Cited by 15
Measuring and Improving Faithfulness of Attention in Neural Machine Translation
Moradi, P., Kambhatla, N., & Sarkar, A. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.243 · Cited by 22
“In another work, (Lei et al., 2016; Bastings et al., 2019) propose models in which first a rationale is selected from the input and then is further used for prediction.”
Flexible Instance-Specific Rationalization of NLP Models
Chrysostomou, G., & Αλέτρας, Ν. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21298 · Cited by 24
“The first consists of two modules jointly trained on an end-task, e.g. sentiment analysis (Lei, Barzilay, and Jaakkola 2016; Bastings, Aziz, and Titov 2019).”
MethodGenerating Token-Level Explanations for Natural Language InferencePreprint
Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2019) · arXiv (Cornell University) · doi:10.17863/cam.53800 · Cited by 17
Learning Fair Rule ListsPreprint
Aïvodji, U., Ferry, J., Gambs, S., Huguet, M., & Siala, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.03977 · Cited by 18
Weakly- and Semi-supervised Evidence Extraction
Pruthi, D., Dhingra, B., Neubig, G., & Lipton, Z. C. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.353 · Cited by 13
Multivariate Time Series Analysis: An Interpretable CNN-based Model
Younis, R., Zerr, S., & Ahmadi, Z. (2022) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa54385.2022.10032335 · Cited by 19
“The literature generally divides the proposed methods into two categories: The first category comprises the self-interpretable models, in which the explanation of a decision is integrated inside the architecture of the deep learning model [12].”
MethodGraphNarrator: Generating Textual Explanations for Graph Neural Networks
Pan, B., Xiong, Z., Wu, G., Zhang, Z., Zhang, Y., Hu, Y., & Zhao, L. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.2 · Cited by 26
“The traditional explanation methods for NLP, such as feature importance and saliency maps (Lei et al., 2016; Yu et al., 2019), often fall short in providing human-interpretable insights, motivating the development of more intuitive approaches like nat-ural language explanations (Cambria et al.,…”
Unifying Model Explainability and Robustness for Joint Text Classification and Rationale Extraction
Li, D., Hu, B., Chen, Q., Xu, T., Tao, J., & Zhang, Y. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21342 · Cited by 24
On the Impact of Temporal Concept Drift on Model Explanations
Zhao, Z., Chrysostomou, G., Bontcheva, K., & Αλέτρας, Ν. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.298 · Cited by 27
Semantic Explanations of PredictionsPreprint
Lécué, F., & Wu, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.10587 · Cited by 13
“(Lei, Barzilay, and Jaakkola 2016) study similar models and aim at identifying candidate rationales i.e., core elements of the model which aims at generalizing any prediction.”
BackgroundFinding Generalizable Evidence by Learning to Convince Q&A ModelsPreprint
Perez, E., Karamcheti, S., Fergus, R., Weston, J., Kiela, D., & Cho, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.05863 · Cited by 15
“Inspired by work in interpreting neural networks (Lei et al., 2016), we have agents find a subset of the passage (i.”
MethodFIND: Human-in-the-Loop Debugging Deep Text ClassifiersPreprint
Lertvittayakumjorn, P., Specia, L., & Toni, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.04987 · Cited by 16
“, 2018), extracted rationales (Lei et al., 2016), and attribution scores”
BackgroundMethodLeveraging Structured Information for Explainable Multi-hop Question Answering and Reasoning
Li, R., & Du, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.452 · Cited by 12
Das, A. R., Pillai, N., Nanduri, B., Rothrock, M. J., & Ramkumar, M. (2024) · Microorganisms · doi:10.3390/microorganisms12071274 · Cited by 12
“While exploring attention weights in transformer models to demystify their “black-box” nature is not entirely new [12–14], we recognize the substantial potential of these weights.”
BackgroundExploring Distantly-Labeled Rationales in Neural Network Models
Huang, Q., Zhu, S., Feng, Y., & Zhao, D. (2021) · doi:10.18653/v1/2021.acl-long.433 · Cited by 12
“Lei et al. (2016) use reinforcement learning to identify keywords as rationales to improve model interpretability.”
MethodLegal Judgment Elements Extraction Approach with Law Article-aware Mechanism
Hu, Z., Pan, B., & Li, R. (2021) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3485244 · Cited by 12
“Lei improved the interpretability of the text classification model by selecting the most supported fragments from the original texts [18].”
BackgroundWeiss, G., Goldberg, Y., & Yahav, E. (2022) · Machine Learning · doi:10.1007/s10994-022-06163-2 · Cited by 22
“…et al., 1993; Omlin and Giles, 1996; Cechin et al., 2003; Jacobsson, 2005; Karpathy et al., 2015; Li et al., 2015; Linzen et al., 2016; Strobelt et al., 2016; Lei et al., 2016; K´ad´ar et al., 2016; Shi et al., 2016; Adi et al., 2016; Murdoch and Szlam, 2017; Wang et al., 2017; Arras et al., 2017).”
Decoupled Rationalization with Asymmetric Learning Rates: A Flexible Lipschitz Restraint
Liu, W., Wang, J., Wang, H., Li, R., Qiu, Y., Zhang, Y., Han, J., & Zou, Y. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599299 · Cited by 12
“[23] rst proposed the rationalization framework RNP in which a generator selects human-intelligible subsets (i.”
BackgroundInfluentialAdaptive Perturbation-Based Gradient Estimation for Discrete Latent Variable Models
Minervini, P., Franceschi, L., & Niepert, M. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i8.26103 · Cited by 29
Saliency Learning: Teaching the Model Where to Pay Attention
Ghaeini, R., Fern, X. Z., Shahbazi, H., & Tadepalli, P. V. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1404 · Cited by 29
“Recent work on explanation and interpretation has introduced a few approaches (Simonyan et al., 2013; Ribeiro et al., 2016; Lei et al., 2016; Li et al., 2016, 2017; Ghaeini et al., 2018b; Ribeiro et al., 2018) for explanation.”
BackgroundExplaining Text Matching on Neural Natural Language Inference
Kim, Y., Jang, M., & Allan, J. (2020) · ACM Transactions on Information Systems · doi:10.1145/3418052 · Cited by 12
“Existing work in other text classification tasks have been accompanied with a token-level explanation to explain models’ predictions [4, 27, 28].”
BackgroundMethodExplaining the Suspicion: Design of an XAI-Based User-Focused Anti-Phishing Measure
Kluge, K., & Eckhardt, R. (2021) · Lecture notes in information systems and organisation · doi:10.1007/978-3-030-86797-3_17 · Cited by 14
Interpretable Sentiment Analysis based on Deep Learning: An overview
Jawale, S. S., & Sawarkar, S. D. (2020) · IEEE Pune Section International Conference (PuneCon) · doi:10.1109/punecon50868.2020.9362361 · Cited by 13
“Lei et. al [35] extracted input text as fair rationales that attune to be short and logical, for making the prediction.”
BackgroundMethodHierarchical Question Answering for Long DocumentsPreprint
Choi, E., Hewlett, D., Lacoste, A., Polosukhin, I., Uszkoreit, J., & Berant, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01839 · Cited by 13
“Treating sentence selection as a latent variable has been explored in classifi-cation (Yessenalina et al., 2010; Lei et al., 2016), however, to our knowledge, has not been applied for question answering.”
BackgroundInterpreting Neural Networks Using Flip PointsPreprint
Yousefzadeh, R., & O’Leary, D. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.08789 · Cited by 12
“In the context of text analysis, Lei et al. (2016) has introduced a model that first specifies distributions over text fragments as candidate rationales and then uses the rationales to make predictions.”
MethodPair the Dots: Jointly Examining Training History and Test Stimuli for Model InterpretabilityPreprint
Meng, Y., Fan, C. C., Sun, Z., Hovy, E. H., Wu, F., & Li, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.06943 · Cited by 13
“…natural language inference In the context of NLP, methods for model interpretation include generating justifications to form interpretable summaries (Lei et al., 2016), studying the roles of vector dimensions (Shi et al., 2016), and analyzing hidden state dynamics in recurrent neural networks…”
MethodRogha, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.11507 · Cited by 12
Human-annotated rationales and explainable text classification: a survey
Herrewijnen, E., Nguyen, D., Bex, F., & van Deemter, K. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1260952 · Cited by 22
“The annotator model can be trained separately from the ante-hoc explainer model (Yessenalina et al., 2010; Jain et al., 2020) or the models can be trained jointly (Lei et al., 2016; Bastings et al., 2019).”
BackgroundMethodInfluentialGao, Y., Zhu, Y., & Yu, Q. (2022) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3524842.3528472 · Cited by 12
Weakly Supervised Attention Networks for Fine-Grained Opinion Mining and Public Health
Karamanolakis, G., Hsu, D., & Gravano, L. (2019) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/d19-5501 · Cited by 11
“Indeed, choosing segments of the review text that explain the review-level decisions can help interpretability (Lei et al., 2016; Yessenalina et al., 2010; Biran and Cotton, 2017).”
BackgroundNeuro-symbolic Natural Logic with Introspective Revision for Natural Language Inference
Feng, Y., Yang, X., Zhu, X., & Greenspan, M. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00458 · Cited by 20
“The summary statistics in Table 8 shows that our model matches Lei et al. (2016) on the IOU score, and that it produces rationales with significantly higher precision and F1-scores on the e-SNLI test set.”
MethodResultRationalizing predictions by adversarial information calibration
Sha, L., Camburu, O.-M., & Lukasiewicz, T. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2022.103828 · Cited by 14
“Regularizing Rationales with Language Models For NLP tasks, it is often desirable that a rationale is formed of fluent subphrases [75].”
BackgroundMethodResultInfluentialMahoney, C. J., Gronvall, P., Huber-Fliflet, N., & Zhang, J. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020626 · Cited by 11
“Lei et al. [9] proposed a neural network approach to generate rationales for text classification.”
MethodLocal Interpretation of Transformer Based on Linear Decomposition
Yang, S., Huang, S., Zou, W., Zhang, J., Dai, X., & Chen, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.572 · Cited by 22
“Most existing works on local explanation algorithms in NLP strive to understand such dynamics on word-level or phrase-level by assigning importance scores on input features (Ribeiro et al., 2016a; Lei et al., 2016; Lund-berg et al., 2018; Plumb et al., 2018).”
BackgroundControlling Bias Exposure for Fair Interpretable Predictions
He, Z., Wang, Y., McAuley, J., & Majumder, B. P. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.431 · Cited by 22
Teaching Meaningful ExplanationsPreprint
Codella, N., Hind, M., Ramamurthy, K. N., Campbell, M. S., Dhurandhar, A., Varshney, K. R., Wei, D., & Mojsilović, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.11648 · Cited by 12
Deep Learning in Information SecurityPreprint
Thaler, S., Menkovski, V., & Petković, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.04332 · Cited by 12
“worked on explaining predictions of classifiers [80; 116] by extending the model with parts that would highlight the inputs that were responsible for causing the result.”
BackgroundHuman-grounded Evaluations of Explanation Methods for Text ClassificationPreprint
Lertvittayakumjorn, P., & Toni, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.11355 · Cited by 17
“Besides, there are other explanation methods designed for specific deep learning architectures such as attention mechanism (Ghaeini et al., 2018) and extractive rationale generation (Lei et al., 2016).”
MethodImproving Robustness and Generality of NLP Models Using Disentangled RepresentationsPreprint
Wu, J., Li, X., Ao, X., Meng, Y., Wu, F. Q., & Li, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.09587 · Cited by 14
“Due to the lack of interpretability (Li et al., 2015; Linzen et al., 2016; Lei et al., 2016; Koh and Liang, 2017) of neural models, it is impossible to disentangle them.”
BackgroundAre Training Resources Insufficient? Predict First Then Explain!Preprint
Jang, M., & Lukasiewicz, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.02056 · Cited by 14
“In natural language processing (NLP), extracting rationales , which is a short but sufficient part of an input to make a decision on its own, has been widely studied to provide interpretability (Lei et al., 2016; Bastings et al., 2019; Yu et al., 2019; Jain et al., 2020; Paranjape et al., 2020).”
Toward a traceable, explainable, and fairJD/Resume recommendation systemPreprint
Barrak, A., Adams, B., & Zouaq, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.08960 · Cited by 12
“Similarly, Rationalizing Neural Predictions is trying to predict what part of a paragraph a model is used to make a decision [188].”
BackgroundEx-Twit: Explainable Twitter Mining on Health DataPreprint
Islam, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02132 · Cited by 11
“To justify predictions made by neural networks, [Lei et al., 2016] built modular neural framework to automatically generate concise yet sufficient text fragments.”
BackgroundMethodThe Why Behind the Action: Unveiling Internal Drivers via Agentic AttributionPreprint
Qian, C., Wang, P., Liu, D., Yang, J., Guo, D., Tang, L., Mei, J., Ren, Q., Shao, S., Liu, Y., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.15075 · Cited by 12
“We adopt a perturbation-based paradigm (Lei et al., 2016; Liu et al., 2024; Cohen-Wang et al., 2024; Chuang et al., 2025), which measures causal influence by observing the model’s prediction shifts when a certain sentence is ablated.”
Measuring and Improving BERT’s Mathematical Abilities by Predicting the Order of Reasoning.
Piekos, P., Malinowski, M., & Michalewski, H. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-short.49 · Cited by 33
“…et al., 2014; Mnih et al., 2014; Gulcehre et al., 2016; Malinowski et al., 2018; Xu and Saenko, 2016; Yang et al., 2016), phrase selection (Lei et al., 2016), distillation into programs (Hajipour et al., 2020), or decision trees (Alaniz and Akata, 2019) can potentially increase the…”
BackgroundEnhancing the Rationale-Input Alignment for Self-explaining Rationalization
Liu, W., Wang, H., Wang, J., Deng, Z., Zhang, Y., Wang, C., & Li, R. (2024) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde60146.2024.00176 · Cited by 19
“The difference is that they generate an explanation for an already-trained predictor.”
BackgroundMethodD-REX: Dialogue Relation Extraction with Explanations
Albalak, A., Embar, V. R., Tuan, Y.-L., Getoor, L. C., & Wang, W. Y. (2022) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2022.nlp4convai-1.4 · Cited by 10
Critiquing Protein Family Classification Models Using Sufficient Input Subsets
Carter, B., Bileschi, M. L., Smith, J., Sanderson, T., Bryant, D., Belanger, D., & Colwell, L. J. (2019) · Journal of Computational Biology · doi:10.1089/cmb.2019.0339 · Cited by 10
“proach of Lei et al. (2016). However, in practice, these tech-”
Predict then Interpolate: A Simple Algorithm to Learn Stable ClassifiersPreprint
Bao, Y., Chang, S., & Barzilay, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.12628 · Cited by 17
“For BeerReview, we consider three aspect-level sentiment classification tasks: look, aroma and palate (Lei et al., 2016; Bao et al., 2018).”
MethodSelf-citationA Framework for Evaluating Post Hoc Feature-Additive ExplainersPreprint
Carmichael, Z., & Scheirer, W. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.08376 · Cited by 12
“The framework uses the RCNN rationale generator [48] to produce ground truth explanations; the subset of features in the rationale is simply the explanation.”
MethodDai, J., Kim, M.-Y., Sutton, R. T., Mitchell, J. R., Goebel, R. G., & Baumgart, D. C. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01729-5 · Cited by 11
What’s in Your Head? Emergent Behaviour in Multi-Task Transformer Models
Geva, M., Katz, U., Ben-Arie, A., & Berant, J. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.646 · Cited by 13
“and extract rationales through unsupervised training (Lei et al., 2016).”
BackgroundSchuff, H., Yang, H.-Y., Adel, H., & Vu, N. T. (2021) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2021.blackboxnlp-1.3 · Cited by 13
Fei, H., Li, C., Ji, D., & Li, F. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3532029 · Cited by 11
“For the 10-scale BeerAdvocate anotation, we divide 1&2&3&4 rating as negative, 5&6 rating as neutral, and 7&8&9&10 as positive rating.”
MethodInfluentialDu, H., & Xing, W. (2023) · Distance Education · doi:10.1080/01587919.2022.2150145 · Cited by 9
“Lei et al. (2016) proposed a neural framework for automatically predicting the sentiments of a document and extracting rationales from the document.”
BackgroundLearning to Identify Critical States for Reinforcement Learning from Videos
Liu, H., Zhuge, M., Li, B., Wang, Y., Faccio, F., Ghanem, B., & Schmidhuber, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00187 · Cited by 18
“79 Rationale Net [36] (s, a, y, π) -29.”
Tell Me Why: Using Question Answering as Distant Supervision for Answer Justification
Sharp, R., Surdeanu, M., Jansen, P. A., Valenzuela-Escárcega, M. A., Clark, P. E., & Hammond, M. (2017) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k17-1009 · Cited by 9
“We follow the intuition of Lei et al. (2016), whose two-component network first generates text spans from an input document, and then uses these text spans to make predictions.”
BackgroundInfluentialState gradients for analyzing memory in LSTM language models
Verwimp, L., hamme, H. V., & Wambacq, P. (2019) · Computer Speech & Language · doi:10.1016/j.csl.2019.101034 · Cited by 11
Weakly Supervised Medication Regimen Extraction from Medical Conversations
Patel, D., Konam, S., & Prabhakar, S. (2020) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2020.clinicalnlp-1.20 · Cited by 12
Counterfactual Explanations for Natural Language Interfaces
Tolkachev, G., Mell, S., Zdancewic, S. A., & Bastani, O. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.14 · Cited by 15
AutoCAD: Automatically Generate Counterfactuals for Mitigating Shortcut Learning
Wen, J., Zhu, Y., Zhang, J., Zhou, J., & Huang, M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.170 · Cited by 26
ZARA: Improving Few-Shot Self-Rationalization for Small Language Models
Chen, W., Yen, A.-Z., Wu, C.-K., Huang, H., & Chen, H. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.310 · Cited by 13
A Circumstance-Aware Neural Framework for Explainable Legal Judgment Prediction
Yue, L., Liu, Q., Jin, B., Wu, H., & An, Y. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3387580 · Cited by 9
“Selective Rationalizations have received increasing attention for providing the model explain-ability [7], [25], [26], [27], [28], [29], [30].”
BackgroundMethodInfluentialOpen-Domain Neural Conversational Agents: The Step Towards Artificial General Intelligence
Arsovski, S., Hui, S., & Cheok, A. D. (2018) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2018.090654 · Cited by 12
“Current cutting-edge approach presented in [23], incorporate rationale generation as an integral part of the learning problem.”
BackgroundInfluentialMulti-Dimensional Explanation of Target Variables from Documents
Antognini, D., Musat, C., & Faltings, B. V. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i14.17483 · Cited by 9
“…describe each aspect (one switch ? per aspect), while the attention model has many short, noisy interleaving sequences. and suffice on their own to yield the prediction, it can potentially be understood and verified against domain knowledge (Lei, Barzilay, and Jaakkola 2016; Chang et al. 2019).”
BackgroundMethodInfluentialLearning to predict with supporting evidence: applications to clinical risk prediction
Raghu, A., Guttag, J. V., Young, K., Pomerantsev, E. V., Dalca, A. V., & Stultz, C. M. (2021) · ACM Conference on Health, Inference, and Learning (CHIL) · doi:10.1145/3450439.3451869 · Cited by 11
“Work on improving trust has often focused on ML model interpretability, including post hoc interpretation using input feature attribution [25, 40, 43, 50]; post hoc concept-based attribution [10, 16]; and constructing explainable models through regularisation [27, 33] or prototype/input extraction [1, 22, 24].”
BackgroundModel Distillation for Faithful Explanations of Medical Code Predictions
Wood-Doughty, Z., Cachola, I. A., & Dredze, M. H. (2022) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2022.bionlp-1.41 · Cited by 14
“Prior work has explored methods such as forcing a faithful classifier to make predictions from a limited set of (plausible) rationales (Jain et al., 2020), or focusing on extracting rationales to constrain predictors to be inherently interpretable (Lei et al., 2016; Bastings et al., 2019).”
MethodImproving the Interpretability of Neural Sentiment Classifiers via Data Augmentation.Preprint
Chen, H., & Ji, Y. (2019) · arXiv (Cornell University) · Cited by 9
Connecting Attributions and QA Model Behavior on Realistic CounterfactualsPreprint
Ye, X., Nair, R., & Durrett, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.04515 · Cited by 9
“Past work has evaluated along plausibility criteria (Lei et al., 2016; Strout et al., 2019; Thorne et al., 2019), but as we see from this example, faithful explanations (Subramanian et al.”
BackgroundTalking Nonsense: Probing Large Language Models' Understanding of Adversarial Gibberish InputsPreprint
Cherepanova, V., & Zou, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.17120 · Cited by 14
“…the explainability of these models include saliency maps, which are designed to high-light the input data regions (Simonyan et al., 2013; Zeiler and Fergus, 2014; Lei et al., 2016; Feldhus et al., 2023) or model parameters (Levin et al., 2022) that significantly influence the model’s outputs.”
BackgroundDeepResearch-Slice: Bridging the Retrieval-Utilization Gap via Explicit Text SlicingPreprint
Lu, S., Xu, Y., Cheng, J., He, L., Wang, M., & Liang, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.03261 · Cited by 9
“We introduce S LICE , a model-agnostic operator that extracts precise, index-addressable spans—akin to extractive rationale selection (Lei et al., 2016; Yu et al., 2022) but applied to open-ended web traces.”
Learning to Rationalize for Nonmonotonic Reasoning with Distant Supervision
Brahman, F., Shwartz, V., Rudinger, R., & Choi, Y. (2021) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v35i14.17492 · Cited by 31
Lin, T., Lau, R. Y. K., & Hu, S. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11565-y · Cited by 11
“Early work, such as Rationalizing Neural Predictions (RNP) (Lei et al. 2016 ), achieved interpretability through a generator-encoder framework, optimizing for accuracy, simplicity, and continuity of reasoning.”
InfluentialJourdan, F., Picard, A. M., Fel, T., Risser, L., Loubes, J., & Asher, N. M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.317 · Cited by 15
Morality in the Mundane: Categorizing Moral Reasoning in Real-Life Social Situations
Xi, R., & Singh, M. P. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31415 · Cited by 12
From outputs to insights: a survey of rationalization approaches for explainable text classification
Guzman, E. M., Schlegel, V., & Batista-Navarro, R. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1363531 · Cited by 13
Probabilistic Automata-Based Method for Enhancing Performance of Deep Reinforcement Learning Systems
Yang, M., Liu, G., Zhou, Z., & Wang, J. (2024) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2024.124818 · Cited by 8
Challenges and Perspectives of Hate Speech Research
Strippel, C., Paasch‐Colberg, S., Emmer, M., & Trebbe, J. (2023) · Weizenbaum Library (Weizenbaum Institute) · doi:10.48541/dcr.v12.0 · Cited by 25
Multi-Granular Text Encoding for Self-Explaining Categorization
Wang, Z., Zhang, Y., Yu, M., Zhang, W., Pan, L., Song, L., Xu, K., & El-Kurdi, Y. (2019) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w19-4805 · Cited by 9
“Lei et al. (2016) introduce a regularizer over the selected (single-word) positions to encourage the model to extract larger phrases.”
BackgroundMethodIST-Unbabel 2021 Submission for the Explainable Quality Estimation Shared Task
Treviso, M., Guerreiro, N. M., Rei, R., & Martins, A. F. T. (2021) · Workshop on Evaluation and Comparison of NLP Systems (Eval4NLP) · doi:10.18653/v1/2021.eval4nlp-1.14 · Cited by 17
“In contrast, ratio-nalizers with hard attention are arguably more faithful but require stochastic networks (Lei et al., 2016; Bastings et al., 2019), with recent works avoiding stochasticity via sparse deterministic selections (Guerreiro and Martins, 2021).”
BackgroundMethodHop, Union, Generate: Explainable Multi-hop Reasoning without Rationale Supervision
Zhao, W., Chiu, J., Cardie, C., & Rush, A. M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1001 · Cited by 13
Plausible Extractive Rationalization through Semi-Supervised Entailment Signal
Jie, Y. W., Satapathy, R., & Cambria, E. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.307 · Cited by 12
“Extractive Rationalization : The foundational work by (Lei et al., 2016) introduced extractive rationalization using REINFORCE with sparsity regularization for end-to-end training of explainers via predictors’ objectives.”
MethodThe Emergence of Chunking Structures with Hierarchical RNN
Wu, Z., Deshmukh, A. A., Wu, Y., Lin, J., & Mou, L. (2024) · Computational Linguistics · doi:10.1162/coli_a_00545 · Cited by 10
“For example, Lei, Barzilay, and Jaakkola (2016) extract key phrases that rationalize the neural model’s predictions on classification tasks.”
Riboexp: an interpretable reinforcement learning framework for ribosome density modeling
Hu, H., Liu, X., Xiao, A., Li, Y., Zhang, C., Jiang, T., Zhao, D. Y., Song, S., & Zeng, J. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbaa412 · Cited by 12
Nishida, K., Nishida, K., Saito, I., & Yoshida, S. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534370 · Cited by 10
“In this study, we propose to use a pipeline model for RC that first extracts rationales from the passage and then predicts the answer; this model has been used in the past for interpretation of text classification tasks [8]–[10].”
Are Shortest Rationales the Best Explanations for Human Understanding?
Shen, H., Wu, T., Guo, W., & Huang, T.-H. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.2 · Cited by 8
Using Program Induction to Interpret Transition System DynamicsPreprint
Penkov, S., & Ramamoorthy, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.00376 · Cited by 9
“Methods for textual rationalisation of the predictions made by deep models have also been proposed (Harrison et al., 2017; Hendricks et al., 2016; Lei et al., 2016).”
BackgroundGaussian Process-Based Models for Clinical Time Series in Healthcare
Futoma, J. (2018) · Cited by 12
AI as a Design Material: Dealing with new agencies
Ramon Sangüesa, A., & Guersenzvaig, A. (2019) · Temes de disseny · doi:10.46467/tdd35.2019.6-25 · Cited by 8
Weakly Supervised Explainable Phrasal Reasoning with Neural Fuzzy LogicPreprint
Wu, Z., Zhang, Z. X., Naik, A., Mei, Z., Firdaus, M., & Mou, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.08927 · Cited by 15
“For example, (Lei, Barzilay, and Jaakkola 2016) extract key phrases for a text classification task, and (Liu et al. 2018) extract key sentences for paragraph classification.”
Proceedings of the Third Workshop on Fact Extraction and VERification (FEVER)
(2020) · doi:10.18653/v1/2020.fever-1 · Cited by 8
“This approach was shown to perform better than simply relying on the internal attention weights of a classifier (Lei et al., 2016; Jain and Wallace, 2019).”
Jourdan, F., Picard, A. M., Fel, T., Risser, L., Loubes, J., & Asher, N. M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.06754 · Cited by 10
“Lei et al. (2016) defined rationales as "a minimal set of text spans that are sufficient to support a given claim or decision".”
BackgroundMethodInfluential
Metadata
- Authors
- Lei, T., Barzilay, R., & Jaakkola, T.
- Affiliations
- MIT Computer Science and Artificial Intelligence Laboratory (US)
- Venue
- Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing
- Published
- 2016
- Pages
- 107-117
- OpenAlex
- W2440159969
- Cited by
- 777
- References
- 27
- Citations by year
- 2026: 27 · 2025: 33 · 2024: 39 · 2023: 88 · 2022: 92 · 2021: 138 · 2020: 126 · 2019: 101 · 2018: 78 · 2017: 39 · 2016: 4
- FWCI
- 39.87
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- explanation generation, question retrieval, model regularization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.