Student or educator? Sign up with your school email and get 6 months of Pro free.
Cited by 120

The DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index.

Towards Faithful Model Explanation in NLP: A Survey

Lyu, Qing, Apidianaki, Marianna, Callison-Burch, Chris·2024·Computational Linguistics

Abstract

End-to-end neural Natural Language Processing (NLP) models are notoriously difficult to understand. This has given rise to numerous efforts towards model explainability in recent years. One desideratum of model explanation is faithfulness, that is, an explanation should accurately represent the reasoning process behind the model’s prediction. In this survey, we review over 110 model explanation methods in NLP through the lens of faithfulness. We first discuss the definition and evaluation of faithfulness, as well as its significance for explainability. We then introduce recent advances in faithful explanation, grouping existing approaches into five categories: similarity-based methods, analysis of model-internal structures, backpropagation-based methods, counterfactual intervention, and self-explanatory models. For each category, we synthesize its representative studies, strengths, and weaknesses. Finally, we summarize their common virtues and remaining challenges, and reflect on future work directions towards faithful explainability in NLP.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1162/coli_a_00511

Cites

75 of 75 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 75 references

Research works printed but not linked: 175

  1. Abraham, 2022, CEBaB: Estimating the causal effects of real-world concepts on NLP model behavior, Advances in Neural Information Processing Systems, 35, 17582
  2. Adebayo, 2018, Sanity checks for saliency maps, 9525
  3. Adebayo, 2020, Debugging tests for model explanations, Advances in Neural Information Processing Systems, 33, 700
  4. Adi, 2017, Fine-grained analysis of sentence embeddings using auxiliary prediction tasks
  5. Alvarez-Melis, 2018, Towards robust interpretability with self-explaining neural networks, 7786
  6. Amini, 2022, Naturalistic causal probing for morpho-syntax, Transactions of the Association for Computational Linguistics, 11, 384, 10.1162/tacl_a_00554 · DOI
  7. Andreas, 2016, Learning to compose neural networks for question answering, 1545, 10.18653/v1/N16-1181 · DOI · Paper page
  8. Andreas, 2016, Neural module networks, 39, 10.1109/CVPR.2016.12 · DOI · Paper page
  9. Arras, 2016, Explaining predictions of non-linear classifiers in NLP, 1, 10.18653/v1/W16-1601 · DOI
  10. Atanasova, 2020, A diagnostic study of explainability techniques for text classification, 3256, 10.18653/v1/2020.emnlp-main.263 · DOI · Paper page
  11. Bahdanau, 2015, Neural machine translation by jointly learning to align and translate
  12. Bastings, 2019, Interpretable neural predictions with differentiable binary variables, 2963, 10.18653/v1/P19-1284 · DOI
  13. Bastings, 2022, “Will you find these shortcuts?” A protocol for evaluating the faithfulness of input salience methods for text classification, 976, 10.18653/v1/2022.emnlp-main.64 · DOI
  14. Bastings, 2020, The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?, 149, 10.18653/v1/2020.blackboxnlp-1.14 · DOI
  15. Basu, 2021, Influence functions in deep learning are fragile
  16. Bau, 2019, Identifying and controlling important neurons in neural machine translation
  17. Belinkov, 2020, On the linguistic representational power of neural machine translation models, Computational Linguistics, 46, 1, 10.1162/coli_a_00367 · DOI
  18. Belinkov, 2019, Analysis methods in neural language processing: A survey, Transactions of the Association for Computational Linguistics, 7, 49, 10.1162/tacl_a_00254 · DOI · Paper page
  19. Bogin, 2021, Latent compositional representations improve systematic generalization in grounded question answering, Transactions of the Association for Computational Linguistics, 9, 195, 10.1162/tacl_a_00361 · DOI
  20. Bowman, 2015, A large annotated corpus for learning natural language inference, 632, 10.18653/v1/D15-1075 · DOI · Paper page
  21. Brunner, 2018, Natural language multitasking: Analyzing and improving syntactic saliency of hidden representations, arXiv preprint arXiv:1801.06024 · arXiv
  22. Calderon, 2022, DoCoGen: Domain counterfactual generation for low resource domain adaptation, 7727, 10.18653/v1/2022.acl-long.533 · DOI
  23. Camburu, 2020, Make up your mind! Adversarial generation of inconsistent natural language explanations, 4157, 10.18653/v1/2020.acl-main.382 · DOI
  24. Caruana, 1999, Case-based explanation of non-case-based learning methods., Proceedings of the AMIA Symposium, 212
  25. Cashman, 2018, RNNbow: Visualizing learning via backpropagation gradients in RNNs, IEEE Computer Graphics and Applications, 38, 39, 10.1109/MCG.2018.2878902 · DOI
  26. Chan, 2022, A comparative study of faithfulness metrics for model interpretability methods, 5029, 10.18653/v1/2022.acl-long.345 · DOI
  27. Chen, 2022, REV: Information-theoretic evaluation of free-text rationales, Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2007, 10.18653/v1/2023.acl-long.112 · DOI
  28. Chen, 2018, Learning to explain: An information-theoretic perspective on model interpretation, 882
  29. Chen, 2022, Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks, Transactions on Machine Learning Research
  30. Choenni, 2021, Stepmothers are mean and academics are pretentious: What do pretrained language models learn about you?, 1477, 10.18653/v1/2021.emnlp-main.111 · DOI
  31. Clark, 2020, ELECTRA: Pre-training text encoders as discriminators rather than generators
  32. Clinciu, 2021, A study of automatic metrics for the evaluation of natural language explanations, 2376, 10.18653/v1/2021.eacl-main.202 · DOI
  33. Conneau, 2018, What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties, 2126, 10.18653/v1/P18-1198 · DOI · Paper page
  34. Creswell, 2022, Faithful reasoning using large language models, ArXiv preprint
  35. Dalvi, 2021, Explaining answers with entailment trees, 7358, 10.18653/v1/2021.emnlp-main.585 · DOI
  36. Dalvi, 2022, Discovering latent concepts learned in BERT
  37. Danilevsky, 2020, A survey of the state of explainable AI for natural language processing, 447
  38. De Cao, 2022, Sparse interventions in language models with differentiable masking, 16, 10.18653/v1/2022.blackboxnlp-1.2 · DOI
  39. Denil, 2015, Extraction of salient sentences from labelled documents, ArXiv preprint, arXiv:1412.6815 [cs] · arXiv
  40. Deutsch, 2019, A general-purpose algorithm for constrained sequential inference, 482, 10.18653/v1/K19-1045 · DOI
  41. Ding, 2021, Evaluating saliency methods for neural language models, 5034, 10.18653/v1/2021.naacl-main.399 · DOI
  42. Eberle, 2022, Do transformer models show similar attention patterns to task-specific human gaze?, 4295, 10.18653/v1/2022.acl-long.296 · DOI
  43. Ebrahimi, 2018, HotFlip: White-box adversarial examples for text classification, 31, 10.18653/v1/P18-2006 · DOI · Paper page
  44. Ethayarajh, 2021, Attention flows are Shapley Value explanations, 49, 10.18653/v1/2021.acl-short.8 · DOI
  45. Feder, 2021, CausaLM: Causal model explanation through counterfactual language models, Computational Linguistics, 47, 333, 10.1162/coli_a_00404 · DOI
  46. Feng, 2018, Pathologies of neural models make interpretations difficult, 3719, 10.18653/v1/D18-1407 · DOI · Paper page
  47. Finlayson, 2021, Causal analysis of syntactic agreement mechanisms in neural language models, 1828, 10.18653/v1/2021.acl-long.144 · DOI
  48. Gardner, 2020, Evaluating models’ local decision boundaries via contrast sets, 1307, 10.18653/v1/2020.findings-emnlp.117 · DOI · Paper page
  49. Gupta, 2020, Neural module networks for reasoning over text
  50. Haghighatkhah, 2022, Better hit the nail on the head than beat around the bush: Removing protected attributes with a single projection, 8395, 10.18653/v1/2022.emnlp-main.575 · DOI
  51. Halpern, 2005, Causes and explanations: A structural-model approach. Part I: Causes, The British Journal for the Philosophy of Science, 56, 843, 10.1093/bjps/axi147 · DOI · Paper page
  52. Hamilton, 2022, Is neuro-symbolic AI meeting its promises in natural language processing? A structured review, 1, 10.3233/SW-223228 · DOI
  53. Han, 2020, Explaining black box predictions and unveiling data artifacts through influence functions, 5553, 10.18653/v1/2020.acl-main.492 · DOI
  54. Hao, 2021, Self-attention attribution: Interpreting information interactions inside Transformer, 12963, 10.1609/aaai.v35i14.17533 · DOI · Paper page
  55. Harrington, 1985, Harvey Friedman’s Research on the Foundations of Mathematics
  56. Hase, 2022, When can models learn from explanations? A formal framework for understanding the roles of explanation data, 29, 10.18653/v1/2022.lnls-1.4 · DOI
  57. Hendrycks, 2021, Measuring massive multitask language understanding
  58. Herman, 2017, The promise and peril of human evaluation for model interpretability, ArXiv preprint
  59. Hiebert, 2018, Interpreting word-level hidden state behaviour of character-level LSTM language models, 258, 10.18653/v1/W18-5428 · DOI
  60. Hong, 2022, METGEN: A module-based entailment tree generation framework for answer explanation, 1887, 10.18653/v1/2022.findings-naacl.145 · DOI
  61. Hooker, 2019, A benchmark for interpretability methods in deep neural networks, 9734
  62. Hu, 2017, Learning to reason: End-to-end module networks for visual question answering, 804, 10.1109/ICCV.2017.93 · DOI · Paper page
  63. Jacovi, 2020, Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness?, 4198, 10.18653/v1/2020.acl-main.386 · DOI · Paper page
  64. Jacovi, 2021, Aligning faithful interpretations with their social attribution, Transactions of the Association for Computational Linguistics, 9, 294, 10.1162/tacl_a_00367 · DOI
  65. Jain, 2019, Attention is not explanation, 3543
  66. Jain, 2020, Learning to faithfully rationalize by construction, 4459, 10.18653/v1/2020.acl-main.409 · DOI
  67. Janizek, 2021, Explaining explanations: Axiomatic feature interactions for deep networks, Journal of Machine Learning Research, 22, 104:1
  68. Jiang, 2020, Cold-start and interpretability: Turning regular expressions into trainable recurrent neural networks, 3193, 10.18653/v1/2020.emnlp-main.258 · DOI
  69. Jiang, 2019, Explore, propose, and assemble: An interpretable model for multi-hop reading comprehension, 2714, 10.18653/v1/P19-1261 · DOI
  70. Ju, 2022, Logic traps in evaluating attribution scores, 5911, 10.18653/v1/2022.acl-long.407 · DOI
  71. Kádár, 2017, Representation of linguistic form and function in recurrent neural networks, Computational Linguistics, 43, 761, 10.1162/COLI_a_00300 · DOI
  72. Karidi, 2021, Putting words in BERT’s mouth: Navigating contextualized vector spaces with pseudowords, 10300, 10.18653/v1/2021.emnlp-main.806 · DOI
  73. Kassner, 2021, BeliefBank: Adding memory to a pre-trained language model for a systematic notion of belief, 8849, 10.18653/v1/2021.emnlp-main.697 · DOI
  74. Kaushik, 2020, Learning the difference that makes a difference with counterfactually-augmented data
  75. Kaushik, 2018, How much reading does reading comprehension require? A critical investigation of popular benchmarks, 5010, 10.18653/v1/D18-1546 · DOI · Paper page
  76. Kindermans, 2019, The (un)reliability of saliency methods, 267, 10.1007/978-3-030-28954-6_14 · DOI · Paper page
  77. Kindermans, 2018, Learning how to explain neural networks: PatternNet and PatternAttribution
  78. Kokhlikyan, 2020, Captum: A unified and generic model interpretability library for PyTorch, ArXiv preprint
  79. Krishnamurthy, 2013, Jointly learning to parse and perceive: Connecting natural language to the physical world, Transactions of the Association for Computational Linguistics, 1, 193, 10.1162/tacl_a_00220 · DOI
  80. Kumar, 2022, Probing classifiers are unreliable for concept removal and detection, Advances in Neural Information Processing Systems, 35, 17994
  81. Kumar, 2020, NILE: Natural language inference with faithful natural language explanations, 8730, 10.18653/v1/2020.acl-main.771 · DOI
  82. Lakkaraju, 2020, “How do I fool you?”: Manipulating user trust via misleading black box explanations, 79, 10.1145/3375627.3375833 · DOI · Paper page
  83. Laugel, 2018, Defining locality for surrogates in post-hoc interpretablity
  84. Levesque, 2012, The Winograd Schema Challenge, 10
  85. Li, 2020, Evaluating explanation methods for neural machine translation, 365, 10.18653/v1/2020.acl-main.35 · DOI
  86. Li, 2016, Visualizing and understanding neural models in NLP, 681, 10.18653/v1/N16-1082 · DOI · Paper page
  87. Li, 2016, Understanding neural networks through representation erasure, ArXiv preprint
  88. Lipton, 2016, The Mythos of Model Interpretability, ArXiv preprint
  89. Liu, 2022, Rethinking attention-model explainability through faithfulness violation test, 13807
  90. Lovering, 2020, Information-theoretic probing explains reliance on spurious features
  91. Lu, 2021, Influence patterns for explaining information flow in BERT, 4461
  92. Madaan, 2021, Improving neural model performance through natural language feedback on their explanations, ArXiv preprint
  93. Mao, 2019, The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision
  94. Marasovic, 2022, Few-shot self-rationalization with natural language prompts, 410, 10.18653/v1/2022.findings-naacl.31 · DOI
  95. Martins, 2016, From softmax to sparsemax: A sparse model of attention and multi-label classification, 1614
  96. McCoy, 2019, Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference, 3428, 10.18653/v1/P19-1334 · DOI · Paper page
  97. Ming, 2017, Understanding hidden memories of recurrent neural networks, 13, 10.1109/VAST.2017.8585721 · DOI · Paper page
  98. Montavon, 2019, Layer-wise relevance propagation: An overview, 193, 10.1007/978-3-030-28954-6_10 · DOI · Paper page
  99. Mosca, 2022, SHAP-based explanation methods: A review for NLP interpretability, 4593
  100. Mueller, 2022, Causal analysis of syntactic agreement neurons in multilingual language models, 95, 10.18653/v1/2022.conll-1.8 · DOI
  101. Mullenbach, 2018, Explainable prediction of medical codes from clinical text, 1101, 10.18653/v1/N18-1100 · DOI · Paper page
  102. Mylonas, 2022, An attention matrix for every decision: Faithfulness-based arbitration among multiple attention-based interpretations of transformers in text classification, Data Mining and Knowledge Discovery, 38, 128, 10.1007/s10618-023-00962-4 · DOI
  103. Narang, 2020, WT5?! Training Text-to-Text Models to Explain their predictions, ArXiv preprint
  104. Nie, 2018, A theoretical explanation for perplexing behaviors of backpropagation-based visualizations, 3806
  105. Nye, 2021, Show your work: Scratchpads for intermediate computation with language models, Deep Learning for Code Workshop
  106. OpenAI. 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774. · arXiv
  107. Parcalabescu, 2024, On measuring faithfulness or self-consistency of natural language explanations, arXiv
  108. Park, 2018, Multimodal explanations: Justifying decisions and pointing to the evidence, 8779, 10.1109/CVPR.2018.00915 · DOI · Paper page
  109. Pascual, 2021, Telling BERT’s full story: From local attention to global aggregation, 105, 10.18653/v1/2021.eacl-main.9 · DOI
  110. Petroni, 2019, Language models as knowledge bases?, 2463, 10.18653/v1/D19-1250 · DOI · Paper page
  111. Pezeshkpour, 2021, An empirical comparison of instance attribution methods for NLP, 967, 10.18653/v1/2021.naacl-main.75 · DOI
  112. Poerner, 2018, Interpretable textual neuron representations for NLP, 325, 10.18653/v1/W18-5437 · DOI
  113. Poerner, 2018, Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement, 340, 10.18653/v1/P18-1032 · DOI
  114. Poliak, 2018, Hypothesis only baselines in natural language inference, 180, 10.18653/v1/S18-2023 · DOI · Paper page
  115. Pruthi, 2022, Evaluating explanations: How much do explanations from the teacher aid students?, Transactions of the Association for Computational Linguistics, 10, 359, 10.1162/tacl_a_00465 · DOI
  116. Qian, 2022, Limitations of language models in arithmetic and symbolic induction, 10.18653/v1/2023.acl-long.516 · DOI
  117. Qian, 2016, Analyzing linguistic knowledge in sequential model of sentence, 826, 10.18653/v1/D16-1079 · DOI
  118. Raganato, 2020, Fixed encoder self-attention patterns in transformer-based machine translation, 556, 10.18653/v1/2020.findings-emnlp.49 · DOI
  119. Rajagopal, 2021, SELFEXPLAIN: A self-explaining architecture for neural text classifiers, 836, 10.18653/v1/2021.emnlp-main.64 · DOI
  120. Rajani, 2019, Explain yourself! Leveraging language models for commonsense reasoning, 4932, 10.18653/v1/P19-1487 · DOI · Paper page
  121. Ramamurthy, 2020, Model agnostic multilevel explanations, 5968
  122. Ravfogel, 2022, Log-linear guardedness and its implications, 9413, 10.18653/v1/2023.acl-long.523 · DOI
  123. Ravichander, 2021, Probing the probing paradigm: Does probing accuracy entail task relevance?, 3363, 10.18653/v1/2021.eacl-main.295 · DOI · Paper page
  124. Reif, 2019, Visualizing and measuring the geometry of BERT, 8592
  125. Ribeiro, 2016, “Why should I trust you?”: Explaining the predictions of any classifier, 1135, 10.1145/2939672.2939778 · DOI · Paper page
  126. Roese, 1995, Counterfactual thinking: A critical overview, 1
  127. Sajjad, 2022, Neuron-level interpretation of deep NLP models: A survey, Transactions of the Association for Computational Linguistics, 10, 1285, 10.1162/tacl_a_00519 · DOI
  128. Schwartz, 2018, Bridging CNNs, RNNs, and weighted finite-state machines, 295, 10.18653/v1/P18-1028 · DOI
  129. Shrikumar, 2017, Learning important features through propagating activation differences, 3145
  130. Shrikumar, 2017, Not just a black box: Learning important features through propagating activation differences, ArXiv preprint, arXiv:1605.01713 [cs] · arXiv
  131. Sia, 2022, Logical satisfiability of counterfactuals for faithful explanations in NLI, 9837, 10.1609/aaai.v37i8.26174 · DOI
  132. Slack, 2020, Fooling LIME and SHAP: Adversarial attacks on post hoc explanation methods, 180, 10.1145/3375627.3375830 · DOI · Paper page
  133. Smilkov, 2017, SmoothGrad: Removing noise by adding noise, ArXiv preprint
  134. Strobelt, 2019, Seq2seq-Vis: A visual debugging tool for sequence-to-sequence models, IEEE Transactions on Visualization and Computer Graphics, 25, 353, 10.1109/TVCG.2018.2865044 · DOI · Paper page
  135. Strobelt, 2018, LSTMVis: A tool for visual analysis of hidden state dynamics in recurrent neural networks, IEEE Transactions on Visualization and Computer Graphics, 24, 667, 10.1109/TVCG.2017.2744158 · DOI · Paper page
  136. Subramanian, 2020, Obtaining faithful interpretations from compositional neural networks, 5594, 10.18653/v1/2020.acl-main.495 · DOI
  137. Sundararajan, 2017, Axiomatic attribution for deep networks, 3319
  138. Sushil, 2018, Patient representation learning and interpretable evaluation using clinical notes, Journal of Biomedical Informatics, 84, 103, 10.1016/j.jbi.2018.06.016 · DOI
  139. Tsang, 2020, How does this interaction affect me? Interpretable attribution for feature interactions, 6147
  140. Tutek, 2020, Staying true to your word: (How) can attention become explanation?, 131, 10.18653/v1/2020.repl4nlp-1.17 · DOI
  141. Vashishth, 2019, Attention interpretability across NLP tasks, ArXiv preprint
  142. Vaswani, 2017, Attention is all you need, 5998
  143. Veldhoen, 2016, Diagnostic classifiers: Revealing how neural networks process hierarchical structure, CoCo@ NIPS, 69
  144. Vig, 2019, Visualizing attention in transformer-based language representation models, ArXiv preprint
  145. Vig, 2020, Investigating gender bias in language models using causal mediation analysis, 12388
  146. Voita, 2020, Information-theoretic probing with minimum description length, 183, 10.18653/v1/2020.emnlp-main.14 · DOI · Paper page
  147. Wallace, 2018, Interpreting neural networks with nearest neighbors, 136, 10.18653/v1/W18-5416 · DOI
  148. Wallace, 2019, Universal adversarial triggers for attacking and analyzing NLP, 2153, 10.18653/v1/D19-1221 · DOI · Paper page
  149. Wallace, 2020, Interpreting predictions of NLP models, 20, 10.18653/v1/2020.emnlp-tutorials.3 · DOI
  150. Wang, 2019, SuperGLUE: A stickier benchmark for general-purpose language understanding systems, 3261
  151. Wang, 2019, GLUE: A multi-task benchmark and analysis platform for natural language understanding, 10.18653/v1/W18-5446 · DOI · Paper page
  152. Wang, 2020, Gradient-based analysis of NLP models is manipulable, 247, 10.18653/v1/2020.findings-emnlp.24 · DOI
  153. Wang, 2022, A fine-grained interpretability evaluation benchmark for neural NLP, 70, 10.18653/v1/2022.conll-1.6 · DOI
  154. Wang, 2022, Self-consistency improves chain of thought reasoning in language models
  155. Wexler, 2020, The What-If Tool: Interactive probing of machine learning models, IEEE Transactions on Visualization and Computer Graphics, 26, 56, 10.1109/TVCG.2019.2934619 · DOI · Paper page
  156. Wiegreffe, 2022, Reframing human-AI collaboration for generating free-text explanations, 632, 10.18653/v1/2022.naacl-main.47 · DOI
  157. Wiegreffe, 2019, Attention is not not explanation, 11, 10.18653/v1/D19-1002 · DOI · Paper page
  158. Wu, 2021, Polyjuice: Generating counterfactuals for explaining, evaluating, and improving models, 6707, 10.18653/v1/2021.acl-long.523 · DOI · Paper page
  159. Xie, 2017, An interpretable knowledge transfer model for knowledge base completion, 950, 10.18653/v1/P17-1088 · DOI
  160. Yang, 2019, Benchmarking attribution methods with relative feature importance, arXiv preprint arXiv:1907.09701 · arXiv
  161. Ye, 2023, Explanation Selection Using Unlabeled Data for In-Context Learning, 10.18653/v1/2023.emnlp-main.41 · DOI
  162. Ye, 2022, Complementary explanations for effective in-context learning, 4469, 10.18653/v1/2023.findings-acl.273 · DOI
  163. Ye, 2021, Connecting attributions and QA model behavior on realistic counterfactuals, 5496, 10.18653/v1/2021.emnlp-main.447 · DOI
  164. Yeh, 2019, On the (in)fidelity and sensitivity of explanations, 10965
  165. Yeh, 2020, On completeness-aware concept-based explanations in deep neural networks, 20554
  166. Yi, 2018, Neural-symbolic VQA: Disentangling reasoning from vision and language understanding, 1039
  167. Yin, 2022, On the sensitivity and stability of model interpretations in NLP, 2631, 10.18653/v1/2022.acl-long.188 · DOI
  168. Yin, 2022, Interpreting language models with contrastive explanations, 184, 10.18653/v1/2022.emnlp-main.14 · DOI
  169. Zaidan, 2007, Using “Annotator Rationales” to Improve Machine Learning for Text Categorization, 260
  170. Zheng, 2022, The irrationality of neural rationale models, 64, 10.18653/v1/2022.trustnlp-1.6 · DOI
  171. Zhou, 2022, Least-to-most prompting enables complex reasoning in large language models
  172. Zhou, 2022, Do feature attribution methods correctly attribute features?, 9623, 10.1609/aaai.v36i9.21196 · DOI
  173. Zhou, 2022, ExSum: From local explanations to model understanding, 5359, 10.18653/v1/2022.naacl-main.392 · DOI
  174. Zhou, 2023, The Solvability of Interpretability Evaluation Metrics, 2399, 10.18653/v1/2023.findings-eacl.182 · DOI
  175. Zmigrod, 2019, Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology, 1651, 10.18653/v1/P19-1161 · DOI · Paper page

Cited by

See all 120

Metadata

Authors
Lyu, Q., Apidianaki, M., & Callison-Burch, C.
Affiliations
  • University of Pennsylvania, Department of Computer and Information Science. lyuqing@sas.upenn.edu
  • University of Pennsylvania, Department of Computer and Information Science. marapi@seas.upenn.edu
  • University of Pennsylvania, Department of Computer and Information Science. ccb@seas.upenn.edu
Venue
Computational Linguistics
Publisher
MIT Press
Published
2024
Volume
50
Issue
2
Pages
657-723
ISSN
  • 0891-2017
  • 1530-9312
Cited by
120
References
75
Language
en

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Towards Faithful Model Explanation in NLP: A Survey. https://app.cochat.ai/papers/10.1162/coli_a_00511
The paper
Lyu, Q., Apidianaki, M., & Callison-Burch, C. (2024). Towards Faithful Model Explanation in NLP: A Survey. Computational Linguistics. https://doi.org/10.1162/coli_a_00511

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.