CoChat

Towards Faithful Model Explanation in NLP: A Survey

Lyu, Qing, Apidianaki, Marianna, Callison-Burch, Chris · 2024 · Computational Linguistics
Cited by 120

The DOI matches its Crossref record; 102 of the 102 references it cites were found in a registry or an index.

Abstract

End-to-end neural Natural Language Processing (NLP) models are notoriously difficult to understand. This has given rise to numerous efforts towards model explainability in recent years. One desideratum of model explanation is faithfulness, that is, an explanation should accurately represent the reasoning process behind the model’s prediction. In this survey, we review over 110 model explanation methods in NLP through the lens of faithfulness. We first discuss the definition and evaluation of faithfulness, as well as its significance for explainability. We then introduce recent advances in faithful explanation, grouping existing approaches into five categories: similarity-based methods, analysis of model-internal structures, backpropagation-based methods, counterfactual intervention, and self-explanatory models. For each category, we synthesize its representative studies, strengths, and weaknesses. Finally, we summarize their common virtues and remaining challenges, and reflect on future work directions towards faithful explainability in NLP.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1162/coli_a_00511

Cites

102 of 102 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 152

  1. Abraham, 2022, CEBaB: Estimating the causal effects of real-world concepts on NLP model behavior, Advances in Neural Information Processing Systems, 35, 17582
  2. Adi, 2017, Fine-grained analysis of sentence embeddings using auxiliary prediction tasks
  3. Alvarez-Melis, 2018, Towards robust interpretability with self-explaining neural networks, 7786
  4. Amini, 2022, Naturalistic causal probing for morpho-syntax, Transactions of the Association for Computational Linguistics, 11, 384, 10.1162/tacl_a_00554 · DOI
  5. Andreas, 2016, Learning to compose neural networks for question answering, 1545, 10.18653/v1/N16-1181 · DOI · Paper page
  6. Andreas, 2016, Neural module networks, 39, 10.1109/CVPR.2016.12 · DOI · Paper page
  7. Arras, 2016, Explaining predictions of non-linear classifiers in NLP, 1, 10.18653/v1/W16-1601 · DOI · Paper page
  8. Atanasova, 2020, A diagnostic study of explainability techniques for text classification, 3256, 10.18653/v1/2020.emnlp-main.263 · DOI · Paper page
  9. Bastings, 2019, Interpretable neural predictions with differentiable binary variables, 2963, 10.18653/v1/P19-1284 · DOI
  10. Bastings, 2022, “Will you find these shortcuts?” A protocol for evaluating the faithfulness of input salience methods for text classification, 976, 10.18653/v1/2022.emnlp-main.64 · DOI
  11. Bastings, 2020, The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?, 149, 10.18653/v1/2020.blackboxnlp-1.14 · DOI · Paper page
  12. Basu, 2021, Influence functions in deep learning are fragile
  13. Bau, 2019, Identifying and controlling important neurons in neural machine translation
  14. Belinkov, 2020, On the linguistic representational power of neural machine translation models, Computational Linguistics, 46, 1, 10.1162/coli_a_00367 · DOI · Paper page
  15. Belinkov, 2019, Analysis methods in neural language processing: A survey, Transactions of the Association for Computational Linguistics, 7, 49, 10.1162/tacl_a_00254 · DOI · Paper page
  16. Bogin, 2021, Latent compositional representations improve systematic generalization in grounded question answering, Transactions of the Association for Computational Linguistics, 9, 195, 10.1162/tacl_a_00361 · DOI
  17. Brunner, 2018, Natural language multitasking: Analyzing and improving syntactic saliency of hidden representations, arXiv preprint arXiv:1801.06024 · arXiv
  18. Calderon, 2022, DoCoGen: Domain counterfactual generation for low resource domain adaptation, 7727, 10.18653/v1/2022.acl-long.533 · DOI
  19. Camburu, 2020, Make up your mind! Adversarial generation of inconsistent natural language explanations, 4157, 10.18653/v1/2020.acl-main.382 · DOI
  20. Caruana, 1999, Case-based explanation of non-case-based learning methods., Proceedings of the AMIA Symposium, 212
  21. Cashman, 2018, RNNbow: Visualizing learning via backpropagation gradients in RNNs, IEEE Computer Graphics and Applications, 38, 39, 10.1109/MCG.2018.2878902 · DOI
  22. Chan, 2022, A comparative study of faithfulness metrics for model interpretability methods, 5029, 10.18653/v1/2022.acl-long.345 · DOI · Paper page
  23. Chen, 2022, REV: Information-theoretic evaluation of free-text rationales, Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2007, 10.18653/v1/2023.acl-long.112 · DOI
  24. Chen, 2018, Learning to explain: An information-theoretic perspective on model interpretation, 882
  25. Choenni, 2021, Stepmothers are mean and academics are pretentious: What do pretrained language models learn about you?, 1477, 10.18653/v1/2021.emnlp-main.111 · DOI
  26. Clark, 2020, ELECTRA: Pre-training text encoders as discriminators rather than generators
  27. Clinciu, 2021, A study of automatic metrics for the evaluation of natural language explanations, 2376, 10.18653/v1/2021.eacl-main.202 · DOI
  28. Conneau, 2018, What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties, 2126, 10.18653/v1/P18-1198 · DOI · Paper page
  29. Creswell, 2022, Faithful reasoning using large language models, ArXiv preprint
  30. Dalvi, 2021, Explaining answers with entailment trees, 7358, 10.18653/v1/2021.emnlp-main.585 · DOI · Paper page
  31. Dalvi, 2022, Discovering latent concepts learned in BERT
  32. Danilevsky, 2020, A survey of the state of explainable AI for natural language processing, 447
  33. De Cao, 2022, Sparse interventions in language models with differentiable masking, 16, 10.18653/v1/2022.blackboxnlp-1.2 · DOI
  34. Denil, 2015, Extraction of salient sentences from labelled documents, ArXiv preprint, arXiv:1412.6815 [cs] · arXiv
  35. Deutsch, 2019, A general-purpose algorithm for constrained sequential inference, 482, 10.18653/v1/K19-1045 · DOI
  36. Ding, 2021, Evaluating saliency methods for neural language models, 5034, 10.18653/v1/2021.naacl-main.399 · DOI
  37. Eberle, 2022, Do transformer models show similar attention patterns to task-specific human gaze?, 4295, 10.18653/v1/2022.acl-long.296 · DOI
  38. Ebrahimi, 2018, HotFlip: White-box adversarial examples for text classification, 31, 10.18653/v1/P18-2006 · DOI · Paper page
  39. Ethayarajh, 2021, Attention flows are Shapley Value explanations, 49, 10.18653/v1/2021.acl-short.8 · DOI
  40. Feder, 2021, CausaLM: Causal model explanation through counterfactual language models, Computational Linguistics, 47, 333, 10.1162/coli_a_00404 · DOI · Paper page
  41. Feng, 2018, Pathologies of neural models make interpretations difficult, 3719, 10.18653/v1/D18-1407 · DOI · Paper page
  42. Finlayson, 2021, Causal analysis of syntactic agreement mechanisms in neural language models, 1828, 10.18653/v1/2021.acl-long.144 · DOI · Paper page
  43. Gardner, 2020, Evaluating models’ local decision boundaries via contrast sets, 1307, 10.18653/v1/2020.findings-emnlp.117 · DOI · Paper page
  44. Gupta, 2020, Neural module networks for reasoning over text
  45. Haghighatkhah, 2022, Better hit the nail on the head than beat around the bush: Removing protected attributes with a single projection, 8395, 10.18653/v1/2022.emnlp-main.575 · DOI
  46. Halpern, 2005, Causes and explanations: A structural-model approach. Part I: Causes, The British Journal for the Philosophy of Science, 56, 843, 10.1093/bjps/axi147 · DOI · Paper page
  47. Hamilton, 2022, Is neuro-symbolic AI meeting its promises in natural language processing? A structured review, 1, 10.3233/SW-223228 · DOI · Paper page
  48. Han, 2020, Explaining black box predictions and unveiling data artifacts through influence functions, 5553, 10.18653/v1/2020.acl-main.492 · DOI · Paper page
  49. Hao, 2021, Self-attention attribution: Interpreting information interactions inside Transformer, 12963, 10.1609/aaai.v35i14.17533 · DOI · Paper page
  50. Harrington, 1985, Harvey Friedman’s Research on the Foundations of Mathematics
  51. Hase, 2022, When can models learn from explanations? A formal framework for understanding the roles of explanation data, 29, 10.18653/v1/2022.lnls-1.4 · DOI · Paper page
  52. Hendrycks, 2021, Measuring massive multitask language understanding
  53. Herman, 2017, The promise and peril of human evaluation for model interpretability, ArXiv preprint
  54. Hiebert, 2018, Interpreting word-level hidden state behaviour of character-level LSTM language models, 258, 10.18653/v1/W18-5428 · DOI
  55. Hong, 2022, METGEN: A module-based entailment tree generation framework for answer explanation, 1887, 10.18653/v1/2022.findings-naacl.145 · DOI
  56. Hooker, 2019, A benchmark for interpretability methods in deep neural networks, 9734
  57. Hu, 2017, Learning to reason: End-to-end module networks for visual question answering, 804, 10.1109/ICCV.2017.93 · DOI · Paper page
  58. Jacovi, 2021, Aligning faithful interpretations with their social attribution, Transactions of the Association for Computational Linguistics, 9, 294, 10.1162/tacl_a_00367 · DOI · Paper page
  59. Jain, 2019, Attention is not explanation, 3543
  60. Jain, 2020, Learning to faithfully rationalize by construction, 4459, 10.18653/v1/2020.acl-main.409 · DOI · Paper page
  61. Janizek, 2021, Explaining explanations: Axiomatic feature interactions for deep networks, Journal of Machine Learning Research, 22, 104:1
  62. Jiang, 2020, Cold-start and interpretability: Turning regular expressions into trainable recurrent neural networks, 3193, 10.18653/v1/2020.emnlp-main.258 · DOI
  63. Jiang, 2019, Explore, propose, and assemble: An interpretable model for multi-hop reading comprehension, 2714, 10.18653/v1/P19-1261 · DOI · Paper page
  64. Ju, 2022, Logic traps in evaluating attribution scores, 5911, 10.18653/v1/2022.acl-long.407 · DOI
  65. Kádár, 2017, Representation of linguistic form and function in recurrent neural networks, Computational Linguistics, 43, 761, 10.1162/COLI_a_00300 · DOI · Paper page
  66. Karidi, 2021, Putting words in BERT’s mouth: Navigating contextualized vector spaces with pseudowords, 10300, 10.18653/v1/2021.emnlp-main.806 · DOI
  67. Kassner, 2021, BeliefBank: Adding memory to a pre-trained language model for a systematic notion of belief, 8849, 10.18653/v1/2021.emnlp-main.697 · DOI
  68. Kaushik, 2020, Learning the difference that makes a difference with counterfactually-augmented data
  69. Kaushik, 2018, How much reading does reading comprehension require? A critical investigation of popular benchmarks, 5010, 10.18653/v1/D18-1546 · DOI · Paper page
  70. Kindermans, 2019, The (un)reliability of saliency methods, 267, 10.1007/978-3-030-28954-6_14 · DOI · Paper page
  71. Kindermans, 2018, Learning how to explain neural networks: PatternNet and PatternAttribution
  72. Kokhlikyan, 2020, Captum: A unified and generic model interpretability library for PyTorch, ArXiv preprint
  73. Krishnamurthy, 2013, Jointly learning to parse and perceive: Connecting natural language to the physical world, Transactions of the Association for Computational Linguistics, 1, 193, 10.1162/tacl_a_00220 · DOI · Paper page
  74. Kumar, 2022, Probing classifiers are unreliable for concept removal and detection, Advances in Neural Information Processing Systems, 35, 17994
  75. Kumar, 2020, NILE: Natural language inference with faithful natural language explanations, 8730, 10.18653/v1/2020.acl-main.771 · DOI · Paper page
  76. Lakkaraju, 2020, “How do I fool you?”: Manipulating user trust via misleading black box explanations, 79, 10.1145/3375627.3375833 · DOI · Paper page
  77. Laugel, 2018, Defining locality for surrogates in post-hoc interpretablity
  78. Levesque, 2012, The Winograd Schema Challenge, 10
  79. Li, 2020, Evaluating explanation methods for neural machine translation, 365, 10.18653/v1/2020.acl-main.35 · DOI
  80. Li, 2016, Visualizing and understanding neural models in NLP, 681, 10.18653/v1/N16-1082 · DOI · Paper page
  81. Li, 2016, Understanding neural networks through representation erasure, ArXiv preprint
  82. Lipton, 2016, The Mythos of Model Interpretability, ArXiv preprint
  83. Liu, 2022, Rethinking attention-model explainability through faithfulness violation test, 13807
  84. Lovering, 2020, Information-theoretic probing explains reliance on spurious features
  85. Lu, 2021, Influence patterns for explaining information flow in BERT, 4461
  86. Madaan, 2021, Improving neural model performance through natural language feedback on their explanations, ArXiv preprint
  87. Marasovic, 2022, Few-shot self-rationalization with natural language prompts, 410, 10.18653/v1/2022.findings-naacl.31 · DOI · Paper page
  88. Martins, 2016, From softmax to sparsemax: A sparse model of attention and multi-label classification, 1614
  89. Ming, 2017, Understanding hidden memories of recurrent neural networks, 13, 10.1109/VAST.2017.8585721 · DOI · Paper page
  90. Montavon, 2019, Layer-wise relevance propagation: An overview, 193, 10.1007/978-3-030-28954-6_10 · DOI · Paper page
  91. Mosca, 2022, SHAP-based explanation methods: A review for NLP interpretability, 4593
  92. Mylonas, 2022, An attention matrix for every decision: Faithfulness-based arbitration among multiple attention-based interpretations of transformers in text classification, Data Mining and Knowledge Discovery, 38, 128, 10.1007/s10618-023-00962-4 · DOI
  93. Narang, 2020, WT5?! Training Text-to-Text Models to Explain their predictions, ArXiv preprint
  94. Nie, 2018, A theoretical explanation for perplexing behaviors of backpropagation-based visualizations, 3806
  95. Nye, 2021, Show your work: Scratchpads for intermediate computation with language models, Deep Learning for Code Workshop
  96. OpenAI. 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774. · arXiv
  97. Park, 2018, Multimodal explanations: Justifying decisions and pointing to the evidence, 8779, 10.1109/CVPR.2018.00915 · DOI · Paper page
  98. Pascual, 2021, Telling BERT’s full story: From local attention to global aggregation, 105, 10.18653/v1/2021.eacl-main.9 · DOI
  99. Pezeshkpour, 2021, An empirical comparison of instance attribution methods for NLP, 967, 10.18653/v1/2021.naacl-main.75 · DOI
  100. Poerner, 2018, Interpretable textual neuron representations for NLP, 325, 10.18653/v1/W18-5437 · DOI
  101. Poerner, 2018, Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement, 340, 10.18653/v1/P18-1032 · DOI · Paper page
  102. Poliak, 2018, Hypothesis only baselines in natural language inference, 180, 10.18653/v1/S18-2023 · DOI · Paper page
  103. Pruthi, 2022, Evaluating explanations: How much do explanations from the teacher aid students?, Transactions of the Association for Computational Linguistics, 10, 359, 10.1162/tacl_a_00465 · DOI · Paper page
  104. Qian, 2022, Limitations of language models in arithmetic and symbolic induction, 10.18653/v1/2023.acl-long.516 · DOI
  105. Qian, 2016, Analyzing linguistic knowledge in sequential model of sentence, 826, 10.18653/v1/D16-1079 · DOI · Paper page
  106. Raganato, 2020, Fixed encoder self-attention patterns in transformer-based machine translation, 556, 10.18653/v1/2020.findings-emnlp.49 · DOI · Paper page
  107. Rajagopal, 2021, SELFEXPLAIN: A self-explaining architecture for neural text classifiers, 836, 10.18653/v1/2021.emnlp-main.64 · DOI · Paper page
  108. Rajani, 2019, Explain yourself! Leveraging language models for commonsense reasoning, 4932, 10.18653/v1/P19-1487 · DOI · Paper page
  109. Ramamurthy, 2020, Model agnostic multilevel explanations, 5968
  110. Ravfogel, 2022, Log-linear guardedness and its implications, 9413, 10.18653/v1/2023.acl-long.523 · DOI
  111. Reif, 2019, Visualizing and measuring the geometry of BERT, 8592
  112. Roese, 1995, Counterfactual thinking: A critical overview, 1
  113. Sajjad, 2022, Neuron-level interpretation of deep NLP models: A survey, Transactions of the Association for Computational Linguistics, 10, 1285, 10.1162/tacl_a_00519 · DOI
  114. Schwartz, 2018, Bridging CNNs, RNNs, and weighted finite-state machines, 295, 10.18653/v1/P18-1028 · DOI
  115. Shrikumar, 2017, Not just a black box: Learning important features through propagating activation differences, ArXiv preprint, arXiv:1605.01713 [cs] · arXiv
  116. Sia, 2022, Logical satisfiability of counterfactuals for faithful explanations in NLI, 9837, 10.1609/aaai.v37i8.26174 · DOI
  117. Strobelt, 2019, Seq2seq-Vis: A visual debugging tool for sequence-to-sequence models, IEEE Transactions on Visualization and Computer Graphics, 25, 353, 10.1109/TVCG.2018.2865044 · DOI · Paper page
  118. Strobelt, 2018, LSTMVis: A tool for visual analysis of hidden state dynamics in recurrent neural networks, IEEE Transactions on Visualization and Computer Graphics, 24, 667, 10.1109/TVCG.2017.2744158 · DOI · Paper page
  119. Subramanian, 2020, Obtaining faithful interpretations from compositional neural networks, 5594, 10.18653/v1/2020.acl-main.495 · DOI · Paper page
  120. Sundararajan, 2017, Axiomatic attribution for deep networks, 3319
  121. Sushil, 2018, Patient representation learning and interpretable evaluation using clinical notes, Journal of Biomedical Informatics, 84, 103, 10.1016/j.jbi.2018.06.016 · DOI · Paper page
  122. Tsang, 2020, How does this interaction affect me? Interpretable attribution for feature interactions, 6147
  123. Tutek, 2020, Staying true to your word: (How) can attention become explanation?, 131, 10.18653/v1/2020.repl4nlp-1.17 · DOI
  124. Vashishth, 2019, Attention interpretability across NLP tasks, ArXiv preprint
  125. Vaswani, 2017, Attention is all you need, 5998
  126. Veldhoen, 2016, Diagnostic classifiers: Revealing how neural networks process hierarchical structure, CoCo@ NIPS, 69
  127. Vig, 2019, Visualizing attention in transformer-based language representation models, ArXiv preprint
  128. Vig, 2020, Investigating gender bias in language models using causal mediation analysis, 12388
  129. Wallace, 2018, Interpreting neural networks with nearest neighbors, 136, 10.18653/v1/W18-5416 · DOI · Paper page
  130. Wallace, 2019, Universal adversarial triggers for attacking and analyzing NLP, 2153, 10.18653/v1/D19-1221 · DOI · Paper page
  131. Wallace, 2020, Interpreting predictions of NLP models, 20, 10.18653/v1/2020.emnlp-tutorials.3 · DOI
  132. Wang, 2020, Gradient-based analysis of NLP models is manipulable, 247, 10.18653/v1/2020.findings-emnlp.24 · DOI · Paper page
  133. Wang, 2022, A fine-grained interpretability evaluation benchmark for neural NLP, 70, 10.18653/v1/2022.conll-1.6 · DOI
  134. Wexler, 2020, The What-If Tool: Interactive probing of machine learning models, IEEE Transactions on Visualization and Computer Graphics, 26, 56, 10.1109/TVCG.2019.2934619 · DOI · Paper page
  135. Wiegreffe, 2022, Reframing human-AI collaboration for generating free-text explanations, 632, 10.18653/v1/2022.naacl-main.47 · DOI · Paper page
  136. Wu, 2021, Polyjuice: Generating counterfactuals for explaining, evaluating, and improving models, 6707, 10.18653/v1/2021.acl-long.523 · DOI · Paper page
  137. Xie, 2017, An interpretable knowledge transfer model for knowledge base completion, 950, 10.18653/v1/P17-1088 · DOI · Paper page
  138. Yang, 2019, Benchmarking attribution methods with relative feature importance, arXiv preprint arXiv:1907.09701 · arXiv
  139. Ye, 2023, Explanation Selection Using Unlabeled Data for In-Context Learning, 10.18653/v1/2023.emnlp-main.41 · DOI
  140. Ye, 2022, Complementary explanations for effective in-context learning, 4469, 10.18653/v1/2023.findings-acl.273 · DOI · Paper page
  141. Ye, 2021, Connecting attributions and QA model behavior on realistic counterfactuals, 5496, 10.18653/v1/2021.emnlp-main.447 · DOI
  142. Yeh, 2019, On the (in)fidelity and sensitivity of explanations, 10965
  143. Yeh, 2020, On completeness-aware concept-based explanations in deep neural networks, 20554
  144. Yi, 2018, Neural-symbolic VQA: Disentangling reasoning from vision and language understanding, 1039
  145. Yin, 2022, On the sensitivity and stability of model interpretations in NLP, 2631, 10.18653/v1/2022.acl-long.188 · DOI
  146. Yin, 2022, Interpreting language models with contrastive explanations, 184, 10.18653/v1/2022.emnlp-main.14 · DOI · Paper page
  147. Zaidan, 2007, Using “Annotator Rationales” to Improve Machine Learning for Text Categorization, 260
  148. Zheng, 2022, The irrationality of neural rationale models, 64, 10.18653/v1/2022.trustnlp-1.6 · DOI
  149. Zhou, 2022, Do feature attribution methods correctly attribute features?, 9623, 10.1609/aaai.v36i9.21196 · DOI · Paper page
  150. Zhou, 2022, ExSum: From local explanations to model understanding, 5359, 10.18653/v1/2022.naacl-main.392 · DOI
  151. Zhou, 2023, The Solvability of Interpretability Evaluation Metrics, 2399, 10.18653/v1/2023.findings-eacl.182 · DOI
  152. Zmigrod, 2019, Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology, 1651, 10.18653/v1/P19-1161 · DOI · Paper page

Cited by

See all 120

Metadata

Authors
Lyu, Q., Apidianaki, M., & Callison-Burch, C.
Affiliations
  • University of Pennsylvania, Department of Computer and Information Science. lyuqing@sas.upenn.edu
  • University of Pennsylvania, Department of Computer and Information Science. marapi@seas.upenn.edu
  • University of Pennsylvania, Department of Computer and Information Science. ccb@seas.upenn.edu
Venue
Computational Linguistics
Publisher
MIT Press
Published
2024
Volume
50
Issue
2
Pages
657-723
ISSN
  • 0891-2017
  • 1530-9312
Cited by
120
References
102
Language
en

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.