What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models
Open accessThe DOI matches its Crossref record; 36 of the 38 references it cites were found in a registry or an index.
Abstract
Pre-training by language modeling has become a popular and successful approach to NLP tasks, but we have yet to understand exactly what linguistic capacities these pre-training processes confer upon models. In this paper we introduce a suite of diagnostics drawn from human language experiments, which allow us to ask targeted questions about information used by language models for generating predictions in context. As a case study, we apply these diagnostics to the popular BERT model, finding that it can generally distinguish good from bad completions involving shared category or role reversal, albeit with less sensitivity than humans, and it robustly retrieves noun hypernyms, but it struggles with challenging inference and role-based event prediction— and, in particular, it shows clear insensitivity to the contextual impacts of negation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The second trend is that neural networks routinely fail on exactly the phenomena where such symbolic models succeed, for example, logical inference [6,9], compositionality [7,23] and out-of-distribution generalization [24–26].”
From Symbols and grounding in large language models (2023) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Besides factual knowledge, large-scale pre-training also allows LMs to handle linguistic phenomena such as analogies [9], negations [25], semantic role sensitivity [25], semantic similarity [131], cant understanding [131], and rare word understanding [116].”
From Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2023) · cites this paper as [25]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For these kinds of tasks, we can also score the generation probability of each candidate and choose the one with the highest probability [25].”
From Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2023) · cites this paper as [25]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2×
“BERT is a deep bidirectional transformer network (Vaswani et al., 2017) pre-trained on tasks of masked language modeling (predicting masked words given bidirectional context) and next-sentence prediction (binary classification of whether two sentences are a sequence).”
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“As a case study, we use these three diagnostics to examine the predictive capacities of the pre-trained BERT model (Devlin et al., 2019), which has been the basis of impressive performance across a wide range of tasks.”
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · cited 3×
“, 2019) and ELMo (Peters et al., 2018a) have advanced the state of the art in a wide variety of tasks, suggesting that these models acquire valuable, generalizable linguistic competence during the pre-training process.”
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222 · cited 2×
“Much work has attempted to evaluate systems’ overall level of ‘‘understanding’’, often with tasks such as semantic similarity and entailment (Wang et al., 2018; Bowman et al., 2015; Agirre et al., 2012; Dagan et al., 2005; Bentivogli et al., 2016), and additional work has been done to design curated versions of these…”
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377
Brain potentials during reading reflect word expectancy and semantic association
Kutas, M., & Hillyard, S. A. (1984) · Nature · doi:10.1038/307161a0 · Cited by 2,218 · cited 1×
“Like cloze, the N400 can be used to gauge how expected a word w is in a context c —the amplitude of the N400 response appears to be sensitive to fit of a word in context, and has been shown to correlate with cloze in many cases (Kutas and Hillyard, 1984).”
MethodWhat Does BERT Look at? An Analysis of BERT’s Attention
Clark, K., Khandelwal, U., Levy, O., & Manning, C. D. (2019) · Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w19-4828 · Cited by 1,373
BERT Rediscovers the Classical NLP Pipeline
Tenney, I., Das, D., & Pavlick, E. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1452 · Cited by 1,307
SemEval-2012 Task 6: A Pilot on Semantic Textual Similarity
Agirre, E., Cer, D. M., Diab, M., & González-Agirre, A. (2012) · Joint Conference on Lexical and Computational Semantics · Cited by 1,063
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
McCoy, R. T., Pavlick, E., & Linzen, T. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1334 · Cited by 944
Research works printed but not linked: 19
- Adi Yossi, 2016, International Conference on Learning Representations
- Agirre Eneko, 2012, Proceedings of the First Joint Conference on Lexical and Computational Semantics-Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation, 385
- Chowdhury Shammur Absar, 2018, Proceedings of the 27th International Conference on Computational Linguistics, 133
- Clark Kevin, 2019, arXiv preprint arXiv:1906.04341 · arXiv
- Conneau Alexis, 2018, ACL 2018-56th Annual Meeting of the Association for Computational Linguistics, 2126
- Dagan Ido, 2005, Machine Learning Challenges Workshop, 177
- Dasgupta Ishita, 2018, Proceedings of the 40th Annual Meeting of the Cognitive Science Society
- Devlin Jacob, 2019, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Ettinger Allyson, 2018, Proceedings of the 27th International Conference on Computational Linguistics, 1790
- Frank Stefan L., 2013, ACL 2013-51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, 2, 878
- Futrell Richard, 2019, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 32
- Giulianelli Mario, 2018, Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, 240, 10.18653/v1/W18-5426 · DOI
- Gulordava Kristina, 2018, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), 1, 1195
- Kim Najoung, 2019, Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), 235
- Peters Matthew, 2018, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), 1, 2227
- Tenney Ian, 2019, arXiv preprint arXiv:1905.05950 · arXiv
- Tenney Ian, 2019, Proceedings of the International Conference on Learning Representations
- Vaswani Ashish, 2017, Advances in Neural Information Processing Systems, 5998
- Wang Alex, 2018, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 353
Cited by
Items 101–200 of 553
Modeling Structure‐Building in the Brain With CCG Parsing and Large Language Models
Stanojević, M., Brennan, J., Dunagan, D. G., Steedman, M. J., & Hale, J. (2023) · Cognitive Science · doi:10.1111/cogs.13312 · Cited by 62
Compositional and Lexical Semantics in RoBERTa, BERT and DistilBERT: A Case Study on CoQA
Staliūnaitė, I., & Iacobacci, I. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.573 · Cited by 50
Wright, A. P., Shaikh, O., Park, H., Epperson, W., Ahmed, M., Pinel, S., Chau, D. H., & Yang, D. (2021) · Proc. ACM Hum. Comput. Interact. · doi:10.1145/3449280 · Cited by 35
“Issues include bias [1, 33, 40], as well as a general lack of semantic understanding [11].”
BackgroundAn Analysis of Negation in Natural Language Understanding Corpora
Hossain, M. M., Chinnappa, D., & Blanco, E. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.81 · Cited by 67
On the Blind Spots of Model-Based Evaluation Metrics for Text Generation
He, T., Zhang, J., Wang, T., Kumar, S., Cho, K., Glass, J. R., & Tsvetkov, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.674 · Cited by 85
Impact of Co-occurrence on Factual Knowledge of Large Language Models
Kang, C., & Choi, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.518 · Cited by 78
Gubelmann, R. (2024) · Philosophy & Technology · doi:10.1007/s13347-024-00696-1 · Cited by 36
“On the other hand, they still struggle with somewhat simple semantic structures as negations, see Warstadt et al. (2020); Ettinger (2020); Kassner and Schütze (2020); Gubelmann and Handschuh (2022), and they find it difficult to master both inductive and deductive modes of inferences, see Gubelmann…”
Pandia, L., & Ettinger, A. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.119 · Cited by 22
“, 2020) and semantic/pragmatic phenomena (Ettinger, 2020).”
Self-citationKumar, Y., Parekh, S., Singh, S., Li, J. J., Shah, R. R., & Chen, C. (2023) · Dialogue & Discourse · doi:10.5210/dad.2023.101 · Cited by 45
“Ettinger (2020) in their ‘negation test’ also observe similar results to us.”
ResultDeriving Contextualised Semantic Features from BERT (and Other Transformer Model) Embeddings
Turton, J., Smith, R. E., & Vinson, D. (2021) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2021.repl4nlp-1.26 · Cited by 46
Ryu, S. H., & Lewis, R. L. (2021) · Workshop on Cognitive Modeling and Computational Linguistics (CMCL) · doi:10.18653/v1/2021.cmcl-1.6 · Cited by 64
Language models are not naysayers: an analysis of language models on negation benchmarks
Truong, T. H., Baldwin, T., Verspoor, K., & Cohn, T. (2023) · arXiv · doi:10.48550/arxiv.2306.08189 · Cited by 34
Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge
Chen, J., Shi, W., Fu, Z., Cheng, S., Li, L., & Xiao, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.550 · Cited by 65
Hessel, J., & Schofield, A. (2021) · doi:10.18653/v1/2021.acl-short.27 · Cited by 60
“Ettinger (2020) show that shuffling BERT inputs decreases word cloze prediction performance on a corpus of 102 sentences without fine-tuning.”
BackgroundCONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation
Ravichander, A., Gardner, M., & Marasović, A. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.598 · Cited by 62
Probing Neural Language Models for Human Tacit AssumptionsPreprint
Weir, N., Poliak, A., & Durme, B. V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.04877 · Cited by 36
“, a task issue noted by Ettinger (2020).”
BackgroundMethodAwakening Latent Grounding from Pretrained Language Models for Semantic Parsing
Liu, Q., Yang, D., Zhang, J., Guo, J., Zhou, B., & Lou, J. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.100 · Cited by 50
“As for semantic knowledge, although it is less explored than syntactic knowledge, previous work showed that BERT contained some semantic information, such as entity types (Ettinger, 2020), semantic roles (Tenney et al., 2019) and factual knowledge (Petroni et al., 2019).”
BackgroundMethodExplainable Natural Language Processing
Søgaard, A. (2021) · Synthesis lectures on human language technologies · doi:10.2200/s01118ed1v01y202107hlt051 · Cited by 38
Trott, S., Torrent, T. T., Chang, N., & Schneider, N. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.462 · Cited by 37
Scope Ambiguities in Large Language Models
Kamath, G., Schuster, S., Vajjala, S., & Reddy, S. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00670 · Cited by 61
“…as Q&A. 4 Instead, following work that brings psycholinguistic methods to language model analysis (see Linzen et al., 2016; Futrell et al., 2019; Ettinger, 2020; Baroni, 2022; Schuster and Linzen, 2022), we investigate the question by observing the probabilities a model assigns to different…”
BackgroundInfluentialEvaluation Benchmarks and Learning Criteria for Discourse-Aware Sentence Representations
Chen, M., Chu, Z., & Gimpel, K. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1060 · Cited by 54
“Benchmark datasets (Adi et al., 2017; Conneau and Kiela, 2018; Wang et al., 2018a, 2019) have been proposed to evaluate the encoded knowledge, where the focus has been primarily on natural language understanding capabilities of the representation of a stand-alone sentence, such as its semantic…”
BackgroundThe World of an Octopus: How Reporting Bias Influences a Language Model’s Perception of Color
Paik, C., Aroca-Ouellette, S., Roncone, A., & Kann, K. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.63 · Cited by 47
“Probing LMs A wide range of papers have probed LMs in a zero-shot fashion by looking at how they fill in a [MASK] token in handcrafted (Weir et al., 2020; Petroni et al., 2019; Jiang et al., 2020; Ettinger, 2020; Lin et al., 2020) or automatically generated (Bouraoui et al.”
BackgroundCan Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal View
Cao, B., Lin, H., Han, X., Liu, F., & Sun, L. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.398 · Cited by 20
Do language models learn typicality judgments from text?Preprint
Misra, K., Ettinger, A., & Rayz, J. T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.02987 · Cited by 31
“Our choice of LMs is motivated by recent evidence that shows qualitative alignment of category knowledge (“a robin is a bird”, “a bear has fur, has claws.”) in pre-trained LMs (Ettinger, 2020; Weir et al., 2020).”
BackgroundInfluentialSelf-citationNegation mitigates rather than inverts the neural representations of adjectives
Zuanazzi, A., Ripollés, P., Lin, W. M., Gwilliams, L., King, J.-R., & Poeppel, D. (2024) · PLoS Biology · doi:10.1371/journal.pbio.3002622 · Cited by 31
“The longer RTs and lower feedback score for negated phases shown in Experiment 1 ( Fig 2A ), in the replication experiment ( Fig 3A ), and in Experiment 2, are consistent with data demonstrating that negation incurs increased processing costs [13–18 ,27 ,32].”
Jump-Starting Item Parameters for Adaptive Language Tests
McCarthy, A. D., Yancey, K. P., LaFlair, G. T., Egbert, J., Liao, M., & Settles, B. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.67 · Cited by 56
“For passage featurization, we use a Transformer-based passage representation using BERT (Devlin et al., 2019), which is able to implicitly represent linguistic content of its input text (Tenney et al., 2019; Ettinger, 2020).”
MethodFact-Driven Logical Reasoning for Machine Reading Comprehension
Ouyang, S., Zhang, Z., & Zhao, H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29850 · Cited by 29
“However, according to diagnostic tests (Et-tinger 2020; Rogers, Kovaleva, and Rumshisky 2020), PLMs like BERT (Devlin et al. 2019), despite encoding syntactic and semantic information through large-scale pre-training, they tend to struggle with understanding role reversal and pragmatic inference,…”
MethodFiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation
Lakhotia, K., Paranjape, B., Ghoshal, A., Yih, S., Mehdad, Y., & Iyer, S. V. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.301 · Cited by 38
“…al., 2018; He-witt and Manning, 2019; Lin et al., 2019) and behavioral probing methods (McCoy et al., 2020; Goldberg, 2019; Warstadt et al., 2019; Ettinger, 2020), as well as input saliency maps to highlight the most important tokens/sentences in the input for each prediction (Serrano and Smith,…”
BackgroundOn the Gap between Adoption and Understanding in NLP
Bianchi, F., & Hovy, D. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.340 · Cited by 35
“E.g., Ettinger (2020) shows that BERT has very low sensitivity to the concept of negation shows.”
BackgroundCloze Distillation: Improving Neural Language Models with Human Next-Word Prediction
Eisape, T., Zaslavsky, N., & Lévy, R. (2020) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2020.conll-1.49 · Cited by 47
Contextualized Word Embeddings Encode Aspects of Human-Like Word Sense KnowledgePreprint
Nair, S., Srinivasan, M., & Meylan, S. C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13057 · Cited by 49
“Ettinger (2020) points out the need for more psycholinguistic diagnostics of neural language models like BERT, and compares BERT to human data from tasks such as commonsense inference.”
MethodThe Singleton Fallacy: Why Current Critiques of Language Models Miss the Point
Sahlgren, M., & Carlsson, F. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.682578 · Cited by 34
“Even if current language models have been shown to underperform in some specific test settings (such as their ability to handle negation [9]), there is an overwhelming body of empirical results to demonstrate that current language models have a passable capacity to detect the symbols and rules that define human langua…”
BackgroundHow is BERT surprised? Layerwise detection of linguistic anomalies
Li, B., Zhu, Z., Thomas, G., Xu, Y., & Rudzicz, F. (2021) · doi:10.18653/v1/2021.acl-long.325 · Cited by 40
“A popular approach is to formulate the problem as a multiple-choice task, where the LM is considered correct if it assigns higher likelihood to the appropriate word than an inappropriate one, given context (Gulordava et al., 2018; Ettinger, 2020; Warstadt et al., 2020).”
BackgroundMethodA Brief Overview of Universal Sentence Representation Methods: A Linguistic View
Li, R., Zhao, X. H., & Moens, M. (2022) · ACM Computing Surveys · doi:10.1145/3482853 · Cited by 48
“Contextual word embeddings, similar to sentence embeddings, have been proved to contain semantic and syntactic features to some degree [39, 46, 52, 60, 65].”
BackgroundLi, J., & Ettinger, A. (2022) · Cognition · doi:10.1016/j.cognition.2022.105359 · Cited by 38
“…important to acknowledge that the exact information encoded in these sentence representations remains relatively opaque, and their ability to capture complex semantics is likely still limited (Ettinger, 2020; McCoy, Min, & Linzen, 2020; Min, McCoy, Das, Pitler, & Linzen, 2020; Yu & Ettinger, 2020).”
Self-citationNatural Language Processing for Breast Imaging: A Systematic Review
Diab, K. M., Deng, J., Wu, Y., Yesha, Y., Collado‐Mesa, F., & Nguyen, P. (2023) · Diagnostics · doi:10.3390/diagnostics13081420 · Cited by 30
“Recent evidence shows that embeddings generated by BERT fail to show generalizable understanding of negation [57], an essential factor in interpreting radiology reports effectively.”
BackgroundAcquiring and modeling abstract commonsense knowledge via conceptualization
He, M., Fang, T., Wang, W., & Song, Y. (2024) · Artificial Intelligence · doi:10.1016/j.artint.2024.104149 · Cited by 41
“Nevertheless, they could still only capture limited aspects of commonsense [38, 39, 40, 41], often fail on negations [42], and suffer from reporting bias [23, 43].”
BackgroundCan Linguistic Knowledge Improve Multimodal Alignment in Vision-Language Pretraining?
Wang, F., Ding, L., Rao, J., Liu, Y., Shen, L., & Ding, C. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3690640 · Cited by 40
“Ettinger [15] conducted psychological tasks to evaluate the contextual information of BERT and discovered its insensitivity to negation, a characteristic we also observe in VLP models.”
The Sensitivity of Language Models and Humans to Winograd Schema Perturbations
Abdou, M., Ravishankar, V., Barrett, M., Belinkov, Y., Elliott, D., & Søgaard, A. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.679 · Cited by 38
Do PLMs Know and Understand Ontological Knowledge?
Wu, W., Jiang, C., Jiang, Y., Xie, P.-J., & Tu, K. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2023.acl-long.173 · Cited by 40
“Progress has been reported in using PLMs to perform reasoning tasks, including arithmetic (Wang et al., 2022; Wei et al., 2022), commonsense (Talmor et al., 2019, 2020; Wei et al., 2022), logical (Creswell et al., 2022) and symbolic reasoning (Wei et al., 2022).”
BackgroundMethodNeural reality of argument structure constructions
Li, B., Zhu, Z., Thomas, G., Rudzicz, F., & Xu, Y. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.512 · Cited by 64
Can language representation models think in bets?
Tang, Z., & Kejriwal, M. (2023) · Royal Society Open Science · doi:10.1098/rsos.221585 · Cited by 23
Morphosyntactic probing of multilingual BERT models
Ács, J., Hamerlik, E., Schwartz, R., Smith, N. A., & Kornai, A. (2023) · Natural Language Engineering · doi:10.1017/s1351324923000190 · Cited by 40
“It has been shown (Ettinger, 2020; Sinha et al., 2021) that BERT models are sensitive to word order in a variety of English and Mandarin tasks.”
PROST: Physical Reasoning about Objects through Space and Time
Aroca-Ouellette, S., Paik, C., Roncone, A., & Kann, K. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.404 · Cited by 52
Alajrami, A., & Αλέτρας, Ν. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.16 · Cited by 32
COCO: an annotated Twitter dataset of COVID-19 conspiracy theories
Langguth, J., Schroeder, D. T., Filkuková, P., Brenner, S., Phillips, J., & Pogorelov, K. (2023) · Journal of Computational Social Science · doi:10.1007/s42001-023-00200-3 · Cited by 42
“While BERT-based approaches [44] worked 1 3 Journal of Computational Social Science (2023) 6:443–484 reasonably well in the MediaEval2021 challenge, it was observed that BERT some-times struggles with negations [73] which are common in the Related category.”
MethodLanguage Models with Rationality
Kassner, N., Tafjord, O., Sabharwal, A., Richardson, K., Schuetze, H., & Clark, P. A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.877 · Cited by 50
Character-level Representations Improve DRS-based Semantic Parsing Even in the Age of BERT
van Noord, R., Toral, A., & Bos, J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.371 · Cited by 45
Michaelov, J. A., Bardolph, M. D., Coulson, S., & Bergen, B. K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.09648 · Cited by 39
“…the success T-LMs evince in predicting N400 amplitude (Merkx & Frank, 2020) is connected to other findings; for example, they appear to show semantic priming effects and, like the N400 (Nieuwland & Kuperberg, 2008), they are rather insensitive to negation (Ettinger, 2020; Misra et al., 2020).”
BackgroundResultInfluentialDoes Vision-and-Language Pretraining Improve Lexical Grounding?
Yun, T., Sun, C., & Pavlick, E. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.370 · Cited by 22
Mahowald, K. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.20 · Cited by 46
A Theoretically Grounded Question Answering Data Set for Evaluating Machine Common Sense
Santos, H., Shen, K., Mulvehill, A. M., Kejriwal, M., & McGuinness, D. L. (2023) · Data Intelligence · doi:10.1162/dint_a_00234 · Cited by 21
“Indeed, some recent research has shown that, in carefully constructed experiments, language models may have trouble with concepts such as rationality and negation [50, 51].”
BackgroundLanguage Models Use Monotonicity to Assess NPI Licensing
Jumelet, J., Denić, M., Szymanik, J., Hupkes, D., & Steinert‐Threlkeld, S. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.439 · Cited by 48
Geographic Adaptation of Pretrained Language Models
Hofmann, V., Glavaš, G., Ljubešić, N., Pierrehumbert, J. B., & Schütze, H. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00652 · Cited by 26
“…al., 2023): morphological (Edmiston, 2020; Hofmann et al., 2020; Weissweiler et al., 2023), lexical (Ethayarajh, 2019; Vuli´c et al., 2020), syntactic (Hewitt and Manning, 2019; Jawahar et al., 2019; Wei et al., 2021; Weissweiler et al., 2022), and semantic (Wiedemann et al., 2019; Ettinger, 2020).”
Context Matters: A Pragmatic Study of PLMs’ Negation Understanding
Gubelmann, R., & Handschuh, S. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.315 · Cited by 36
“These results put the pioneering findings by Ettinger (2020), Kassner and Schütze (2020), Warstadt et al. (2020), and Ribeiro et al.”
BackgroundResultInfluentialThis is not correct! Negation-aware Evaluation of Language Generation Systems
Anschütz, M., Lozano, D. M., & Groh, G. (2023) · International Conference on Natural Language Generation · doi:10.18653/v1/2023.inlg-main.12 · Cited by 37
AND does not mean OR: Using Formal Languages to Study Language Models’ Representations
Traylor, A., Feiman, R., & Pavlick, E. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) · doi:10.18653/v1/2021.acl-short.21 · Cited by 19
“…that assesses whether models can perform symbolic reasoning about language (Kassner et al., 2020) e.g., quantifiers or negation (Talmor et al., 2020; Ettinger, 2020; Kassner and Sch¨utze, 2020; Wang et al., 2018) or by measuring the sys-tematicity of models’ inferences (Goodwin et al., 2020; Kim…”
BackgroundIncorporating Distributions of Discourse Structure for Long Document Abstractive Summarization
Pu, D., Wang, Y., & Demberg, V. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.306 · Cited by 39
Scientific and Creative Analogies in Pretrained Language Models
Czinczoll, T., Yannakoudakis, H., Mishra, P., & Shutova, E. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.153 · Cited by 19
Enhancing inter-sentence attention for Semantic Textual Similarity
Zhao, Y., Xia, T., Jiang, Y., & Tian, Y. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103535 · Cited by 25
Does BERT agree? Evaluating knowledge of structure dependence through agreement relationsPreprint
Bacon, G., & Regier, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.09892 · Cited by 20
McCoy, R. T., Min, J., & Linzen, T. (2020) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.4110592 · Cited by 16
Cassani, G., Bianchi, F., Attanasio, G., Marelli, M., & Günther, F. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/b45ys · Cited by 23
The Life Cycle of Knowledge in Big Language Models: A Survey
Cao, B., Lin, H., Han, X., & Sun, L. (2023) · arXiv · doi:10.48550/arxiv.2303.07616 · Cited by 18
“…Htut et al., 2019; Hewitt and Manning, 2019; Goldberg, 2019; Warstadt et al., 2019), semantic knowledge (Tenney et al., 2019; Wallace et al., 2019; Ettinger, 2020) and world knowledge (Davison et al., 2019; Bouraoui et al., 2020; Forbes et al., 2019; Zhou et al., 2020b; Roberts et al., 2020; Lin…”
BackgroundMethodInfluentialThe Life Cycle of Knowledge in Big Language Models: A Survey
Cao, B., Lin, H., Han, X., & Sun, L. (2024) · Machine Intelligence Research · doi:10.1007/s11633-023-1416-x · Cited by 18
“…Htut et al., 2019; Hewitt and Manning, 2019; Goldberg, 2019; Warstadt et al., 2019), semantic knowledge (Tenney et al., 2019; Wallace et al., 2019; Ettinger, 2020) and world knowledge (Davison et al., 2019; Bouraoui et al., 2020; Forbes et al., 2019; Zhou et al., 2020b; Roberts et al., 2020; Lin…”
BackgroundMethodInfluentialNarrative event segmentation in the cortical reservoir
Dominey, P. F. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008993 · Cited by 21
Apidianaki, M., & Soler, A. G. (2021) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2021.blackboxnlp-1.7 · Cited by 18
Analyzing BERT’s Knowledge of Hypernymy via Prompting
Hanna, M., & Mareček, D. (2021) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2021.blackboxnlp-1.20 · Cited by 40
“The “such-as” prompt is more effective than the prior two: its precision is 18 points higher than the basic prompt, and its MRR is 6 points higher; more-over, its WordNet P@1 is the highest of all prompts using only one hyponym.”
BackgroundInfluentialSchuster, S., & Linzen, T. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.71 · Cited by 15
Brown, K., Yee, E., Joergensen, G. H., Troyer, M., Saltzman, E. L., Rueckl, J. G., Magnuson, J. S., & McRae, K. (2023) · Cognitive Science · doi:10.1111/cogs.13291 · Cited by 19
“BERT learns some syntactic information (Liu, Gardner, Belinkov, Peters, & Smith, 2019; Tenney et al., 2019) and semantic roles (Ettinger, 2020; Ten-ney et al., 2019); however, this knowledge has been revealed using either cloze tasks (Taylor, 1953) or by learned classification using BERT’s…”
MethodLanguage-specific representation of emotion-concept knowledge causally supports emotion inference
Li, M., Su, Y.-S., Huang, H.-Y., Cheng, J., Hu, X., Zhang, X., Wang, H., Qin, Y., Wang, X., Lindquist, K. A., et al. (2024) · iScience · doi:10.1016/j.isci.2024.111401 · Cited by 30
Bertinho: Galician BERT RepresentationsPreprint
Vilares, D., García, M., & Gómez‐Rodríguez, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.13799 · Cited by 28
“Moreover, it has attracted the interest of the research community regarding how its deep architecture encodes linguistic knowledge (Lin, Tan, and Frank, 2019; Vilares et al., 2020; Ettinger, 2020).”
BackgroundCharacter-based Neural Semantic Parsing
van Noord, R. (2021) · University of Groningen research database (University of Groningen / Centre for Information Technology) · doi:10.33612/diss.169308968 · Cited by 18
Shen, K., & Kejriwal, M. (2023) · PLoS ONE · doi:10.1371/journal.pone.0295925 · Cited by 17
Clinical Concept-Based Radiology Reports Classification Pipeline for Lung Carcinoma
Mithun, S., Jha, A. K., Sherkhane, U. B., Jaiswar, V., Purandare, N. C., Dekker, A. L. A. J., Puts, S., Bermejo, I., Rangarajan, V., Zegers, C. M. L., et al. (2023) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-023-00787-z · Cited by 18
“It has also been shown to perform poorly with pragmatic inference [86].”
BackgroundValdez, D., Mena‐Meléndez, L., Crawford, B. L., Arvind, A., & Jozkowski, K. N. (2023) · Sexuality Research and Social Policy · doi:10.1007/s13178-023-00892-2 · Cited by 17
“By extension, S-BERT (and the larger BERT family) are supervised algorithms trained on human language (Ettinger, 2020).”
A howling success or a working sea? Testing what BERT knows about metaphors
Pedinotti, P., Palma, E. D., Cerini, L., & Lenci, A. (2021) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2021.blackboxnlp-1.13 · Cited by 30
“Even if some recent work has approached the interpretation task (Su et al., 2016; Mao et al., 2018; Rosen, 2018), much of the literature in the last years has been devoted to metaphor identification (Leong et al., 2018; Gao et al., 2018; Dankers et al., 2019, 2020; Leong et al., 2020).”
MethodResultLocal Structure Matters Most: Perturbation Study in NLU
Clouâtre, L., Parthasarathi, P., Zouaq, A., & Chandar, S. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.293 · Cited by 30
“Ettinger (2020); Pham et al. (2019); Sinha et al. (2020); Gupta et al. (2021) show that large language models are insensitive to minor perturbations highlighting the lack of syntactic knowledge used in syntax rich NLP tasks.”
BackgroundScoNe: Benchmarking Negation Reasoning in Language Models With Fine-Tuning and In-Context Learning
She, J. S., Potts, C., Bowman, S. R., & Geiger, A. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.154 · Cited by 34
“For instance, Cooper et al. (1996), McCoy and Linzen (2018), Wang et al. (2019), Ettinger (2020), Hartmann et al. (2021), and Kassner and Schütze (2020) all conduct evaluations with minimal pairs of examples that are identical except for a negative morpheme.”
BackgroundConsolidating Commonsense KnowledgePreprint
Ilievski, F., Szekely, P. A., Cheng, J., Zhang, F., & Qasemi, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06114 · Cited by 17
“steps [Richardson and Sabharwal, 2019], role-based event prediction [Ettinger, 2020], as well as numeric, emotional, and spatial inference [Bhagavatula et al.”
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models
Zhang, Y., Yasunaga, M., Zhou, Z., HaoChen, J. Z., Zou, J., Liang, P., & Yeung, S. Y. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.472 · Cited by 28
Zeng, F., & Gao, W. G. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.278 · Cited by 40
Michaelov, J. A., & Bergen, B. K. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.891 · Cited by 18
Filtered Corpus Training (FiCT) Shows that Language Models Can Generalize from Indirect Evidence
Patil, A., Jumelet, J., Chiu, Y. Y., Lapastora, A., Shen, P. S., Wang, L., Willrich, C., & Steinert‐Threlkeld, S. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00720 · Cited by 43
“…of-speech, morphological features, and syntactic dependency annotations generated via the use of Stanza (Qi et al., 2020), an off-the-shelf package that uses pretrained neural models to generate grammatical annotations within the framework of Universal Dependencies (UD) (Nivre et al., 2017, 2020).”
Dentella, V., Günther, F., & Leivada, E. (2025) · PLoS ONE · doi:10.1371/journal.pone.0327794 · Cited by 22
“In this line of research, the language capabilities of LLMs are often evaluated through obtaining direct measurements of the probabilities they assign over minimal pairs of sentences, one of which is grammatical and the other one ungrammatical (Hu & Levy, 2023).”
BackgroundExplaining Contextualization in Language Models using Visual Analytics
Sevastjanova, R., Kalouli, A.-L., Beck, C., Schäfer, H., & El‐Assady, M. (2021) · doi:10.18653/v1/2021.acl-long.39 · Cited by 27
“…that BERT’s transformer architecture encodes semantic information such as word senses and semantic roles (Reif et al., 2019; Tenney et al., 2019b; Ettinger, 2020; Zhao et al., 2020), syntactic information in the form of constituents and hierarchical structure (Goldberg, 2019; Hewitt and Manning,…”
BackgroundShapley Idioms: Analysing BERT Sentence Embeddings for General Idiom Token Identification
Nedumpozhimana, V., Klubička, F., & Kelleher, J. D. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.813967 · Cited by 16
“…chunks, and roles (Liu et al., 2019) and subject-predicate agreement (van Schijndel et al., 2019); and semantic knowledge, such as semantic roles (Ettinger, 2020), and entity types and relations (Tenney et al., 2019); and we direct the reader to Rogers et al. (2020) for a comprehensive recent…”
BackgroundAlleman, M., Mamou, J., Rio, M. A. D., Tang, H., Kim, Y., & Chung, S. (2021) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2021.repl4nlp-1.27 · Cited by 23
The bewitching AI: The Illusion of Communication with Large Language Models
Bottazzi Grifoni, E., & Ferrario, R. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00893-6 · Cited by 15
“The problem of inconsistencies in language models remains unsolved (Jang & Lukasiewicz, 2023), with negation handling limitations noted by researchers (Ettinger, 2020; Kassner & Schutze, 2020).”
Learn to Understand Negation in Video Retrieval
Wang, Z., Chen, A., Hu, F., & Li, X. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547968 · Cited by 24
Evaluating deep generative models on cognitive tasks: a case study
Tang, Z., & Kejriwal, M. (2023) · Discover Artificial Intelligence · doi:10.1007/s44163-023-00067-3 · Cited by 16
“Perhaps because of its success, researchers have been attempting to empirically understand the properties (including biases and blind spots [4]) of even early transformer models, such as BERT, along multiple dimensions [5–7].”
BackgroundBautista, Y. J. P., Theran, C., Aló, R., & Lima, V. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-47454-5_17 · Cited by 18
Dove, G. O. (2024) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2023.0149 · Cited by 21
“For example, some systems containing LLMs have been found to struggle with negation [18–20].”
Static Embeddings as Efficient Knowledge Bases?
Dufter, P., Kassner, N., & Schütze, H. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.186 · Cited by 25
Semantically Distributed Robust Optimization for Vision-and-Language Inference
Gokhale, T., Chaudhary, A., Banerjee, P., Baral, C. R., & Yang, Y. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.118 · Cited by 16
Help! Need Advice on Identifying Advice
Govindarajan, V. S., Chen, B. T., Warholic, R., Erk, K., & Li, J. J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.427 · Cited by 20
“However, these same models have been found to struggle at tasks that require higher-level processing (Ettinger, 2020), including giving advice (Zellers et al.”
BackgroundCommonsense Knowledge in WikidataPreprint
Ilievski, F., Szekely, P. A., & Schwabe, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.08114 · Cited by 18
“Still, they often exhibit shallow understanding of the world [8].”
Wilson, M., Petty, J., & Frank, R. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00608 · Cited by 42
“…(Chowdhury and Zamparelli, 2019; Hawkins et al., 2020b; Loáiciga et al., 2021; Metheniti et al., 2020; Petty et al., 2022; Warstadt and Bowman, 2020; Yoshida and Oseki, 2022), and whether LLMs distinguish plausible from implausible argument-role mappings in role-reversal sentences (Ettinger, 2020).”
Background
Metadata
- Authors
- Ettinger, A.
- Affiliations
- University of Chicago (US)
- Corresponding author
- Ettinger, Allyson
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2020-12
- Volume
- 8
- Pages
- 34-48
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W2965647350
- Cited by
- 511
- References
- 38
- Citations by year
- 2026: 16 · 2025: 32 · 2024: 39 · 2023: 122 · 2022: 92 · 2021: 151 · 2020: 53
- FWCI
- 41.25
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Social Sciences › Arts and Humanities › Language and Linguistics › Syntax, Semantics, Linguistic Variation
- Keywords
- BERT, language model pretraining, role reversal, negation, event prediction, language model evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.