What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models
Open accessThe DOI matches its Crossref record; 36 of the 38 references it cites were found in a registry or an index.
Abstract
Pre-training by language modeling has become a popular and successful approach to NLP tasks, but we have yet to understand exactly what linguistic capacities these pre-training processes confer upon models. In this paper we introduce a suite of diagnostics drawn from human language experiments, which allow us to ask targeted questions about information used by language models for generating predictions in context. As a case study, we apply these diagnostics to the popular BERT model, finding that it can generally distinguish good from bad completions involving shared category or role reversal, albeit with less sensitivity than humans, and it robustly retrieves noun hypernyms, but it struggles with challenging inference and role-based event prediction— and, in particular, it shows clear insensitivity to the contextual impacts of negation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The second trend is that neural networks routinely fail on exactly the phenomena where such symbolic models succeed, for example, logical inference [6,9], compositionality [7,23] and out-of-distribution generalization [24–26].”
From Symbols and grounding in large language models (2023) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Besides factual knowledge, large-scale pre-training also allows LMs to handle linguistic phenomena such as analogies [9], negations [25], semantic role sensitivity [25], semantic similarity [131], cant understanding [131], and rare word understanding [116].”
From Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2023) · cites this paper as [25]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For these kinds of tasks, we can also score the generation probability of each candidate and choose the one with the highest probability [25].”
From Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2023) · cites this paper as [25]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2×
“BERT is a deep bidirectional transformer network (Vaswani et al., 2017) pre-trained on tasks of masked language modeling (predicting masked words given bidirectional context) and next-sentence prediction (binary classification of whether two sentences are a sequence).”
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“As a case study, we use these three diagnostics to examine the predictive capacities of the pre-trained BERT model (Devlin et al., 2019), which has been the basis of impressive performance across a wide range of tasks.”
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · cited 3×
“, 2019) and ELMo (Peters et al., 2018a) have advanced the state of the art in a wide variety of tasks, suggesting that these models acquire valuable, generalizable linguistic competence during the pre-training process.”
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222 · cited 2×
“Much work has attempted to evaluate systems’ overall level of ‘‘understanding’’, often with tasks such as semantic similarity and entailment (Wang et al., 2018; Bowman et al., 2015; Agirre et al., 2012; Dagan et al., 2005; Bentivogli et al., 2016), and additional work has been done to design curated versions of these…”
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377
Brain potentials during reading reflect word expectancy and semantic association
Kutas, M., & Hillyard, S. A. (1984) · Nature · doi:10.1038/307161a0 · Cited by 2,218 · cited 1×
“Like cloze, the N400 can be used to gauge how expected a word w is in a context c —the amplitude of the N400 response appears to be sensitive to fit of a word in context, and has been shown to correlate with cloze in many cases (Kutas and Hillyard, 1984).”
MethodWhat Does BERT Look at? An Analysis of BERT’s Attention
Clark, K., Khandelwal, U., Levy, O., & Manning, C. D. (2019) · Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w19-4828 · Cited by 1,373
BERT Rediscovers the Classical NLP Pipeline
Tenney, I., Das, D., & Pavlick, E. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1452 · Cited by 1,307
SemEval-2012 Task 6: A Pilot on Semantic Textual Similarity
Agirre, E., Cer, D. M., Diab, M., & González-Agirre, A. (2012) · Joint Conference on Lexical and Computational Semantics · Cited by 1,063
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
McCoy, R. T., Pavlick, E., & Linzen, T. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1334 · Cited by 944
Research works printed but not linked: 19
- Adi Yossi, 2016, International Conference on Learning Representations
- Agirre Eneko, 2012, Proceedings of the First Joint Conference on Lexical and Computational Semantics-Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation, 385
- Chowdhury Shammur Absar, 2018, Proceedings of the 27th International Conference on Computational Linguistics, 133
- Clark Kevin, 2019, arXiv preprint arXiv:1906.04341 · arXiv
- Conneau Alexis, 2018, ACL 2018-56th Annual Meeting of the Association for Computational Linguistics, 2126
- Dagan Ido, 2005, Machine Learning Challenges Workshop, 177
- Dasgupta Ishita, 2018, Proceedings of the 40th Annual Meeting of the Cognitive Science Society
- Devlin Jacob, 2019, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Ettinger Allyson, 2018, Proceedings of the 27th International Conference on Computational Linguistics, 1790
- Frank Stefan L., 2013, ACL 2013-51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, 2, 878
- Futrell Richard, 2019, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 32
- Giulianelli Mario, 2018, Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, 240, 10.18653/v1/W18-5426 · DOI
- Gulordava Kristina, 2018, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), 1, 1195
- Kim Najoung, 2019, Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), 235
- Peters Matthew, 2018, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), 1, 2227
- Tenney Ian, 2019, arXiv preprint arXiv:1905.05950 · arXiv
- Tenney Ian, 2019, Proceedings of the International Conference on Learning Representations
- Vaswani Ashish, 2017, Advances in Neural Information Processing Systems, 5998
- Wang Alex, 2018, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 353
Cited by
Items 301–400 of 553
Beyond the Benchmarks: Toward Human-Like Lexical Representations
Stevenson, S., & Merlo, P. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.796741 · Cited by 9
“Beyond the similarity structure of word embeddings, it remains unclear how much current models capture the many other semantic relations that people are sensitive to (e.g., Köper et al., 2015; Santus et al., 2016; Ettinger, 2020).”
BackgroundDevelopmental Negation Processing in Transformer Language Models
Laverghetta, A., & Licato, J. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.60 · Cited by 6
TINA: Textual Inference with Negation Augmentation
Helwe, C., Coumes, S., Clavel, C., & Suchanek, F. M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.301 · Cited by 12
“However, one of the pitfalls for such models is negation (Ettinger, 2020; Helwe et al., 2021; Kassner and Schütze, 2020).”
BackgroundOrmerod, M., del Rincón, J. M., & Devereux, B. J. (2023) · Computational Linguistics · doi:10.1162/coli_a_00495 · Cited by 7
“However, in cases where Yu and Ettinger (2020) identify limited evidence of compositionality, they find that RoBERTa outperforms both XLM and XLNet, a result that generally aligns with our findings.”
BackgroundMethodResultInfluentialComparing Dependency-based Compositional Models with Contextualized Word Embeddings
Gamallo, P., Prada-Corral, M., & García, M. (2021) · International Conference on Agents and Artificial Intelligence (ICAART) · doi:10.5220/0010391812581265 · Cited by 6
Identifying the Limits of Cross-Domain Knowledge Transfer for Pretrained Models
Wu, Z., Liu, N. F., & Potts, C. (2022) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2022.repl4nlp-1.11 · Cited by 8
Qualitative Analysis of Depression Models by Demographics
Aguirre, C., & Dredze, M. H. (2021) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.18653/v1/2021.clpsych-1.19 · Cited by 6
Porada, I., Sordoni, A., & Cheung, J. C. K. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.337 · Cited by 11
Leveraging Affirmative Interpretations from Negation Improves Natural Language Understanding
Hossain, M. M., & Blanco, E. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.393 · Cited by 13
Zhu, H., Chen, Y., Yan, J., Liu, J., Yu, H. Z., Chen, Y., Wu, H., & Wang, H. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.531 · Cited by 6
Cognitive Modeling of Semantic Fluency Using TransformersPreprint
Nighojkar, A., Khlyzova, A., & Licato, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.09719 · Cited by 6
“Some work has also been done to explore the information language models capture [28, 29], but we note that at present, the ability of TLMs to model”
BackgroundSamuel, D. (2023) · Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning · doi:10.18653/v1/2023.conll-babylm.19 · Cited by 5
Linguistics and human brain: a perspective of computational neuroscience
Zhang, F., Chai, B., Wu, Y., Siok, W. T., & Wang, N. (2026) · Cognitive Neurodynamics · doi:10.1007/s11571-026-10496-2 · Cited by 7
“It therefore offers a quantitative bridge between com - putational modeling and neural mechanisms of language processing (Mitchell et al. 2008; Ettinger 2020).”
LLMs model how humans induce logically structured rules
Loo, A. M., Pavlick, E., & Feiman, R. (2025) · Journal of Memory and Language · doi:10.1016/j.jml.2025.104675 · Cited by 11
Chersoni, E., & Huang, C. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442442.3451387 · Cited by 5
“capabilities in associating nouns with their hypernyms, especially when the latter have to be picked from a closed set of possible answers [15, 35].”
BackgroundRepresentation and Pre-Activation of Lexical-Semantic Knowledge in Neural Language Models
Derby, S., Devereux, B. J., & Miller, P. C. (2021) · Workshop on Cognitive Modeling and Computational Linguistics (CMCL) · doi:10.18653/v1/2021.cmcl-1.25 · Cited by 7
“Ettinger (2020) devised a set of psycholinguistic diagnostic tasks to evaluate language understanding in BERT, demonstrating that some phenomena such as semantic role labelling and event knowledge are well-inferred, though others such as negation are less so.”
BackgroundAttentive Multiview Text Representation for Differential Diagnosis
Amiri, H., Mohtarami, M., & Kohane, I. S. (2021) · doi:10.18653/v1/2021.acl-short.128 · Cited by 8
Tuckute, G., Sathe, A., Wang, M., Yoder, H., Shain, C., & Fedorenko, E. G. (2022) · doi:10.18653/v1/2022.naacl-demo.11 · Cited by 7
“…example, NLP practitioners are interested in understanding and improving language model output, and bringing it closer to human generated text e.g., Ettinger (2020); Hollenstein et al. (2021); Meister et al. (2022); similarly, language scientists are highly interested in using large-scale language…”
BackgroundRepresenting affect information in word embeddings
Zhang, Y., Chen, W., Zhang, R., & Zhang, X. (2023) · Experiments in Linguistic Meaning · doi:10.3765/elm.2.5391 · Cited by 8
Pérez, N., Cuadros, M., & Rigau, G. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102682 · Cited by 5
Trustworthy Formal Natural Language Specifications
Gordon, C. S., & Matskevich, S. (2023) · ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward!) · doi:10.1145/3622758.3622890 · Cited by 5
“, via the well-known attention [127] mechanism, which essentially heuristically identifies chunks of input text to copy into the output), but meaning is not (which is why large language models consistently struggle with basic boolean reasoning [36, 90, 123, 124]).”
BackgroundMethodThe linguistic dead zone of value-aligned agency, natural and artificial
LaCroix, T. (2024) · Philosophical Studies · doi:10.1007/s11098-024-02257-w · Cited by 6
Effects of Architecture and Training on Embedding Geometry and Feature Discriminability in BERT
Podkorytov, M., Biś, D., Cai, J., Amirizirtol, K., & Liu, X. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9206645 · Cited by 7
A Comparison between Pre-training and Large-scale Back-translation for Neural Machine Translation
Huang, D., Wang, K., & Zhang, Y. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.150 · Cited by 10
“…Much work has discussed BERT with respect to morphology (Edmiston, 2020; Ha-ley, 2020), syntax (Hewitt and Manning, 2019; Lin et al., 2019; Goldberg, 2019), semantics (Ettinger, 2020; Warstadt et al., 2019; Tenney et al., 2019), and world knowledge (Poerner et al., 2019; Zhou et al., 2020).”
BackgroundBhattasali, S., & Resnik, P. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.332 · Cited by 10
Improving Contextual Representation with Gloss Regularized Pre-training
Yu, L., An, Z., Wu, P., & Ma, Z. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.68 · Cited by 5
RECAST: Interactive Auditing of Automatic Toxicity Detection Models
Wright, A. P., Shaikh, O., Park, H., Epperson, W., Ahmed, M., Pinel, S., Yang, D., & Chau, D. H. (2020) · doi:10.1145/3403676.3403691 · Cited by 7
“ards certain subgroups such as gender and race [1,8]. Some widely deployed NLP models, specifically BERT, also tend to overlook simple linguistic structures like negation, reducing their effectiveness [3]. A serious problem is that it is difficult to fix the potential erroneous model outputs due to a lack of the interpre…”
BackgroundEvaluation Benchmarks and Learning Criteria for Discourse-Aware Sentence RepresentationsPreprint
Chen, M., Chu, Z., & Gimpel, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.00142 · Cited by 6
“Benchmark datasets (Adi et al., 2017; Conneau and Kiela, 2018; Wang et al., 2018a, 2019) have been proposed to evaluate the encoded knowledge, where the focus has been primarily on natural language understanding capabilities of the representation of a stand-alone sentence, such as its semantic…”
BackgroundQuantifying the Task-Specific Information in Text-Based ClassificationsPreprint
Zhu, Z., Balagopalan, A., Ghassemi, M., & Rudzicz, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08931 · Cited by 5
“What drives this performance? A popular argument is: neural models learn certain linguistic skills for these tasks, and their representations encode linguistic knowledge (Lakretz et al., 2019; Hewitt and Manning, 2019; Chen et al., 2019; Tenney et al., 2019; Jiang and de Marneffe, 2019; Zhu et al., 2020; Ettinger, 202…”
BackgroundKarabašević, D., Vujko, A., Mirčetić, V., Popović, G., & Stanujkić, D. (2026) · Axioms · doi:10.3390/axioms15030175 · Cited by 4
Attention vs non-attention for a Shapley-based explanation method
Kersten, T., Wong, H. M., Jumelet, J., & Hupkes, D. (2021) · Workshop on Knowledge Extraction and Integration for Deep Learning Architectures: Deep Learning Inside Out (DeeLIO) · doi:10.18653/v1/2021.deelio-1.13 · Cited by 7
Is NLP Ready for Standardization?
Aufrant, L. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.202 · Cited by 7
“But since 2018 and the advent of masked language models, the term “language model” has now shifted to refer to Transformer-based contextualized embeddings, regardless of any probability distribution, and not necessarily autoregressive (as in Ettinger, 2020).”
Contextualized word senses: from attention to compositionality
Gamallo, P. (2023) · Linguistics Vanguard · doi:10.1515/lingvan-2022-0125 · Cited by 4
“Ettinger (2020) argues that what probably makes the human brain and artificial neural models different is that the former extracts the meaning of sentences by compositional mechanisms to make judgments of truth, while the latter rely on the technique of prediction in context.”
BackgroundLarge Language Models are biased to overestimate profoundness
Herrera-Berg, E., Browne, T. V., León-Villagrá, P., Vives, M., & Calderon, C. B. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.599 · Cited by 11
“Thus, recent work suggests a series of diagnostics to analyze LLMs inspired by human experiments (Ettinger, 2020).”
BackgroundAssessing Step-by-Step Reasoning against Lexical Negation: A Case Study on Syllogism
Ye, M., Kuribayashi, T., Suzuki, J., Kobayashi, G., & Funayama, H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.912 · Cited by 4
Larger Probes Tell a Different Story: Extending Psycholinguistic Datasets Via In-Context Learning
Shivagunde, N., Lialin, V., & Rumshisky, A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.130 · Cited by 8
Artificial Intelligence Empowered Language Models: A Review
Negi, A., Verma, C. V., & Tayyebi, Y. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-99-9524-0_40 · Cited by 5
Non-native speakers of English or ChatGPT: Who thinks better?
Shormani, M. Q. (2025) · F1000Research · doi:10.12688/f1000research.161306.2 · Cited by 5
A Gentle Introduction to Deep Nets and Opportunities for the Future
Church, K., Kordoni, V., Marcus, G., Davis, E., Ma, Y., & Chen, Z. (2022) · doi:10.18653/v1/2022.acl-tutorials.1 · Cited by 4
Psycholinguistic Diagnosis of Language Models’ Commonsense Reasoning
Cong, Y. (2022) · doi:10.18653/v1/2022.csrr-1.3 · Cited by 4
“Relatively few studies include online human response in the assessment (Ettinger, 2020).”
BackgroundRepresentation of Lexical Stylistic Features in Language Models’ Embedding Space
Lyu, Q., Apidianaki, M., & Callison-Burch, C. (2023) · Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/2023.starsem-1.32 · Cited by 14
Do Pretrained Contextual Language Models Distinguish between Hebrew Homograph Analyses?
Shmidman, A., Shmidman, C. S., Bareket, D., Koppel, M., & Tsarfaty, R. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.59 · Cited by 4
“This is in line with the findings of Ettinger (2020), who shows that BERT is stronger with syntax than semantics; Goldberg (2019) also notes BERT’s strong syntactic abilities.”
ResultGao, Y., Fu, X., Liu, X., & Wu, J. Y. (2023) · Complex & Intelligent Systems · doi:10.1007/s40747-023-01228-8 · Cited by 5
Jang, M., & Lukasiewicz, T. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.527 · Cited by 4
Learning to translate by learning to communicate
Downey, C. M., Zhou, X., Liu, Z., & Steinert‐Threlkeld, S. (2023) · Workshop on Multilingual Representation Learning (MRL) · doi:10.18653/v1/2023.mrl-1.17 · Cited by 7
Investigating Idiomaticity in Word Representations
He, W. P., Vieira, T. K., García, M., Scarton, C., Idiart, M., & Villavicencio, A. (2024) · Computational Linguistics · doi:10.1162/coli_a_00546 · Cited by 20
“…rely on substitutivity and the changes to a larger phrase representation caused by substitutions to its constituents (Garcia et al. 2021b; Yu and Ettinger 2020), alternatively, the notion of localism has also been analysed (Liu and Neubig 2022) focusing on whether the operations of a model are…”
BackgroundInfluentialTransformers in the loop: Polarity in neural models of language
Bylinina, L., & Tikhonov, A. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.455 · Cited by 4
Exploratory Model Analysis Using Data-Driven Neuron Representations
Oba, D., Yoshinaga, N., & Toyoda, M. (2021) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2021.blackboxnlp-1.41 · Cited by 9
“…2019; Mi-aschi and Dell’Orletta, 2020; Wu et al., 2020) , agreement information (Giulianelli et al., 2018; Goldberg, 2019), and semantic knowledge (Ten-ney et al., 2019b; Ettinger, 2020), and evaluates the properties of the internal representation with reference to the accuracy of the classifier.”
BackgroundAd astra or astray: Exploring linguistic knowledge of multilingual BERT through NLI task
Tikhonova, M., Mikhailov, V., Pisarevskaya, D., Malykh, V., & Shavrina, T. (2022) · Natural Language Engineering · doi:10.1017/s1351324922000225 · Cited by 8
“…outputs weakly correlate with the true labels when the sample contains Negation , Double negation , and Morphological negation , indicating that the model fails to infer this core construction, which is a well-studied problem in the field (Naik et al., 2018; Ettinger 2020; Hosseini et al., 2021).”
BackgroundInfluentialDiscourse Markers in Interaction: From Production to Comprehension
Li, R., Allein, L., Sileo, D., & Moens, M. (2022) · De Gruyter eBooks · doi:10.1515/9783110790351 · Cited by 4
“Studies on natural language processing models are valuable additions to psycholinguistics (Brysbaert, Keuleers, and Mandera 2014), as parallels can be established between NLP models and human processing (Mei et al. 2019; Ettinger 2020).”
BackgroundShin, U., Yi, E., & Song, S. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.937656 · Cited by 8
“Let’s Eat Grandma”: Does Punctuation Matter in Sentence Representation?
Karami, M., Mosallanezhad, A., Mancenido, M. V., & Liu, H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26390-3_34 · Cited by 4
“Despite proving its usefulness across a wide range of tasks in NLP, BERT has been shown lacking in some aspects, such as common sense, pragmatic inferences, and the meaning of negations [6].”
BackgroundKasmawati, K., Sisi, L., Juwairiyah, A., & Parizu, C. Z. L. (2021) · UNM Environmental Journals · doi:10.26858/uej.v4i3.28234 · Cited by 4
Beyond the Bias: Unveiling the Quality of Implicit Causality Prompt Continuations in Language Models
Sieker, J., Bott, O., Solstad, T., & Zarrieß, S. (2023) · International Conference on Natural Language Generation · doi:10.18653/v1/2023.inlg-main.15 · Cited by 4
“…a growing interest in probing the performance of large language models (LLMs) on carefully controlled linguistic test suites and experimental datasets to get a deeper understanding of specific linguistic capabilities captured in these models (e.g., Belinkov and Glass, 2019; Ettinger, 2020).”
BackgroundExplaining the Road Not TakenPreprint
Shen, H., & Huang, T.-H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.14973 · Cited by 4
“79%]: classify representation with specific diagnostic dataset [30, 10].”
BackgroundMethodSometimes We Want TranslationesePreprint
Parthasarathi, P., Sinha, K., Pineau, J., & Williams, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.07623 · Cited by 4
“When perturbation has been used to evaluate model performance, the utilized perturbation functions have been predominantly fairly simple, including reverse and word shuffle, and usually target only single sentences (Ettinger, 2020; Li et al., 2020; Sinha et al., 2020).”
MethodDemystifying Neural Language Models' Insensitivity to Word-OrderPreprint
Clouâtre, L., Parthasarathi, P., Zouaq, A., & Chandar, S. (2021) · arXiv (Cornell University) · Cited by 4
NOPE: A Corpus of Naturally-Occurring Presuppositions in EnglishPreprint
Parrish, A., Schuster, S., Warstadt, A., Agha, O., Lee, S.-H., Zhao, Z., Bowman, S. R., & Linzen, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.06987 · Cited by 7
“This could either be because the models are not sensitive to negation at all, as found for other models (Ettinger, 2020), or fail to draw the correct inferences when the context cancels a presupposition.”
Few-shot Named Entity Recognition with Cloze QuestionsPreprint
Gatta, V. L., Moscato, V., Postiglione, M., & Sperlí, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12421 · Cited by 4
“This idea has been successfully applied to unsupervised text classification [20], commonsense knowledge mining [8] and knowledge probing [7, 19].”
MethodRetentive or forgetful? diving into the knowledge memorizing mechanism of language models
Cao, B., Tang, Q., Lin, H., Jiang, S., Dong, B., Han, X., Chen, J., Wang, T., & Sun, L. (2024) · Proceedings of the Language Resources and Evaluation Conference · doi:10.63317/4h8nkuscdqqa · Cited by 4
Bennett, C. L., Kane, S. K., & Harrington, C. N. (2025) · Proceedings of the 5th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3757887.3763012 · Cited by 3
“In some cases, the term ‘invisible’ could be considered an important negation term, though models interpret them poorly [34, 88, 96].”
AKER: Arabic Knowledge-enriched Reader for Machine Reading Comprehension
Albilali, E., Al-Twairesh, N., & Hosny, M. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3736164 · Cited by 3
“When it comes to world knowledge, BERT struggles with pragmatic inference and role-based event knowledge [29].”
In the eyes of a language model: A comprehensive examination through eye-tracking data
Dini, L., Moroni, L., Brunato⋄, D., & Dell’Orletta⋄, F. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130617 · Cited by 5
Lin, J., Kuukka, A., Korpi, T., Tirkkonen, A., Kariluoto, A., Kaijansinkko, J., Satamo, M., Pajulammi, H. M., Haapanen, M. J., Häyrynen, S., et al. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.09.009 · Cited by 5
“However, pertinent to loneliness, to address the known negation limitations in BERT [11], the identification of negative terms (no experiences of loneliness) was additionally performed with a custom application of Levenshtein distance [12] as implemented in Python.”
Zhang, Z., Zeng, W., Tang, J., Wang, J., & Zhao, X. H. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761350 · Cited by 4
Answering Fill-in-the-Blank Questions in Portuguese with Transformer Language Models
Oliveira, H. G. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86230-5_58 · Cited by 3
How Does Data Corruption Affect Natural Language Understanding Models? A Study on GLUE datasets
Talman, A., Apidianaki, M., Chatzikyriakidis, S., & Tiedemann, J. (2022) · Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/2022.starsem-1.20 · Cited by 4
On Robustness and Sensitivity of a Neural Language Model: A Case Study on Italian L1 Learner Errors
Miaschi, A., Brunato⋄, D., Dell’Orletta⋄, F., & Venturi⋄, G. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3226333 · Cited by 7
Cross-Platform and Cross-Domain Abusive Language Detection with Supervised Contrastive Learning
Khondaker, M. T. I., Abdul-Mageed, M., & Lakshmanan, V. S. L. (2023) · doi:10.18653/v1/2023.woah-1.9 · Cited by 8
Predicting Terms in IS-A Relations with Pre-trained Transformers
Nikishina, I., Chernomorchenko, P., Demidova, A., Panchenko, A., & Biemann, C. (2023) · doi:10.18653/v1/2023.findings-ijcnlp.12 · Cited by 10
“…language structure and semantic relations between words: synonymy (Wijesiriwardene et al., 2022), hypernymy (Ravichander et al., 2020), negation (Ettinger, 2020), etc. Applying modern transformers for identifying IS-A relations would allow not only to check the capacities of large language…”
BackgroundMethodDo pretrained transformers infer telicity like humans?
Zhao, Y., Ngui, J. G., Hartley, L. H., & Bethard, S. J. (2021) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2021.conll-1.6 · Cited by 8
“Most works focus on syntactic knowledge (though see Ettinger (2020)’s work on semantic and pragmatic aspect).”
BackgroundUnderstanding Language Model from Questions in Social Studies for Students
Kawashima, K., & Yamaguchi, S. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671646 · Cited by 3
“Improving our understanding of what language models know about language has been actively discussed [1].”
BackgroundTowards Property-Based Tests in Natural Language
Gordon, C. S. (2022) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1145/3510455.3512781 · Cited by 3
“…negation in natural language requires additional lexicon entries, but here again CCGs offer advantages over neural networks: detailed CCG treatments of negations in nat-ural language already exist [58], while neural networks still struggle to handle language surrounding boolean operations [23, 60].”
BackgroundMa, Y., Sun, Z., Zhang, D., & Feng, Y. (2022) · Information · doi:10.3390/info13060296 · Cited by 3
“By executing a characterisation probing operation on MLM, Ettinger et al. [9] investigated the efficacy of each layer vector in BERT to encode word-level, syntactic-level, and semantic-level information, respectively.”
MethodQUASER: Question Answering with Scalable Extractive Rationalization
Ghoshal, A., Iyer, S. V., Paranjape, B., Lakhotia, K., Yih, S., & Mehdad, Y. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3532049 · Cited by 7
“On the other hand, [9, 10, 25, 39] directly probe the linguistic capabilities of neural networks.”
BackgroundASCM: An Answer Space Clustered Prompting Method without Answer Engineering
Wang, Z., Yang, Y., Xi, Z., Ma, B., Wang, L., Dong, R., & Anwar, A. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.193 · Cited by 4
“Prompt-based learning has also been applied to knowledge probing(Ettinger, 2020; Jiang et al., 2020a,c), text generation (Brown et al., 2020; Schick and Schütze, 2020; Dou et al., 2021), machine translation (Radford et al., 2019), question answering (Khashabi et al., 2020; Jiang et al., 2020b) and…”
MethodProbe-Less Probing of BERT’s Layer-Wise Linguistic Knowledge with Masked Word Prediction
Aoyama, T., & Schneider, N. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop (NAACL SRW) · doi:10.18653/v1/2022.naacl-srw.25 · Cited by 11
“(2020) call "stretch effect", although they do sometimes exhibit heterogeneous behaviors, such as responses to perturbation in word prediction (Ettinger, 2020).”
BackgroundLearning Sentence-Level Representations with Predictive Coding
Araujo, V., Moens, M., & Soto, Á. (2023) · Machine Learning and Knowledge Extraction · doi:10.3390/make5010005 · Cited by 10
Conceptual Foundations of Human-Centric AI
Steels, L. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-24349-3_2 · Cited by 6
Knowledge Acquired by Foundation Models
Paaß, G., & Giesselbach, S. (2023) · Artificial intelligence: foundations, theory, and algorithms/Artificial intelligence: Foundations, theory, and algorithms · doi:10.1007/978-3-031-23190-2_4 · Cited by 3
Probing with Noise: Unpicking the Warp and Weft of Embeddings
Klubička, F., & Kelleher, J. D. (2022) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2022.blackboxnlp-1.34 · Cited by 6
Seeking Clozure: Robust Hypernym extraction from BERT with Anchored Prompts
Liu, C., Cohn, T., & Frermann, L. (2023) · Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/2023.starsem-1.18 · Cited by 5
“The unsupervised extraction of hypernyms from PLMs by prompting has attracted recent attention, e.g., using patterns like A dog is a type of [MASK] and retrieving the most likely filler words from the model (Ettinger, 2020; Weir et al., 2020; Jain and Espinosa Anke, 2022).”
BackgroundMethodInfluentialMade of Steel? Learning Plausible Materials for Components in the Vehicle Repair Domain
Eichel, A., Schlipf, H., & Walde, S. S. I. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.104 · Cited by 7
Meaning creation in novel noun-noun compounds: humans and language models
Chen, P., Poeppel, D., & Zuanazzi, A. (2023) · Language Cognition and Neuroscience · doi:10.1080/23273798.2023.2254865 · Cited by 3
“These fi ndings indicate that BERT weighs word function (and possibly word order) more strongly than word semantic features (Ettinger, 2020; Jawahar et al., 2019; Liu et al., 2019; Papadimitriou et al., 2022).”
BackgroundMethodThe Linguistics of Programming
Gordon, C. S. (2024) · ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward!) · doi:10.1145/3689492.3689806 · Cited by 3
Романчук, С., Kuzmenko, O., Бережна, М. В., Vysoskyi, A., & Nikolaieva, T. M. (2024) · Forum for Linguistic Studies · doi:10.30564/fls.v6i6.6673 · Cited by 3
“Given the rapid technological development and changes in society, we can expect that the language personality will also undergo changes [4] .”
Do Language Models Know Who Did What to Whom?
Denning, J. M., Guo, X., Snefjella, B., & Blank, I. (2026) · Open Mind · doi:10.1162/opmi.a.365 · Cited by 4
“…on non-linguistic aspects of understanding): LLMs struggle with tracking the state of entities in a text (Kim and Schuster, 2023), sometimes refer to entities that do not exist (Schuster and Linzen, 2022), and make predictions that are only weakly sensitive to event roles (Ettinger, 2020).”
Pham, T. M., Bui, T., Mai, L., & Nguyen, A. (2022) · Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (AACL-IJCNLP) · doi:10.18653/v1/2022.aacl-main.2 · Cited by 7
Testing Pre-trained Language Models’ Understanding of Distributivity via Causal Mediation Analysis
Ban, P., Jiang, Y., Liu, T., & Steinert‐Threlkeld, S. (2022) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2022.blackboxnlp-1.26 · Cited by 3
Knowledge-Rich BERT Embeddings for Readability Assessment.Preprint
Imperial, J. M. (2021) · arXiv (Cornell University) · Cited by 3
BERTAgent: The Development of a Novel Tool to Quantify Agency in Textual DataPreprint
Nikadon, J., Suitner, C., Erseghe, T., Džanko, L., & Formanowicz, M. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/qw6u3 · Cited by 5
Floyd, S., Jouravlev, O., Mineroff, Z., Poliak, M., Gibson, E. A. F., & Fedorenko, E. G. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/e2xta · Cited by 3
Apidianaki, M., Fourtassi, A., & Padó, S. (2024) · Computational Linguistics · doi:10.1162/coli_e_00539 · Cited by 5
“…inspired by—the field of cognitive science, including eye tracking and brain imaging datasets, have been used to evaluate models’ behavior and to provide additional training signal (Ettinger 2020; Dunbar et al. 2017; Binz and Schulz 2023; Bingel, Barrett, and Søgaard 2016; Hollenstein et al. 2021).”
Pande, M., Budhraja, A., Nema, P., Kumar, P., & Khapra, M. M. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17605 · Cited by 12
The Self-Contained Negation Test Set
Kletz, D., Amsili, P., & Candito, M. (2023) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2023.blackboxnlp-1.16 · Cited by 2
“On the one hand, Kassner and Schütze (2020) and Ettinger (2020) compare the predictions of Transformer-based language models (Vaswani et al., 2017) in minimal pairs varying in polarity (1).”
BackgroundLeveraging Context for Perceptual Prediction Using Word Embeddings
Carter, G., Keller, F., & Hoffman, P. (2025) · Cognitive Science · doi:10.1111/cogs.70072 · Cited by 3
“…this issue is to compare word embeddings with human ratings or neuroimaging data to examine the extent to which they mimic human semantics (Abnar, Ahmed, Mijnheer, & Zuidema, 2018; Ettinger, 2020; Hollenstein, de la Torre, Langer, & Zhang, 2019; Turton, Vinson, & Smith, 2020, 2021; Utsumi, 2020).”
Metadata
- Authors
- Ettinger, A.
- Affiliations
- University of Chicago (US)
- Corresponding author
- Ettinger, Allyson
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2020-12
- Volume
- 8
- Pages
- 34-48
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W2965647350
- Cited by
- 511
- References
- 38
- Citations by year
- 2026: 16 · 2025: 32 · 2024: 39 · 2023: 122 · 2022: 92 · 2021: 151 · 2020: 53
- FWCI
- 41.25
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Social Sciences › Arts and Humanities › Language and Linguistics › Syntax, Semantics, Linguistic Variation
- Keywords
- BERT, language model pretraining, role reversal, negation, event prediction, language model evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.