What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models
Open accessThe DOI matches its Crossref record; 36 of the 38 references it cites were found in a registry or an index.
Abstract
Pre-training by language modeling has become a popular and successful approach to NLP tasks, but we have yet to understand exactly what linguistic capacities these pre-training processes confer upon models. In this paper we introduce a suite of diagnostics drawn from human language experiments, which allow us to ask targeted questions about information used by language models for generating predictions in context. As a case study, we apply these diagnostics to the popular BERT model, finding that it can generally distinguish good from bad completions involving shared category or role reversal, albeit with less sensitivity than humans, and it robustly retrieves noun hypernyms, but it struggles with challenging inference and role-based event prediction— and, in particular, it shows clear insensitivity to the contextual impacts of negation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The second trend is that neural networks routinely fail on exactly the phenomena where such symbolic models succeed, for example, logical inference [6,9], compositionality [7,23] and out-of-distribution generalization [24–26].”
From Symbols and grounding in large language models (2023) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Besides factual knowledge, large-scale pre-training also allows LMs to handle linguistic phenomena such as analogies [9], negations [25], semantic role sensitivity [25], semantic similarity [131], cant understanding [131], and rare word understanding [116].”
From Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2023) · cites this paper as [25]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For these kinds of tasks, we can also score the generation probability of each candidate and choose the one with the highest probability [25].”
From Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2023) · cites this paper as [25]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2×
“BERT is a deep bidirectional transformer network (Vaswani et al., 2017) pre-trained on tasks of masked language modeling (predicting masked words given bidirectional context) and next-sentence prediction (binary classification of whether two sentences are a sequence).”
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“As a case study, we use these three diagnostics to examine the predictive capacities of the pre-trained BERT model (Devlin et al., 2019), which has been the basis of impressive performance across a wide range of tasks.”
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · cited 3×
“, 2019) and ELMo (Peters et al., 2018a) have advanced the state of the art in a wide variety of tasks, suggesting that these models acquire valuable, generalizable linguistic competence during the pre-training process.”
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222 · cited 2×
“Much work has attempted to evaluate systems’ overall level of ‘‘understanding’’, often with tasks such as semantic similarity and entailment (Wang et al., 2018; Bowman et al., 2015; Agirre et al., 2012; Dagan et al., 2005; Bentivogli et al., 2016), and additional work has been done to design curated versions of these…”
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377
Brain potentials during reading reflect word expectancy and semantic association
Kutas, M., & Hillyard, S. A. (1984) · Nature · doi:10.1038/307161a0 · Cited by 2,218 · cited 1×
“Like cloze, the N400 can be used to gauge how expected a word w is in a context c —the amplitude of the N400 response appears to be sensitive to fit of a word in context, and has been shown to correlate with cloze in many cases (Kutas and Hillyard, 1984).”
MethodWhat Does BERT Look at? An Analysis of BERT’s Attention
Clark, K., Khandelwal, U., Levy, O., & Manning, C. D. (2019) · Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w19-4828 · Cited by 1,373
BERT Rediscovers the Classical NLP Pipeline
Tenney, I., Das, D., & Pavlick, E. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1452 · Cited by 1,307
SemEval-2012 Task 6: A Pilot on Semantic Textual Similarity
Agirre, E., Cer, D. M., Diab, M., & González-Agirre, A. (2012) · Joint Conference on Lexical and Computational Semantics · Cited by 1,063
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
McCoy, R. T., Pavlick, E., & Linzen, T. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1334 · Cited by 944
Research works printed but not linked: 19
- Adi Yossi, 2016, International Conference on Learning Representations
- Agirre Eneko, 2012, Proceedings of the First Joint Conference on Lexical and Computational Semantics-Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation, 385
- Chowdhury Shammur Absar, 2018, Proceedings of the 27th International Conference on Computational Linguistics, 133
- Clark Kevin, 2019, arXiv preprint arXiv:1906.04341 · arXiv
- Conneau Alexis, 2018, ACL 2018-56th Annual Meeting of the Association for Computational Linguistics, 2126
- Dagan Ido, 2005, Machine Learning Challenges Workshop, 177
- Dasgupta Ishita, 2018, Proceedings of the 40th Annual Meeting of the Cognitive Science Society
- Devlin Jacob, 2019, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Ettinger Allyson, 2018, Proceedings of the 27th International Conference on Computational Linguistics, 1790
- Frank Stefan L., 2013, ACL 2013-51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, 2, 878
- Futrell Richard, 2019, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 32
- Giulianelli Mario, 2018, Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, 240, 10.18653/v1/W18-5426 · DOI
- Gulordava Kristina, 2018, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), 1, 1195
- Kim Najoung, 2019, Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), 235
- Peters Matthew, 2018, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), 1, 2227
- Tenney Ian, 2019, arXiv preprint arXiv:1905.05950 · arXiv
- Tenney Ian, 2019, Proceedings of the International Conference on Learning Representations
- Vaswani Ashish, 2017, Advances in Neural Information Processing Systems, 5998
- Wang Alex, 2018, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 353
Cited by
Items 501–553 of 553
Human-Like Distractor Response in Vision-Language Model
Xu, X., & Chen, H. (2023) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2023.ijcnlp-main.12
“…their resemblance to human abilities in various aspects such as syntactic knowledge (Linzen et al., 2016; Gulordava et al., 2018), semantic knowledge (Ettinger, 2020; Kementched-jhieva et al., 2021; Misra et al., 2020), and the integration of semantic and syntactic information (Xu and Chen, 2022).”
BackgroundMethodMadrid, J., Diehl, P., Selig, M., Rolauffs, B., Hans, F. P., Busch, H.-J., Scheef, T., & Benning, L. (2024) · doi:10.2196/preprints.58375
Makrai, M. (2024) · Eötvös Loránd Tudományegyetem · doi:10.15476/elte.2023.350
Chiu, S.-I. (2024) · International Conference on Data-driven Optimization of Complex Systems (DOCS) · doi:10.1109/docs63458.2024.10704299
“However, it struggles to perform well for negative posts, which is consistent with the challenges observed in previous studies [22] where BERT-Chinese models exhibited insensitivity to the contextual impacts of negation.”
ResultUnderstanding and Reasoning with Negation
Hossain, M. M. (2022) · University of North Texas Digital Library (University of North Texas) · doi:10.12794/metadc2048613
Animate, or Inanimate, That Is the Question for Large Language Models
Pucci, G., Zanzotto, F. M., & Ranaldi, L. (2025) · Information · doi:10.3390/info16060493 · Cited by 2
Parallel Trees: a novel resource with aligned dependency and constituency syntactic representations
Alzetta, C., Miaschi, A., Dell’Orletta⋄, F., Venturi⋄, G., & Montemagni⋄, S. (2025) · Language Resources and Evaluation · doi:10.1007/s10579-025-09826-3
“To this end, several meth-ods have been implemented also taking inspiration from human language experiments (Ettinger, 2020) and demonstrated that sentence-level representations encode linguistic knowledge in a hierarchical manner (Belinkov et al., 2017; Blevins et al., 2018; Tenney et al., 2019),…”
Bentivogli, L., Menini, S., & Zamparelli, R. (2025) · Elsevier eBooks · doi:10.1016/b978-0-323-95504-1.00595-0
Zhang, E., Zhang, C., & Chen, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5496845
Sentiment Classification Based on Neural Networks and Data Augmentation
Li, Z. (2025) · International Conference on Next Generation Data-driven Networks (NGDN) · doi:10.1109/ngdn66208.2025.11182098
“…long texts or capturing long-distance dependencies [13]; Meanwhile, it has been shown that BERT performs poorly in processing complex semantic structures such as negation and role reversal, suggesting that it is deficient in modeling sequential and structural information in language [14].”
Exploring Lexical Sensitivities in Word Prediction Models: A case study on BERT
Misra, K. (2020) · Purdue e-Pubs (Purdue University System) · doi:10.25394/pgs.13308830
Al-Garadi, M. A., Yang, Y.-C., Cai, H., Ruan, Y., O’Connor, K., Graciela, G.-H., Perrone, J., & Sarker, A. (2021) · Figshare · doi:10.6084/m9.figshare.13646944
Referring expression generation in context
Same, F. (2024) · Directory of Open access Books (OAPEN Foundation) · doi:10.17169/refubium-43774
Initial achievements in relation extraction from RNA-focused scientific papers
Cavalleri, E., Soto-Gomez, M., Pashaeibarough, A., Malchiodi, D., Caufield, H., Reese, J., Mungall, C., Robinson, P., Casiraghi, E., Valentini, G., et al. (2024) · Archivio Istituzionale della Ricerca (Universita Degli Studi Di Milano)
Exploring the limits of systematicity of natural language understanding models
Sinha, K. (2023) · eScholarship@McGill (McGill) · doi:10.82308/21882
Language Model Pretraining with Lexical Semantic Relations
Romascanu, A. (2023) · eScholarship@McGill (McGill) · doi:10.82308/33958
N-Gram Learning and Pretraining Dynamics in Transformer Language Models
Chang, T. (2025) · eScholarship (California Digital Library) · Cited by 1
How Do Pretrained Language Models Deal with Negation? The case of Italian
GULLACE, V. (2024) · Electronic Theses and Dissertations Repository (University of Pisa)
Proceedings of the First NLPL Workshop on Deep Learning for Natural Language Processing
(2019) · DSpace repository (University of Tartu)
Cavalleri, E., & Mesiti, M. (2024) · Archivio Istituzionale della Ricerca (Universita Degli Studi Di Milano)
Proceedings of the First NLPL Workshop on Deep Learning for Natural Language Processing
Nivre, J., Ginter, F., Oepen, S., & Tiedemann, J. (2022) · UTUPub (University of Turku)
Proceedings of the First NLPL Workshop on Deep Learning for Natural Language Processing
Rönnqvist, S., Kanerva, J., Salakoski, T., & Ginter, F. (2025) · UTUPub (University of Turku)
The Role of Psycholinguistics in English Language Learning
Naibaho, L. (2022) · Eximia
Dynamics of Character Embeddings in Indian Mythology: A Physics-Informed Neural Networks Approach
Paul, A., Goswami, A., Mahata, S. K., & Das, D. (2025) · Annual Meeting of the Forum for Information Retrieval Evaluation (FIRE) · doi:10.1145/3777867.3777879
Pay (Cross) Attention to the Melody: Curriculum Masking for Single-Encoder Melodic HarmonizationPreprint
Kaliakatsos-Papakostas, M. A., Makris, D., Soiledis, K., Tsamis, K.-T., Katsouros, V., & Cambouropoulos, E. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.16150
“For example, a C:maj7 spanning two beats occupies two quarter-note grid positions (or eight sixteenth-note positions).”
Dai, R., Ding, S., Li, W., & Lyu, R. (2025) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc58881.2025.11342451
“Recent work has revealed discrepancies between human and PLM outputs, particularly in sensitivity to frequency, compositionality, and semantic plausibility [7].”
Gao, W., & He, W. (2026) · Symmetry · doi:10.3390/sym18020319
Moghaddam, A., Moe, L., & Nguyen, U. T. (2025) · International Conference on Foundation and Large Language Models (FLLM) · doi:10.1109/fllm67465.2025.11390927
“For example, Ettinger [26] shows that in a word prediction test using the sentence ªA robin is a Ð,º BERT predicts bird as the most likely word, followed by robin, person, hunter, and pigeon .”
Michalon, O., & Baggio, G. (2026) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/0952813x.2026.2633150
“While the human language system can overcome this temporary bias in such tasks, BERT’s performance remains affected by it (Ettinger, 2020).”
Structural priming in humans and large language models
Sinclair, A., Klimovich‐Gray, A., Jumelet, J., Adamian, N., & Konopka, A. E. (2026) · Journal of Memory and Language · doi:10.1016/j.jml.2025.104713 · Cited by 3
Lightweight Adaptation of BERT for Negation Sentiment Classification
Hongmo, T., & Kamata, H. (2026) · International Conference on Artificial Intelligence in Information and Communication (ICAIIC) · doi:10.1109/icaiic68212.2026.11454187
“Ettinger et al. [4] report that BERT is notably insensitive to the contextual effect of negation cues.”
Macroeconomic and Financial Essays on Exchange Rates
Fortes, R. (2022) · theses.fr (ABES) · doi:10.70675/27628b11z0665z46f9z9385z53dfaea52f6c
Word Meaning Representation in Neural Language Models : Lexical Polysemy and Semantic Relationships
Soler, A. G. (2021) · theses.fr (ABES) · doi:10.70675/99a10411zc05ez4135z8f31z13573e0fd4d1
Learning with Limited Labeled Data
Ouali, Y. (2023) · theses.fr (ABES) · doi:10.70675/eede5c08zaa1bz4a68z941fzf8c8d07e9020 · Cited by 1
Boualili, L. (2022) · theses.fr (ABES) · doi:10.70675/29d19654z89d4z4f44z85efz0add721cac69
Towards automatic understanding of narrative audiovisual content
Reboud, A. (2022) · theses.fr (ABES) · doi:10.70675/7b1296fez9f91z4484z8448z8040d28ab833
Facing Data Scarcity in Dialogues for Discourse Structure Discovery and Prediction
Li, C. (2023) · theses.fr (ABES) · doi:10.70675/c379d1f5z5324z4865z9c13z84c9508836f1 · Cited by 7
Beyond words : leveraging language models for incremental and context-aware text-to-speech synthesis
Stephenson, B. (2023) · theses.fr (ABES) · doi:10.70675/5b6c082czf0c2z4055zb230z5c5b56888ca5
Data-Based Natural Language Generation : Evaluation and Explainability
Faille, J. (2023) · theses.fr (ABES) · doi:10.70675/3e73dd43z7aa6z4405za5cez27c78b408a3c
Study of the multimodal understanding of vision-language transformer models
Salin, E. (2023) · theses.fr (ABES) · doi:10.70675/2beca704z5d3bz421dzbf61z61e00610aaeb
Négation et Modèles de Langue Pré-entraînés
Kletz, D. (2025) · theses.fr (ABES) · doi:10.70675/4d807ff3z0c1fz4bb3z9c9bz82f07fd0058e · Cited by 1
Evaluating the Novelty of Information: A Key Factor in the Impact on an Individual
Azaria, B., Hirschprung, R. S., & Alkoby, S. (2026) · International Journal on Semantic Web and Information Systems · doi:10.4018/ijswis.407997
Human and AI Preferences in Turkish Subject Realization: Linguistic and Educational Perspectives
Çınar, O., & Çınar, E. K. (2026) · Dil Eğitimi ve Araştırmaları Dergisi · doi:10.31464/jlere.1843496
Xervaser, C. R., & Teh, D. T. B. E. (2026) · Malaysian Journal of ELT Research · doi:10.52696/avwe7447
“The rise of NLP, particularly Transformer-based models (Ettinger, 2020; Vaswani et al., 2017), has accelerated the educational potential of AI.”
Large Language Models as Distributional Baselines for Language Tasks
Trott, S., Michaelov, J. A., Jones, C., Chang, T. A., & Bergen, B. K. (2026) · Open Mind · doi:10.1162/opmi.a.373
“Of course, these metrics also diverge from human behavior in important ways (Ettinger, 2020; Linzen & Leonard, 2018; Michaelov et al., 2025) and often fail to account for theoretically relevant variance in human behavior (Jones & Bergen, 2024b; Trott & Bergen, 2023).”
Three Lenses on Linguistic Knowledge in Pretrained Transformers: A Phenomenon-Centered ReviewPreprint
Wang, Y., & Ling, Z. (2026) · Research Square · doi:10.21203/rs.3.rs-10776566/v1
Vázquez, M. Y. L., Pérez, A. M., & Gutiérrez, B. (2026) · Research Square · doi:10.21203/rs.3.rs-11167683/v1
Skeleton-and-Strategy Prompting: Training-Free Negation Understanding for Vision-Language ModelsPreprint
Cai, Y., Rostami, M., & Thomason, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.01180
“From the perspective of language-only models, existing work has demonstrated that pretrained language models fail to distinguish between negated and non-negated queries (Kassner & Schütze, 2020; Ettinger, 2020).”
Dissecting Deep Language Models: The Explainability and Bias Perspective
ATTANASIO, G. (2022) · PORTO@Iris · Cited by 1
Bridging the Gap: A Neuro-Symbolic Evaluation Framework for LLM-Driven Consensus Building
Triantafyllopoulos, L., Paxinou, E., Verykios, V. S., & Kalles, D. (2026) · Pan-Hellenic Conference on Progress in Computing and Informatics (PCI) · doi:10.1145/3829216.3829328
“While Zhang et al. [13] showed that such metrics correlate well with human judgment in general tasks, recent studies highlight their inability to capture subtle semantic distinctions, such as negation or the use of domain-specific policy terminology [14].”
Metadata
- Authors
- Ettinger, A.
- Affiliations
- University of Chicago (US)
- Corresponding author
- Ettinger, Allyson
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2020-12
- Volume
- 8
- Pages
- 34-48
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W2965647350
- Cited by
- 511
- References
- 38
- Citations by year
- 2026: 16 · 2025: 32 · 2024: 39 · 2023: 122 · 2022: 92 · 2021: 151 · 2020: 53
- FWCI
- 41.25
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Social Sciences › Arts and Humanities › Language and Linguistics › Syntax, Semantics, Linguistic Variation
- Keywords
- BERT, language model pretraining, role reversal, negation, event prediction, language model evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.