Language Model Behavior: A Comprehensive Survey
Open accessThe DOI matches its Crossref record; 425 of the 425 references it cites were found in a registry or an index.
Abstract
Transformer language models have received widespread public attention, yet their generated text is often surprising even to NLP researchers. In this survey, we discuss over 250 recent studies of English language model behavior before task-specific fine-tuning. Language models possess basic capabilities in syntax, semantics, pragmatics, world knowledge, and reasoning, but these capabilities are sensitive to specific inputs and surface features. Despite dramatic increases in generated text quality as models scale to hundreds of billions of parameters, the models are still prone to unfactual responses, commonsense errors, memorized text, and social biases. Many of these weaknesses can be framed as over-generalizations or under-generalizations of learned patterns in text. We synthesize recent results to highlight what is currently known about large language model capabilities, thus providing a resource for applied work and for research in adjacent fields that use language models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“So far, quantitative comparisons have looked mainly at basic grammar and syntax (5) or features such as word choice, punctuation, sentence length, and so on, finding evidence of some differences between human- and LLM-written text (10–14).”
From Do LLMs write like humans? Variation in grammatical and rhetorical styles (2025) · cites this paper as [5]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 401–425 of 425
Deanthropomorphising NLP: Can a Language Model Be Conscious?
Shardlow, M., & Przybyła, P. (2022) · e-space (Manchester Metropolitan University) · doi:10.48550/arxiv.2211.11483 · Cited by 6
Probing Pre-Trained Language Models for Cross-Cultural Differences in ValuesPreprint
Arora, A., Kaffee, L.-A., & Augenstein, I. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.13722 · Cited by 6
The Role of Physical Inference in Pronoun Resolution
Cognitive Science Society 2021, Bergen, B. K., & Jones, C. (2021) · Proceedings of the Annual Meeting of the Cognitive Science Society · doi:10.48448/n9t1-5921 · Cited by 6
Self-citationLogical Tasks for Measuring Extrapolation and Rule ComprehensionPreprint
Fujisawa, I., & Kanai, R. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.07727 · Cited by 5
A fine-grained comparison of pragmatic language understanding in humans and language modelsPreprint
Hu, J. J., Floyd, S., Jouravlev, O., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.06801 · Cited by 5
Psycholinguistic Diagnosis of Language Models’ Commonsense Reasoning
Cong, Y. (2022) · doi:10.18653/v1/2022.csrr-1.3 · Cited by 4
Characterizing verbatim short-term memory in neural language models
Armeni, K., Honey, C., & Linzen, T. (2022) · Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL) · doi:10.18653/v1/2022.conll-1.28 · Cited by 4
Is It Smaller Than a Tennis Ball? Language Models Play the Game of Twenty Questions
Bruyn, M. D., Lotfi, E., Buhmann, J., & Daelemans, W. M. P. (2022) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2022.blackboxnlp-1.7 · Cited by 4
Şenel, L. K., & Schütze, H. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.42 · Cited by 3
Deep learning can contrast the minimal pairs of syntactic data
Park, K., Park, M.-K., & Song, S. (2021) · 언어연구 · Cited by 3
Implicit causality in GPT-2: a case studyPreprint
Huynh, H., Lentz, T. O., & van Miltenburg, E. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.04348 · Cited by 3
MiQA: A Benchmark for Inference on Metaphorical QuestionsPreprint
Comșa, I. M., Eisenschlos, J. M., & Narayanan, S. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.07993 · Cited by 3
Michaelov, J. A., & Bergen, B. K. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.08700 · Cited by 3
Self-citationDo ever larger octopi still amplify reporting biases? Evidence from judgments of typical colour
Liu, F., Eisenschlos, J. M., Cole, J. R., & Collier, N. (2022) · Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (AACL-IJCNLP) · doi:10.18653/v1/2022.aacl-short.27 · Cited by 3
Gender Biases Unexpectedly Fluctuate in the Pre-training Stage of Masked Language ModelsPreprint
Tang, K., & Jiang, H. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.14639 · Cited by 2
On the Cusp of Comprehensibility: Can Language Models Distinguish Between Metaphors and Nonsense?
Griciūtė, B., Tanti, M., & Donatelli, L. E. (2022) · Workshop on Figurative Language Processing (FigLang) · doi:10.18653/v1/2022.flp-1.25 · Cited by 2
Understanding Stereotypes in Language Models: Towards Robust Measurement and Zero-Shot DebiasingPreprint
Mattern, J., Jin, Z., Sachan, M., Mihalcea, R., & Schölkopf, B. (2022) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000653515 · Cited by 2
"No, they did not": Dialogue response dynamics in pre-trained language modelsPreprint
Kim, S. J., Yu, L., & Ettinger, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.02526 · Cited by 2
Poverty of the Stimulus? A Rational Approach
Prefors, A., Regier, T., & Tenenbaum, J. B. (2006) · eScholarship (California Digital Library) · Cited by 2
ProSPer: Probing Human and Neural Network Language Model Understanding of Spatial Perspective
Masis, T., & Anderson, C. J. (2021) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2021.blackboxnlp-1.8 · Cited by 1
Do Children Texts Hold The Key To Commonsense Knowledge?
Romero, J., & Razniewski, S. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.752 · Cited by 1
Lee, S.-H., & Schuster, S. (2022) · Scholarworks (University of Massachusetts Amherst) · doi:10.7275/s1kt-qg26 · Cited by 1
OPT: Open Pre-trained Transformer Language Models
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.01068
Preventing verbatim memorization in language models gives a false sense of privacy
Ippolito, D., Tramèr, F., Nasr, M., Zhang, C., Jagielski, M., Lee, K., Choquette-Choo, C. A., & Carlini, N. (2022) · arXiv · doi:10.48550/arxiv.2210.17546
Research works printed but not linked: 28
- Aher, 2022, Using large language models to simulate multiple humans, ArXiv
- Broyde, 2021, Build a medical sentence matching application using BERT and Amazon SageMaker, AWS Machine Learning Blog
- Jigsaw, 2017, Perspective API, Google Jigsaw
- Johnson, 2022, A.I. is mastering language. Should we trust what it says?, The New York Times
- Jones, 2022, Distributional semantics still can’t account for affordances, 482
- Kabbara, 2022, Investigating the performance of transformer-based NLI models on presuppositional inferences, 779
- Kalo, 2022, KAMEL: Knowledge analysis with multitoken entities in language models
- Kavumba, 2020, Balanced COPA: Countering superficial cues in causal reasoning, Association for Natural Language Processing, 1105
- Krawczyk, 2023, Bard is getting better at logic and reasoning, The Keyword: Google Blog
- Lakretz, 2022, Can transformers process recursive nested constructions, like humans?, 3226
- Lee, 2023, What are large language models used for?, NVIDIA Blog
- Li, 2022, Counterfactual reasoning: Do language models need world knowledge for causal inference?
- Lieber, 2021, Jurassic-1: Technical details and evaluation, White Paper. AI21 Labs
- Mahowald, 2022, Experimentally measuring the redundancy of grammatical cues in transitive clauses, ArXiv
- Nayak, 2019, Understanding searches better than ever before, The Keyword: Google Blog
- OpenAI, 2022, ChatGPT: Optimizing language models for dialogue, OpenAI Blog
- OpenAI, 2023, GPT-4 technical report, OpenAI
- OpenAI, 2023, Model index for researchers, OpenAI
- Radford, 2018, Improving language understanding by generative pre-training, OpenAI
- Radford, 2019, Language models are unsupervised multitask learners, OpenAI
- Rae, 2021, Scaling language models: Methods, analysis & insights from training gopher, ArXiv
- Reif, 2019, Visualizing and measuring the geometry of BERT, 8594
- Shi, 2021, What Transformers might know about the physical world: T5 and the origins of knowledge, 2218
- Stevenson, 2022, Putting GPT-3’s creativity to the (alternative uses) test, 164
- Tejada, 2021, A study of BERT’s processing of negations to determine sentiment, 47
- Thewsey, 2021, Bring structure to diverse documents with Amazon Textract and transformer-based models on Amazon SageMaker, AWS Machine Learning Blog
- Trott, 2023, In cautious defense of LLM-ology, Blog Post
- Valmeekam, 2022, Large language models still can’t plan (a benchmark for LLMs on planning and reasoning about change)
Cited by
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941
“[14] also discusses the capabilities and limitations of LLMs in various downstream tasks.”
BackgroundA Survey on Large Language Model based Autonomous Agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.11432 · Cited by 1,925
“[14] also discusses the capabilities and limitations of LLMs in various downstream tasks.”
BackgroundSequence modeling and design from molecular to genome scale with Evo
Nguyen, E., Poli, M., Durrant, M. G., Kang, B., Katrekar, D., Li, D. B., Bartie, L. J., Thomas, A. W., King, S. H., Brixi, G., et al. (2024) · Science · doi:10.1126/science.ado9336 · Cited by 580
Yang, L., Chen, H., Li, Z., Ding, X., & Wu, X. (2023) · arXiv · doi:10.48550/arxiv.2306.11489 · Cited by 239
“Furthermore, they have limited capabilities in inferring relationships between actions and events [65].”
BackgroundYang, L., Chen, H., Li, Z., Ding, X., & Wu, X. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3360454 · Cited by 238
“Furthermore, they have limited capabilities in inferring relationships between actions and events [65].”
Background
Metadata
- Authors
- Chang, T. A., & Bergen, B. K.
- Affiliations
- University of California San Diego (US)
- ORCID
- Bergen, Benjamin K.
- Venue
- Computational Linguistics
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 50
- Issue
- 1
- Pages
- 293-350
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W4388691863
- Cited by
- 105
- References
- 379
- Citations by year
- 2026: 33 · 2025: 37 · 2024: 30
- FWCI
- 9.89
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Computational Linguistics (publishedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- Medical Library Association
- University of California, San Diego
- Harvard Data Science Initiative, Harvard University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- large language models, transformer models, syntax, semantics, pragmatics, world knowledge, reasoning, factual errors, memorization, social bias, overgeneralization, undergeneralization, model scaling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.