Language Model Behavior: A Comprehensive Survey
Open accessThe DOI matches its Crossref record; 428 of the 430 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Transformer language models have received widespread public attention, yet their generated text is often surprising even to NLP researchers. In this survey, we discuss over 250 recent studies of English language model behavior before task-specific fine-tuning. Language models possess basic capabilities in syntax, semantics, pragmatics, world knowledge, and reasoning, but these capabilities are sensitive to specific inputs and surface features. Despite dramatic increases in generated text quality as models scale to hundreds of billions of parameters, the models are still prone to unfactual responses, commonsense errors, memorized text, and social biases. Many of these weaknesses can be framed as over-generalizations or under-generalizations of learned patterns in text. We synthesize recent results to highlight what is currently known about large language model capabilities, thus providing a resource for applied work and for research in adjacent fields that use language models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“So far, quantitative comparisons have looked mainly at basic grammar and syntax (5) or features such as word choice, punctuation, sentence length, and so on, finding evidence of some differences between human- and LLM-written text (10–14).”
From Do LLMs write like humans? Variation in grammatical and rhetorical styles (2025) · cites this paper as [5]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“Transformer language models have revolutionized the field of natural language processing (NLP) since their introduction in 2018 (Radford et al. 2018; Devlin et al. 2019).”
BackgroundMethodLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Grice, H. P. (1975) · Speech Acts · doi:10.1163/9789004368811_003 · Cited by 23,221
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,887
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 1×
“increases, the number of pre-training tokens should increase roughly proportionally (Kaplan et al. 2020; Hoffmann et al. 2022).”
BackgroundOn the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Research works printed but not linked: 24
- Broyde, 2021, Build a medical sentence matching application using BERT and Amazon SageMaker, AWS Machine Learning Blog
- Ganguli, 2022, Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned, ArXiv
- 110. Jigsaw. 2017. Perspective API. Google Jigsaw.
- 114. Johnson, Steven. 2022. A.I. is mastering language. Should we trust what it says? The New York Times.
- 120. Kalo, Jan-Christoph and Leandra Fichtel. 2022. KAMEL: Knowledge analysis with multitoken entities in language models. In 4th Conference on Automated Knowledge Base Construction.
- 129. Kavumba, Pride, Naoya Inoue, Benjamin Heinzerling, Keshav Singh, Paul Reisert, and Kentarou Inui. 2020. Balanced COPA: Countering superficial cues in causal reasoning. Association for Natural Language Processing, pages 1105-1108.
- 140. Krawczyk, Jack and Amarnag Subramanya. 2023. Bard is getting better at logic and reasoning. The Keyword: Google Blog.
- 145. Lakretz, Yair, Théo Desbordes, Dieuwke Hupkes, and Stanislas Dehaene. 2022. Can transformers process recursive nested constructions, like humans? In Proceedings of the 29th International Conference on Computational Linguistics, pages 3226-3232, International Committee on Computational Linguistics, Gyeongju, Republic of Korea.
- 152. Lee, Angie. 2023. What are large language models used for? NVIDIA Blog.
- 162. Li, Jiaxuan, Lang Yu, and Allyson Ettinger. 2022. Counterfactual reasoning: Do language models need world knowledge for causal inference? In Workshop on Neuro Causal and Symbolic AI (nCSI).
- 166. Lieber, Opher, Or Sharir, Barak Lenz, and Yoav Shoham. 2021. Jurassic-1: Technical details and evaluation. White Paper. AI21 Labs.
- 204. Nayak, Pandu. 2019. Understanding searches better than ever before. The Keyword: Google Blog.
- OpenAI, 2022, ChatGPT: Optimizing language models for dialogue, OpenAI Blog
- OpenAI, 2023, GPT-4 technical report, OpenAI
- OpenAI, 2023, Model index for researchers, OpenAI
- Petty, 2021, Transformers generalize linearly, ArXiv
- Radford, 2018, Improving language understanding by generative pre-training, OpenAI
- Radford, 2019, Language models are unsupervised multitask learners, OpenAI
- Rae, 2021, Scaling language models: Methods, analysis & insights from training gopher, ArXiv
- 265. Stevenson, Claire, Iris Smal, Matthijs Baas, Raoul Grasman, and Han van der Maas. 2022. Putting GPT-3's creativity to the (alternative uses) test. In International Conference on Computational Creativity, pages 164-168.
- 277. Tejada, Giorgia Nidia Carranza, Johannes Scholtes, and Gerasimos Spanakis. 2021. A study of BERT's processing of negations to determine sentiment. In Benelux Conference on Artificial Intelligence and the Belgian Dutch Conference on Machine Learning, pages 47-59.
- 279. Thewsey, Alex. 2021. Bring structure to diverse documents with Amazon Textract and transformer-based models on Amazon SageMaker. AWS Machine Learning Blog.
- 283. Trott, Sean. 2023. In cautious defense of LLM-ology. Blog Post.
- 289. Valmeekam, Karthik, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. 2022. Large language models still can't plan (a benchmark for LLMs on planning and reasoning about change). In Foundation Models for Decision Making Workshop.
Cited by
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941
“[14] also discusses the capabilities and limitations of LLMs in various downstream tasks.”
BackgroundA Survey on Large Language Model based Autonomous Agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.11432 · Cited by 1,925
“[14] also discusses the capabilities and limitations of LLMs in various downstream tasks.”
BackgroundSequence modeling and design from molecular to genome scale with Evo
Nguyen, E., Poli, M., Durrant, M. G., Kang, B., Katrekar, D., Li, D. B., Bartie, L. J., Thomas, A. W., King, S. H., Brixi, G., et al. (2024) · Science · doi:10.1126/science.ado9336 · Cited by 580
Yang, L., Chen, H., Li, Z., Ding, X., & Wu, X. (2023) · arXiv · doi:10.48550/arxiv.2306.11489 · Cited by 239
“Furthermore, they have limited capabilities in inferring relationships between actions and events [65].”
BackgroundYang, L., Chen, H., Li, Z., Ding, X., & Wu, X. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3360454 · Cited by 238
“Furthermore, they have limited capabilities in inferring relationships between actions and events [65].”
Background
Metadata
- Authors
- Chang, T. A., & Bergen, B. K.
- Affiliations
- University of California San Diego (US)
- ORCID
- Bergen, Benjamin K.
- Venue
- Computational Linguistics
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 50
- Issue
- 1
- Pages
- 293-350
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W4388691863
- Cited by
- 105
- References
- 379
- Citations by year
- 2026: 33 · 2025: 37 · 2024: 30
- FWCI
- 9.89
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Computational Linguistics (publishedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- Medical Library Association
- University of California, San Diego
- Harvard Data Science Initiative, Harvard University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- large language models, transformer models, syntax, semantics, pragmatics, world knowledge, reasoning, factual errors, memorization, social bias, overgeneralization, undergeneralization, model scaling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.