CoChat

Language Model Behavior: A Comprehensive Survey

Open access
Chang, Tyler A., Bergen, Benjamin K. · 2024 · Computational Linguistics

The DOI matches its Crossref record; 428 of the 430 references it cites were found in a registry or an index, and 1 are not checked yet.

Abstract

Transformer language models have received widespread public attention, yet their generated text is often surprising even to NLP researchers. In this survey, we discuss over 250 recent studies of English language model behavior before task-specific fine-tuning. Language models possess basic capabilities in syntax, semantics, pragmatics, world knowledge, and reasoning, but these capabilities are sensitive to specific inputs and surface features. Despite dramatic increases in generated text quality as models scale to hundreds of billions of parameters, the models are still prone to unfactual responses, commonsense errors, memorized text, and social biases. Many of these weaknesses can be framed as over-generalizations or under-generalizations of learned patterns in text. We synthesize recent results to highlight what is currently known about large language model capabilities, thus providing a resource for applied work and for research in adjacent fields that use language models.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1162/coli_a_00492

Cites

428 of 430 references verified0 citation errors · 1 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 430 references

Research works printed but not linked: 24

  1. Broyde, 2021, Build a medical sentence matching application using BERT and Amazon SageMaker, AWS Machine Learning Blog
  2. Ganguli, 2022, Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned, ArXiv
  3. 110. Jigsaw. 2017. Perspective API. Google Jigsaw.
  4. 114. Johnson, Steven. 2022. A.I. is mastering language. Should we trust what it says? The New York Times.
  5. 120. Kalo, Jan-Christoph and Leandra Fichtel. 2022. KAMEL: Knowledge analysis with multitoken entities in language models. In 4th Conference on Automated Knowledge Base Construction.
  6. 129. Kavumba, Pride, Naoya Inoue, Benjamin Heinzerling, Keshav Singh, Paul Reisert, and Kentarou Inui. 2020. Balanced COPA: Countering superficial cues in causal reasoning. Association for Natural Language Processing, pages 1105-1108.
  7. 140. Krawczyk, Jack and Amarnag Subramanya. 2023. Bard is getting better at logic and reasoning. The Keyword: Google Blog.
  8. 145. Lakretz, Yair, Théo Desbordes, Dieuwke Hupkes, and Stanislas Dehaene. 2022. Can transformers process recursive nested constructions, like humans? In Proceedings of the 29th International Conference on Computational Linguistics, pages 3226-3232, International Committee on Computational Linguistics, Gyeongju, Republic of Korea.
  9. 152. Lee, Angie. 2023. What are large language models used for? NVIDIA Blog.
  10. 162. Li, Jiaxuan, Lang Yu, and Allyson Ettinger. 2022. Counterfactual reasoning: Do language models need world knowledge for causal inference? In Workshop on Neuro Causal and Symbolic AI (nCSI).
  11. 166. Lieber, Opher, Or Sharir, Barak Lenz, and Yoav Shoham. 2021. Jurassic-1: Technical details and evaluation. White Paper. AI21 Labs.
  12. 204. Nayak, Pandu. 2019. Understanding searches better than ever before. The Keyword: Google Blog.
  13. OpenAI, 2022, ChatGPT: Optimizing language models for dialogue, OpenAI Blog
  14. OpenAI, 2023, GPT-4 technical report, OpenAI
  15. OpenAI, 2023, Model index for researchers, OpenAI
  16. Petty, 2021, Transformers generalize linearly, ArXiv
  17. Radford, 2018, Improving language understanding by generative pre-training, OpenAI
  18. Radford, 2019, Language models are unsupervised multitask learners, OpenAI
  19. Rae, 2021, Scaling language models: Methods, analysis & insights from training gopher, ArXiv
  20. 265. Stevenson, Claire, Iris Smal, Matthijs Baas, Raoul Grasman, and Han van der Maas. 2022. Putting GPT-3's creativity to the (alternative uses) test. In International Conference on Computational Creativity, pages 164-168.
  21. 277. Tejada, Giorgia Nidia Carranza, Johannes Scholtes, and Gerasimos Spanakis. 2021. A study of BERT's processing of negations to determine sentiment. In Benelux Conference on Artificial Intelligence and the Belgian Dutch Conference on Machine Learning, pages 47-59.
  22. 279. Thewsey, Alex. 2021. Bring structure to diverse documents with Amazon Textract and transformer-based models on Amazon SageMaker. AWS Machine Learning Blog.
  23. 283. Trott, Sean. 2023. In cautious defense of LLM-ology. Blog Post.
  24. 289. Valmeekam, Karthik, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. 2022. Large language models still can't plan (a benchmark for LLMs on planning and reasoning about change). In Foundation Models for Decision Making Workshop.

Cited by

See all 136

Metadata

Authors
Chang, T. A., & Bergen, B. K.
Affiliations
University of California San Diego (US)
Venue
Computational Linguistics
Publisher
MIT Press
Published
2024
Volume
50
Issue
1
Pages
293-350
ISSN
  • 0891-2017 (ISSN-L)
  • 1530-9312
OpenAlex
W4388691863
Cited by
105
References
379
Citations by year
2026: 33 · 2025: 37 · 2024: 30
FWCI
9.89
Type
journal article
Language
en
Open access
Locations
  • Computational Linguistics (publishedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Funders
  • Medical Library Association
  • University of California, San Diego
  • Harvard Data Science Initiative, Harvard University
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
Keywords
large language models, transformer models, syntax, semantics, pragmatics, world knowledge, reasoning, factual errors, memorization, social bias, overgeneralization, undergeneralization, model scaling

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.