CoChat

Language Model Behavior: A Comprehensive Survey

Open access
Chang, Tyler A., Bergen, Benjamin K. · 2024 · Computational Linguistics

The DOI matches its Crossref record; 423 of the 423 references it cites were found in a registry or an index.

Abstract

Transformer language models have received widespread public attention, yet their generated text is often surprising even to NLP researchers. In this survey, we discuss over 250 recent studies of English language model behavior before task-specific fine-tuning. Language models possess basic capabilities in syntax, semantics, pragmatics, world knowledge, and reasoning, but these capabilities are sensitive to specific inputs and surface features. Despite dramatic increases in generated text quality as models scale to hundreds of billions of parameters, the models are still prone to unfactual responses, commonsense errors, memorized text, and social biases. Many of these weaknesses can be framed as over-generalizations or under-generalizations of learned patterns in text. We synthesize recent results to highlight what is currently known about large language model capabilities, thus providing a resource for applied work and for research in adjacent fields that use language models.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1162/coli_a_00492

Cites

423 of 423 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 423 references

Research works printed but not linked: 28

  1. Aher, 2022, Using large language models to simulate multiple humans, ArXiv
  2. Broyde, 2021, Build a medical sentence matching application using BERT and Amazon SageMaker, AWS Machine Learning Blog
  3. Jigsaw, 2017, Perspective API, Google Jigsaw
  4. Johnson, 2022, A.I. is mastering language. Should we trust what it says?, The New York Times
  5. Jones, 2022, Distributional semantics still can’t account for affordances, 482
  6. Kabbara, 2022, Investigating the performance of transformer-based NLI models on presuppositional inferences, 779
  7. Kalo, 2022, KAMEL: Knowledge analysis with multitoken entities in language models
  8. Kavumba, 2020, Balanced COPA: Countering superficial cues in causal reasoning, Association for Natural Language Processing, 1105
  9. Krawczyk, 2023, Bard is getting better at logic and reasoning, The Keyword: Google Blog
  10. Lakretz, 2022, Can transformers process recursive nested constructions, like humans?, 3226
  11. Lee, 2023, What are large language models used for?, NVIDIA Blog
  12. Li, 2022, Counterfactual reasoning: Do language models need world knowledge for causal inference?
  13. Lieber, 2021, Jurassic-1: Technical details and evaluation, White Paper. AI21 Labs
  14. Mahowald, 2022, Experimentally measuring the redundancy of grammatical cues in transitive clauses, ArXiv
  15. Nayak, 2019, Understanding searches better than ever before, The Keyword: Google Blog
  16. OpenAI, 2022, ChatGPT: Optimizing language models for dialogue, OpenAI Blog
  17. OpenAI, 2023, GPT-4 technical report, OpenAI
  18. OpenAI, 2023, Model index for researchers, OpenAI
  19. Radford, 2018, Improving language understanding by generative pre-training, OpenAI
  20. Radford, 2019, Language models are unsupervised multitask learners, OpenAI
  21. Rae, 2021, Scaling language models: Methods, analysis & insights from training gopher, ArXiv
  22. Reif, 2019, Visualizing and measuring the geometry of BERT, 8594
  23. Shi, 2021, What Transformers might know about the physical world: T5 and the origins of knowledge, 2218
  24. Stevenson, 2022, Putting GPT-3’s creativity to the (alternative uses) test, 164
  25. Tejada, 2021, A study of BERT’s processing of negations to determine sentiment, 47
  26. Thewsey, 2021, Bring structure to diverse documents with Amazon Textract and transformer-based models on Amazon SageMaker, AWS Machine Learning Blog
  27. Trott, 2023, In cautious defense of LLM-ology, Blog Post
  28. Valmeekam, 2022, Large language models still can’t plan (a benchmark for LLMs on planning and reasoning about change)

Cited by

See all 135

Metadata

Authors
Chang, T. A., & Bergen, B. K.
Affiliations
University of California San Diego (US)
Venue
Computational Linguistics
Publisher
MIT Press
Published
2024
Volume
50
Issue
1
Pages
293-350
ISSN
  • 0891-2017 (ISSN-L)
  • 1530-9312
OpenAlex
W4388691863
Cited by
105
References
379
Citations by year
2026: 33 · 2025: 37 · 2024: 30
FWCI
9.89
Type
journal article
Language
en
Open access
Locations
  • Computational Linguistics (publishedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Funders
  • Medical Library Association
  • University of California, San Diego
  • Harvard Data Science Initiative, Harvard University
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
Keywords
large language models, transformer models, syntax, semantics, pragmatics, world knowledge, reasoning, factual errors, memorization, social bias, overgeneralization, undergeneralization, model scaling

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Language Model Behavior: A Comprehensive Survey. https://app.cochat.ai/papers/10.1162/coli_a_00492
The paper
Chang, T. A., & Bergen, B. K. (2024). Language Model Behavior: A Comprehensive Survey. Computational Linguistics. https://doi.org/10.1162/coli_a_00492

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.