CoChat

Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models

Open access
McClelland, James L., Hill, Felix, Rudolph, Maja, Baldridge, Jason, Schütze, Hinrich · 2020 · Proceedings of the National Academy of Sciences

The DOI matches its Crossref record; 85 of the 88 references it cites were found in a registry or an index; 4 of the 77 sources its claims cite were checked for support.

Abstract

Language is crucial for human intelligence, but what exactly is its role? We take language to be a part of a system for understanding and communicating about situations. In humans, these abilities emerge gradually from experience and depend on domain-general principles of biological neural networks: connection-based learning, distributed representation, and context-sensitive, mutual constraint satisfaction-based processing. Current artificial language processing systems rely on the same domain general principles, embodied in artificial neural networks. Indeed, recent progress in this field depends on query-based attention, which extends the ability of these systems to exploit context and has contributed to remarkable breakthroughs. Nevertheless, most current models focus exclusively on language-internal tasks, limiting their ability to perform tasks that depend on understanding situations. These systems also lack memory for the contents of prior situations outside of a fixed contextual span. We describe the organization of the brain’s distributed understanding system, which includes a fast learning system that addresses the memory problem. We sketch a framework for future models of understanding drawing equally on cognitive neuroscience and artificial intelligence and exploiting query-based attention. We highlight relevant current directions and consider further developments needed to fully capture human-level language understanding in a computational system.

Overview

CoChat Summary

TL;DR

This perspective argues that language should be treated as one component of an integrated system for understanding situations, rather than as a standalone domain, and that current neural language models are limited by focusing on language-internal tasks and lacking memory beyond a fixed contextual span. The authors sketch a brain- and AI-inspired framework combining query-based attention (QBA) with object, context, and medial-temporal-lobe-like memory subsystems.

Key findings

  • Recent machine language progress rests on domain-general neural principles (connection-based learning, distributed representation, mutual constraint satisfaction) and especially on query-based attention, which underlies breakthroughs such as the Google Neural Machine Translation system and BERT.
  • Current models remain limited: they focus on language-internal tasks, lack memory for prior situations outside a fixed contextual span, and underperform humans on "common sense physics" and entailment tests; GPT-3's training corpora are "far larger than a human learner could process in a lifetime," and its authors suggest further gains may require fundamental changes.
  • The brain's understanding system is construed as neocortical subsystems for object, context, and language/relation representations engaged in mutual constraint satisfaction, plus a hippocampal/MTL fast-learning system (complementary learning systems theory) that supports one-trial memory and later consolidation into neocortex.
  • In a cited situated-agent study (ref. 77), an agent trained to lift 20 objects but to place only 10 in a location could place the remaining objects on command with over 90% accuracy, across 50 objects and up to 70 action steps; egocentric, multimodal, temporally extended experience produced significantly better generalization than a fixed-perspective grid-world agent or a static image classifier.

Methods

  • Conceptual/perspective article (PNAS), not an empirical study: it reviews cognitive neuroscience and AI literature and proposes a framework; no new data are collected.
  • Evidence cited includes human behavioral and eye-movement studies (e.g., Bransford et al.; Altmann and Kamide), neuropsychological lesion findings, brain-imaging studies of narrative scene representations, and prior neural network and agent-based modeling work.
  • The proposed model sketch processes picture–description pairs grouped into scenes and narratives, using mutual QBA within and between object and language subsystems, a sequential context subsystem, an MTL-like memory storing compressed invertible vectors of subsystem states, and a future frontal-lobe-like control subsystem.

Limitations

  • Stated by the authors: the proposed model is a long-term research question and only a sketch that "leaves important issues open and will require substantial development"; the construal of the brain's understanding system is "tentative" in places (e.g., the link between language and relational representation), and researchers hold diverse views about the details of these systems.
  • Stated: the framework initially addresses only concrete situations involving animate beings and physical objects, with extension to abstract situations left as future work; the treatment simplifies language to text rather than speech.
  • Evident from the text: no empirical validation of the proposed architecture is presented, and the supporting evidence is drawn from disparate literatures rather than a unified test of the framework.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1073/pnas.1910416117

Cites

85 of 85 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

4 Valid · 11 Not valid · 53 Not checked · 17 Own results and interpretations

35 more claims

Create a free account to see all 85 claims and to verify the 73 references still open.

See all claims — free account
  • It may state two separate claims as one. 29
  • Probably not a substantial claim. 22

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 85 references

Research works printed but not linked: 4

  1. D. E. Rumelhart, “Toward an interactive model of reading” in Attention & Performance VI, S. Dornic, Ed. (LEA, Hillsdale, NJ, 1977), pp. 573–603.
  2. C. D. Manning, H. Schütze, Foundations of Statistical Language Processing (MIT Press, Cambridge, MA, 1999).
  3. J. Morton, K. Patterson, “Little words–no!” in Deep Dyslexia, M. Coltheart, K. Patterson, J. C. Marshall, Eds. (Routledge and Kegan Paul, London, UK, 1980), pp. 270–285.
  4. J. Russin, R. C. O’Reilly, Y. Bengio, “Deep learning needs a prefrontal cortex” in ICLR Workshop on Bridging AI and Cognitive Science (ICLR, 2020).

Cited by

See all 102

Metadata

Authors
McClelland, J. L., Hill, F., Rudolph, M., Baldridge, J., & Schütze, H.
Affiliations
  • Google DeepMind (United Kingdom) (GB)
  • Robert Bosch (Germany) (DE)
  • Google (United States) (US)
  • Ludwig-Maximilians-Universität München (DE)
Corresponding author
  • McClelland, James L.
  • Hill, Felix
  • Rudolph, Maja
  • Baldridge, Jason
  • Schütze, Hinrich
Venue
Proceedings of the National Academy of Sciences
Published
2020-09-28
Volume
117
Issue
42
Pages
25966-25974
ISSN
  • 0027-8424 (ISSN-L)
  • 1091-6490
OpenAlex
W3088418428
Cited by
91
References
88
Citations by year
2026: 1 · 2025: 16 · 2024: 18 · 2023: 24 · 2022: 22 · 2021: 7 · 2020: 3
FWCI
7.52
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
  • Open access LMU (Ludwid Maxmilian's Universitat Munchen) (submittedVersion, other-oa)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
language comprehension, neural language models, distributed representations
MeSH terms
Artificial Intelligence, Brain, Computer Simulation, Humans, Intelligence, Language, Neural Pathways, Neural Networks, Computer, Comprehension

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models. https://app.cochat.ai/papers/10.1073/pnas.1910416117
The paper
McClelland, J. L., Hill, F., Rudolph, M., Baldridge, J., & Schütze, H. (2020). Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.1910416117

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.