Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models
Open accessThe DOI matches its Crossref record; 85 of the 88 references it cites were found in a registry or an index; 4 of the 77 sources its claims cite were checked for support.
Abstract
Language is crucial for human intelligence, but what exactly is its role? We take language to be a part of a system for understanding and communicating about situations. In humans, these abilities emerge gradually from experience and depend on domain-general principles of biological neural networks: connection-based learning, distributed representation, and context-sensitive, mutual constraint satisfaction-based processing. Current artificial language processing systems rely on the same domain general principles, embodied in artificial neural networks. Indeed, recent progress in this field depends on query-based attention, which extends the ability of these systems to exploit context and has contributed to remarkable breakthroughs. Nevertheless, most current models focus exclusively on language-internal tasks, limiting their ability to perform tasks that depend on understanding situations. These systems also lack memory for the contents of prior situations outside of a fixed contextual span. We describe the organization of the brain’s distributed understanding system, which includes a fast learning system that addresses the memory problem. We sketch a framework for future models of understanding drawing equally on cognitive neuroscience and artificial intelligence and exploiting query-based attention. We highlight relevant current directions and consider further developments needed to fully capture human-level language understanding in a computational system.
Overview
CoChat Summary
TL;DR
This perspective argues that language should be treated as one component of an integrated system for understanding situations, rather than as a standalone domain, and that current neural language models are limited by focusing on language-internal tasks and lacking memory beyond a fixed contextual span. The authors sketch a brain- and AI-inspired framework combining query-based attention (QBA) with object, context, and medial-temporal-lobe-like memory subsystems.
Key findings
- Recent machine language progress rests on domain-general neural principles (connection-based learning, distributed representation, mutual constraint satisfaction) and especially on query-based attention, which underlies breakthroughs such as the Google Neural Machine Translation system and BERT.
- Current models remain limited: they focus on language-internal tasks, lack memory for prior situations outside a fixed contextual span, and underperform humans on "common sense physics" and entailment tests; GPT-3's training corpora are "far larger than a human learner could process in a lifetime," and its authors suggest further gains may require fundamental changes.
- The brain's understanding system is construed as neocortical subsystems for object, context, and language/relation representations engaged in mutual constraint satisfaction, plus a hippocampal/MTL fast-learning system (complementary learning systems theory) that supports one-trial memory and later consolidation into neocortex.
- In a cited situated-agent study (ref. 77), an agent trained to lift 20 objects but to place only 10 in a location could place the remaining objects on command with over 90% accuracy, across 50 objects and up to 70 action steps; egocentric, multimodal, temporally extended experience produced significantly better generalization than a fixed-perspective grid-world agent or a static image classifier.
Methods
- Conceptual/perspective article (PNAS), not an empirical study: it reviews cognitive neuroscience and AI literature and proposes a framework; no new data are collected.
- Evidence cited includes human behavioral and eye-movement studies (e.g., Bransford et al.; Altmann and Kamide), neuropsychological lesion findings, brain-imaging studies of narrative scene representations, and prior neural network and agent-based modeling work.
- The proposed model sketch processes picture–description pairs grouped into scenes and narratives, using mutual QBA within and between object and language subsystems, a sequential context subsystem, an MTL-like memory storing compressed invertible vectors of subsystem states, and a future frontal-lobe-like control subsystem.
Limitations
- Stated by the authors: the proposed model is a long-term research question and only a sketch that "leaves important issues open and will require substantial development"; the construal of the brain's understanding system is "tentative" in places (e.g., the link between language and relational representation), and researchers hold diverse views about the details of these systems.
- Stated: the framework initially addresses only concrete situations involving animate beings and physical objects, with extension to abstract situations left as future work; the treatment simplifies language to text rather than speech.
- Evident from the text: no empirical validation of the proposed architecture is presented, and the supporting evidence is drawn from disparate literatures rather than a unified test of the framework.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 11 Not valid · 53 Not checked · 17 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“A person processing language constructs a representation of the described situation in real time, using both the stream of words and other available information.”
What the sources say
- Not checked yetcites [43] Rumelhart D. E., “Some problems with the notion that words have literal meanings” in Metaphor and Thought, Ortony A., Ed. (Cambridge University Press, Cambridge, UK, 1979), pp. 71–82. [Google Scholar]
- Not checked yetcites [40] John M. F. St., McClelland J. L., Learning and applying contextual constraints in sentence comprehension. Artif. Intell. 46, 217–257 (1990). [Google Scholar] source
- Partly supportedcites [44] Rabovsky M., Hansen S. S., McClelland J. L., Modeling the n400 brain potential as change in a probabilistic representation of meaning. Nat. Hum. Behav. 2, 693–705 (2018). [DOI] [PubMed] [Google Scholar] sourceExperimental — accuracy not yet measured
The 'N400' section of our copy supports the core of the claim: 'listeners continually update their representation of the situation or event being described as each incoming word of a sentence is presented,' and the same section adds that 'the input guiding the formation of these representations could also come from witnessing events directly; from pictures, sounds, or movies; or from any combination of linguistic or other forms of input.' However, the 'other available information' half is stated only as a wider theoretical conception: the paper says 'we focus only on linguistic input here,' the Discussion marks input from other modalities as 'not currently implemented,' and the Distinctive Features section notes that in N400 experiments 'external sources of information about events are not generally available.' So the real-time construction of a situational representation from the word stream is confirmed, while the claim's assertion that this uses other available information goes beyond what this paper demonstrates. (LLM, conf=0.72)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“People exploit such information during language processing (64)”
What the sources say
- Supportedcites [64] Menenti L., Petersson K. M., Scheeringa R., Hagoort P., When elephants fly: Differential sensitivity of right and left inferior frontal gyri to discourse and world knowledge. J. Cogn. Neurosci. 21, 2358–2368 (2009). [DOI] [PubMed] [Google Scholar] source
The INTRODUCTION of our copy states outright that comprehension makes use of such information: "To comprehend a sentence, we rapidly make use of the knowledge we have about the world" and "we are constantly integrating the current word meaning with the preceding information," and that the reader "must not only retrieve and integrate lexical information and world knowledge that is stored in long-term memory" but also form a representation of the ongoing discourse (situation model). The Abstract likewise affirms that "both local discourse and world knowledge are known to influence sentence processing" and affect "the process of meaning unification." This broad background claim is therefore stated in the source, with the paper's fMRI results (right inferior frontal gyrus sensitivity to the discourse manipulation, left angular gyrus sensitivity to coherence) illustrating the exploitation of remotely stored discourse and world-knowledge information during language processing. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
In the paper
“Evidence from eye movements shows that people use linguistic and nonlinguistic input jointly and immediately (46).”
What the sources say
- Not checked yetcites [46] Tanenhaus M., Spivey-Knowlton M., Eberhard K., Sedivy J., Integration of visual and linguistic information in spoken language comprehension. Science 268, 1632–1634 (1995). [DOI] [PubMed] [Google Scholar] source
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“A recent model called GPT-3, which stands for generative pretrained transformer, achieves impressive gains on several benchmarks without requiring fine tuning (28).”
What the sources say
- Not checked yetcites [28] Brown T. B., et al. , Language models are few-shot learners. arXiv:2005.14165 (28 May 2020).
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“A key step was the introduction of methods for learning word representations (now called embeddings) from co-occurrence relationships in large text corpora (18, 19). These embeddings exploit both general and specific predictive relationships of all of the words in the corpus, improving generalization: task-focused neural models trained on small datasets better generalize to infrequent words (e.g.,”
What the sources say
- Not checked yetcites [18] Collobert R., et al. , Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011). [Google Scholar]
- Not checked yetcites [19] Mikolov T., Sutskever I., Chen K., Corrado G. S., Dean J., “Distributed representations of words and phrases and their compositionality” in Proceedings of Neural Information Processing Systems, Burges C. J. C., Bottou L., Ghahramani Z., Weinberger K. Q., Eds. (Curran Associates, Red Hook, NY, 2013), pp. 3111–3119. [Google Scholar]
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
35 more claims
Create a free account to see all 85 claims and to verify the 73 references still open.
- It may state two separate claims as one. 29
- Probably not a substantial claim. 22
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Conversely, richly grounded learning can also support language learning.”
From Scaling Instructable Agents Across Many Simulated Worlds (2024) · cites this paper as McClelland et al., 2020
The cited source says what the claim attributes to it.
In the citing paper
“Since human language use is deeply integrated with our understanding of grounded situations (McClelland et al., 2020), understanding the subtleties of human language will likely benefit from this grounding.”
From Scaling Instructable Agents Across Many Simulated Worlds (2024) · cites this paper as McClelland et al., 2020
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“We also found similar results when using distinct source analogs taken from another study 46 (Supplementary Section S5.3 ).”
From Emergent analogical reasoning in large language models (2023) · cites this paper as [46]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“GPT-3 was also able to solve the radiation problem via an analogy with lesser known source stories. 46”
From Emergent analogical reasoning in large language models (2023) · cites this paper as [46]
Experimental — accuracy not yet measuredIn the citing paper
“These stories each involve a solution based on convergence, but differ from the general story used in the original study of Gick and Holyoak.”
From Emergent analogical reasoning in large language models (2023) · cites this paper as [46]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Lakoff, G., & Johnson, M. (2003) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226470993.001.0001 · Cited by 20,632
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations
Rumelhart, D. E., McClelland, J. L., & AU (1986) · doi:10.7551/mitpress/5236.001.0001 · Cited by 15,686
Self-citationAdvances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,887
Research works printed but not linked: 4
- D. E. Rumelhart, “Toward an interactive model of reading” in Attention & Performance VI, S. Dornic, Ed. (LEA, Hillsdale, NJ, 1977), pp. 573–603.
- C. D. Manning, H. Schütze, Foundations of Statistical Language Processing (MIT Press, Cambridge, MA, 1999).
- J. Morton, K. Patterson, “Little words–no!” in Deep Dyslexia, M. Coltheart, K. Patterson, J. C. Marshall, Eds. (Routledge and Kegan Paul, London, UK, 1980), pp. 270–285.
- J. Russin, R. C. O’Reilly, Y. Bengio, “Deep learning needs a prefrontal cortex” in ICLR Workshop on Bridging AI and Cognitive Science (ICLR, 2020).
Cited by
Using cognitive psychology to understand GPT-3
Binz, M., & Schulz, E. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2218523120 · Cited by 559
Shared computational principles for language processing in humans and deep language models
Goldstein, A., Zada, Z., Buchnik, E., Schain, M., Price, A., Aubrey, B., Nastase, S. A., Feder, A., Emanuel, D., Cohen, A., et al. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01026-4 · Cited by 447
Emergent analogical reasoning in large language models
Webb, T., Holyoak, K. J., & Lu, H. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01659-w · Cited by 404
A hierarchy of linguistic predictions during natural language comprehension
Heilbron, M., Armeni, K., Schoffelen, J.-M., Hagoort, P., & de Lange, F. P. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2201968119 · Cited by 343
Inductive biases for deep learning of higher-level cognition
Goyal, A., & Bengio, Y. (2022) · Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rspa.2021.0068 · Cited by 300
Metadata
- Authors
- McClelland, J. L., Hill, F., Rudolph, M., Baldridge, J., & Schütze, H.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- Robert Bosch (Germany) (DE)
- Google (United States) (US)
- Ludwig-Maximilians-Universität München (DE)
- Corresponding author
- McClelland, James L.
- Hill, Felix
- Rudolph, Maja
- Baldridge, Jason
- Schütze, Hinrich
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2020-09-28
- Volume
- 117
- Issue
- 42
- Pages
- 25966-25974
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC7585006
- OpenAlex
- W3088418428
- Semantic Scholar
- 00b1e962182c42949822821dc1a929bd132ec082
- Cited by
- 91
- References
- 88
- Citations by year
- 2026: 1 · 2025: 16 · 2024: 18 · 2023: 24 · 2022: 22 · 2021: 7 · 2020: 3
- FWCI
- 7.52
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Open access LMU (Ludwid Maxmilian's Universitat Munchen) (submittedVersion, other-oa)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- language comprehension, neural language models, distributed representations
- MeSH terms
- Artificial Intelligence, Brain, Computer Simulation, Humans, Intelligence, Language, Neural Pathways, Neural Networks, Computer, Comprehension
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models. https://app.cochat.ai/papers/10.1073/pnas.1910416117
- The paper
- McClelland, J. L., Hill, F., Rudolph, M., Baldridge, J., & Schütze, H. (2020). Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.1910416117
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.