Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models
Open accessThe DOI matches its Crossref record; 85 of the 88 references it cites were found in a registry or an index; 4 of the 77 sources its claims cite were checked for support.
Abstract
Language is crucial for human intelligence, but what exactly is its role? We take language to be a part of a system for understanding and communicating about situations. In humans, these abilities emerge gradually from experience and depend on domain-general principles of biological neural networks: connection-based learning, distributed representation, and context-sensitive, mutual constraint satisfaction-based processing. Current artificial language processing systems rely on the same domain general principles, embodied in artificial neural networks. Indeed, recent progress in this field depends on query-based attention, which extends the ability of these systems to exploit context and has contributed to remarkable breakthroughs. Nevertheless, most current models focus exclusively on language-internal tasks, limiting their ability to perform tasks that depend on understanding situations. These systems also lack memory for the contents of prior situations outside of a fixed contextual span. We describe the organization of the brain’s distributed understanding system, which includes a fast learning system that addresses the memory problem. We sketch a framework for future models of understanding drawing equally on cognitive neuroscience and artificial intelligence and exploiting query-based attention. We highlight relevant current directions and consider further developments needed to fully capture human-level language understanding in a computational system.
Overview
CoChat Summary
TL;DR
This perspective argues that language should be treated as one component of an integrated system for understanding situations, rather than as a standalone domain, and that current neural language models are limited by focusing on language-internal tasks and lacking memory beyond a fixed contextual span. The authors sketch a brain- and AI-inspired framework combining query-based attention (QBA) with object, context, and medial-temporal-lobe-like memory subsystems.
Key findings
- Recent machine language progress rests on domain-general neural principles (connection-based learning, distributed representation, mutual constraint satisfaction) and especially on query-based attention, which underlies breakthroughs such as the Google Neural Machine Translation system and BERT.
- Current models remain limited: they focus on language-internal tasks, lack memory for prior situations outside a fixed contextual span, and underperform humans on "common sense physics" and entailment tests; GPT-3's training corpora are "far larger than a human learner could process in a lifetime," and its authors suggest further gains may require fundamental changes.
- The brain's understanding system is construed as neocortical subsystems for object, context, and language/relation representations engaged in mutual constraint satisfaction, plus a hippocampal/MTL fast-learning system (complementary learning systems theory) that supports one-trial memory and later consolidation into neocortex.
- In a cited situated-agent study (ref. 77), an agent trained to lift 20 objects but to place only 10 in a location could place the remaining objects on command with over 90% accuracy, across 50 objects and up to 70 action steps; egocentric, multimodal, temporally extended experience produced significantly better generalization than a fixed-perspective grid-world agent or a static image classifier.
Methods
- Conceptual/perspective article (PNAS), not an empirical study: it reviews cognitive neuroscience and AI literature and proposes a framework; no new data are collected.
- Evidence cited includes human behavioral and eye-movement studies (e.g., Bransford et al.; Altmann and Kamide), neuropsychological lesion findings, brain-imaging studies of narrative scene representations, and prior neural network and agent-based modeling work.
- The proposed model sketch processes picture–description pairs grouped into scenes and narratives, using mutual QBA within and between object and language subsystems, a sequential context subsystem, an MTL-like memory storing compressed invertible vectors of subsystem states, and a future frontal-lobe-like control subsystem.
Limitations
- Stated by the authors: the proposed model is a long-term research question and only a sketch that "leaves important issues open and will require substantial development"; the construal of the brain's understanding system is "tentative" in places (e.g., the link between language and relational representation), and researchers hold diverse views about the details of these systems.
- Stated: the framework initially addresses only concrete situations involving animate beings and physical objects, with extension to abstract situations left as future work; the treatment simplifies language to text rather than speech.
- Evident from the text: no empirical validation of the proposed architecture is presented, and the supporting evidence is drawn from disparate literatures rather than a unified test of the framework.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 11 Not valid · 53 Not checked · 17 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“A person processing language constructs a representation of the described situation in real time, using both the stream of words and other available information.”
What the sources say
- Not checked yetcites [43] Rumelhart D. E., “Some problems with the notion that words have literal meanings” in Metaphor and Thought, Ortony A., Ed. (Cambridge University Press, Cambridge, UK, 1979), pp. 71–82. [Google Scholar]
- Not checked yetcites [40] John M. F. St., McClelland J. L., Learning and applying contextual constraints in sentence comprehension. Artif. Intell. 46, 217–257 (1990). [Google Scholar] source
- Partly supportedcites [44] Rabovsky M., Hansen S. S., McClelland J. L., Modeling the n400 brain potential as change in a probabilistic representation of meaning. Nat. Hum. Behav. 2, 693–705 (2018). [DOI] [PubMed] [Google Scholar] sourceExperimental — accuracy not yet measured
The 'N400' section of our copy supports the core of the claim: 'listeners continually update their representation of the situation or event being described as each incoming word of a sentence is presented,' and the same section adds that 'the input guiding the formation of these representations could also come from witnessing events directly; from pictures, sounds, or movies; or from any combination of linguistic or other forms of input.' However, the 'other available information' half is stated only as a wider theoretical conception: the paper says 'we focus only on linguistic input here,' the Discussion marks input from other modalities as 'not currently implemented,' and the Distinctive Features section notes that in N400 experiments 'external sources of information about events are not generally available.' So the real-time construction of a situational representation from the word stream is confirmed, while the claim's assertion that this uses other available information goes beyond what this paper demonstrates. (LLM, conf=0.72)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“People exploit such information during language processing (64)”
What the sources say
- Supportedcites [64] Menenti L., Petersson K. M., Scheeringa R., Hagoort P., When elephants fly: Differential sensitivity of right and left inferior frontal gyri to discourse and world knowledge. J. Cogn. Neurosci. 21, 2358–2368 (2009). [DOI] [PubMed] [Google Scholar] source
The INTRODUCTION of our copy states outright that comprehension makes use of such information: "To comprehend a sentence, we rapidly make use of the knowledge we have about the world" and "we are constantly integrating the current word meaning with the preceding information," and that the reader "must not only retrieve and integrate lexical information and world knowledge that is stored in long-term memory" but also form a representation of the ongoing discourse (situation model). The Abstract likewise affirms that "both local discourse and world knowledge are known to influence sentence processing" and affect "the process of meaning unification." This broad background claim is therefore stated in the source, with the paper's fMRI results (right inferior frontal gyrus sensitivity to the discourse manipulation, left angular gyrus sensitivity to coherence) illustrating the exploitation of remotely stored discourse and world-knowledge information during language processing. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
In the paper
“Evidence from eye movements shows that people use linguistic and nonlinguistic input jointly and immediately (46).”
What the sources say
- Not checked yetcites [46] Tanenhaus M., Spivey-Knowlton M., Eberhard K., Sedivy J., Integration of visual and linguistic information in spoken language comprehension. Science 268, 1632–1634 (1995). [DOI] [PubMed] [Google Scholar] source
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“A recent model called GPT-3, which stands for generative pretrained transformer, achieves impressive gains on several benchmarks without requiring fine tuning (28).”
What the sources say
- Not checked yetcites [28] Brown T. B., et al. , Language models are few-shot learners. arXiv:2005.14165 (28 May 2020).
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“A key step was the introduction of methods for learning word representations (now called embeddings) from co-occurrence relationships in large text corpora (18, 19). These embeddings exploit both general and specific predictive relationships of all of the words in the corpus, improving generalization: task-focused neural models trained on small datasets better generalize to infrequent words (e.g.,”
What the sources say
- Not checked yetcites [18] Collobert R., et al. , Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011). [Google Scholar]
- Not checked yetcites [19] Mikolov T., Sutskever I., Chen K., Corrado G. S., Dean J., “Distributed representations of words and phrases and their compositionality” in Proceedings of Neural Information Processing Systems, Burges C. J. C., Bottou L., Ghahramani Z., Weinberger K. Q., Eds. (Curran Associates, Red Hook, NY, 2013), pp. 3111–3119. [Google Scholar]
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
35 more claims
Create a free account to see all 85 claims and to verify the 73 references still open.
- It may state two separate claims as one. 29
- Probably not a substantial claim. 22
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Conversely, richly grounded learning can also support language learning.”
From Scaling Instructable Agents Across Many Simulated Worlds (2024) · cites this paper as McClelland et al., 2020
The cited source says what the claim attributes to it.
In the citing paper
“Since human language use is deeply integrated with our understanding of grounded situations (McClelland et al., 2020), understanding the subtleties of human language will likely benefit from this grounding.”
From Scaling Instructable Agents Across Many Simulated Worlds (2024) · cites this paper as McClelland et al., 2020
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“We also found similar results when using distinct source analogs taken from another study 46 (Supplementary Section S5.3 ).”
From Emergent analogical reasoning in large language models (2023) · cites this paper as [46]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“GPT-3 was also able to solve the radiation problem via an analogy with lesser known source stories. 46”
From Emergent analogical reasoning in large language models (2023) · cites this paper as [46]
Experimental — accuracy not yet measuredIn the citing paper
“These stories each involve a solution based on convergence, but differ from the general story used in the original study of Gick and Holyoak.”
From Emergent analogical reasoning in large language models (2023) · cites this paper as [46]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 15×
“QBA addresses this by issuing queries for relevant information.”
MethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 1×
“In one important QBA model called BERT, which stands for bidirectional encoder representations from transformers (25), the network is trained to correct missing or replaced words in blocks of text, typically spanning two sentences.”
MethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 1×
“In one important QBA model called BERT, which stands for bidirectional encoder representations from transformers (25), the network is trained to correct missing or replaced words in blocks of text, typically spanning two sentences.”
MethodDistributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 1×
“A key step was the introduction of methods for learning word representations (now called embeddings) from co-occurrence relationships in large text corpora (18, 19).”
MethodLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 7×
“A recent model called GPT-3, which stands for generative pretrained transformer, achieves impressive gains on several benchmarks without requiring fine tuning (28).”
BackgroundMethodInfluentialLakoff, G., & Johnson, M. (2003) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226470993.001.0001 · Cited by 20,632
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 16×
“QBA addresses this by issuing queries for relevant information.”
BackgroundInfluentialParallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations
Rumelhart, D. E., McClelland, J. L., & AU (1986) · doi:10.7551/mitpress/5236.001.0001 · Cited by 15,686
Self-citationAdvances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,887
Women, Fire, and Dangerous Things
Lakoff, G. (1987) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226471013.001.0001 · Cited by 12,386 · cited 1×
“Our proposal to focus on language as part of a system representing situations builds on a long tradition in linguistics (35), human cognitive psychology (36), psycholinguistics (37), philosophy (38), and artificial intelligence (39).”
BackgroundElman, J. L. (1990) · Cognitive Science · doi:10.1207/s15516709cog1402_1 · Cited by 11,212
The origins of intelligence in children.
Piaget, J. (1952) · doi:10.1037/11494-000 · Cited by 10,687
Fodor, J. A. (1983) · doi:10.7551/mitpress/4737.001.0001 · Cited by 7,734
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · cited 1×
“Some machine models adopt this approach (30), and below we adopt a hybrid approach as well.”
MethodGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · cited 1×
“It was used in the Google Neural Machine Translation system (22), a system that attained a sudden leap in performance and attracted widespread public interest (23).”
MethodMcClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256
Self-citationNatural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962 · cited 1×
“A key step was the introduction of methods for learning word representations (now called embeddings) from co-occurrence relationships in large text corpora (18, 19).”
MethodMcClelland, J. L., & Rumelhart, D. E. (1981) · Psychological Review · doi:10.1037/0033-295x.88.5.375 · Cited by 4,889
Self-citationCummins, R., & Fodor, J. (1985) · The Philosophical Review · doi:10.2307/2184717 · Cited by 4,791
Connectionism and cognitive architecture: A critical analysis
Fodor, J. A., & Pylyshyn, Z. W. (1988) · Cognition · doi:10.1016/0010-0277(88)90031-5 · Cited by 4,022
Language identification in the limit
Gold, E. M. (1967) · Information and Control · doi:10.1016/s0019-9958(67)91165-5 · Cited by 3,572 · cited 1×
“2 A ) that captured key characteristics of language structure through learning, a feat once considered impossible (15).”
BackgroundRumelhart, D. E., & McClelland, J. L. (1986) · The MIT Press eBooks · Cited by 2,997
Self-citationIntegration of Visual and Linguistic Information in Spoken Language Comprehension
Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995) · Science · doi:10.1126/science.7777863 · Cited by 2,929
Simple memory: a theory for archicortex
Marr, D. (1971) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1971.0078 · Cited by 2,771
Where do you know what you know? The representation of semantic knowledge in the human brain
Patterson, K., Nestor, P. J., & Rogers, T. T. (2007) · Nature Reviews Neuroscience · doi:10.1038/nrn2277 · Cited by 2,706 · cited 1×
“A brain area near the front of the temporal lobe houses neurons whose activity provides an embedding capturing the properties of objects (52).”
BackgroundBuilding machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x16001837 · Cited by 2,550
Building Machines That Learn and Think Like People
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2016) · arXiv · doi:10.48550/arxiv.1604.00289 · Cited by 2,550
Contextual prerequisites for understanding: Some investigations of comprehension and recall
Bransford, J. D., & Johnson, M. K. (1972) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(72)80006-9 · Cited by 2,131 · cited 2×
“Our proposal to focus on language as part of a system representing situations builds on a long tradition in linguistics (35), human cognitive psychology (36), psycholinguistics (37), philosophy (38), and artificial intelligence (39).”
BackgroundThe Proper Treatment of Quantification in Ordinary English
Montague, R. (1973) · Approaches to Natural Language · doi:10.1007/978-94-010-2506-5_10 · Cited by 2,069
Dynamic Memory: A Theory of Reminding and Learning in Computers and People
Schank, R. C. (1983) · Medical Entomology and Zoology · Cited by 1,972
Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms
White, B. W., & Rosenblatt, F. (1963) · The American Journal of Psychology · doi:10.2307/1419730 · Cited by 1,792
PRINCIPLES OF NEURODYNAMICS. PERCEPTRONS AND THE THEORY OF BRAIN MECHANISMS
ROSENBLATT, F. (1961) · doi:10.21236/ad0256582 · Cited by 1,691
Rumelhart, D. E., & McClelland, J. L. (1982) · Psychological Review · doi:10.1037//0033-295x.89.1.60 · Cited by 1,525
Self-citationHybrid computing using a neural network with dynamic external memory
Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T., Agapiou, J., et al. (2016) · Nature · doi:10.1038/nature20101 · Cited by 1,412 · cited 2×
“While a superpositional, distributed memory (68) will ultimately be the best model of this system, an initial implementation could store each such vector in a separate slot (69, 70).”
BackgroundTwo cortical systems for memory-guided behaviour
Ranganath, C., & Ritchey, M. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3338 · Cited by 1,377
Distributed representations, simple recurrent networks, and grammatical structure
Elman, J. L. (1991) · Machine Learning · doi:10.1007/bf00114844 · Cited by 1,354
Further analysis of the hippocampal amnesic syndrome: 14-year follow-up study of H.M.
Milner, B., Corkin, S., & Teuber, H.-L. (1968) · Neuropsychologia · doi:10.1016/0028-3932(68)90021-3 · Cited by 1,229
Perceptual Restoration of Missing Speech Sounds
Warren, R. M. (1970) · Science · doi:10.1126/science.167.3917.392 · Cited by 1,143 · cited 1×
“Humans appear to exploit past context and a limited window of subsequent context (29), suggesting a hybrid strategy.”
BackgroundSemantics: An Interdisciplinary Reader in Philosophy, Linguistics and Psychology
Lyons, J., Steinberg, D. D., & Jakobovits, L. A. (1973) · The Modern Language Review · doi:10.2307/3724998 · Cited by 1,089
Non-Holographic Associative Memory
WILLSHAW, D. J., BUNEMAN, O. P., & LONGUET-HIGGINS, H. C. (1969) · Nature · doi:10.1038/222960a0 · Cited by 979 · cited 1×
“While a superpositional, distributed memory (68) will ultimately be the best model of this system, an initial implementation could store each such vector in a separate slot (69, 70).”
BackgroundDiscovering Event Structure in Continuous Narrative Perception and Memory
Baldassano, C., Chen, J., Zadbood, A., Pillow, J. W., Hasson, U., & Norman, K. A. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.041 · Cited by 952
Rogers, T. T., Lambon Ralph, M. A., Garrard, P., Bozeat, S., McClelland, J. L., Hodges, J. R., & Patterson, K. (2004) · Psychological Review · doi:10.1037/0033-295x.111.1.205 · Cited by 948
Self-citationSentence memory: A constructive versus interpretive approach
Bransford, J. D., Barclay, J. R., & Franks, J. J. (1972) · Cognitive Psychology · doi:10.1016/0010-0285(72)90003-5 · Cited by 908 · cited 1×
“Evidence that humans construct situation representations from language comes from classic work by Bransford and colleagues (36, 45).”
BackgroundWhat Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated
Kumaran, D., Hassabis, D., & McClelland, J. L. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.05.004 · Cited by 759
Self-citationAdversarial NLI: A New Benchmark for Natural Language Understanding
Nie, Y., Williams, A., Dinan, E., Bansal, M., Weston, J., & Kiela, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.441 · Cited by 656
Rumelhart, D. E., & McClelland, J. L. (1986) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · Cited by 553
Self-citationEmbodied meaning in a neural theory of language
Feldman, J., & Narayanan, S. (2004) · Brain and Language · doi:10.1016/s0093-934x(03)00355-9 · Cited by 503
DEEP STRUCTURE, SURFACE STRUCTURE AND SEMANTIC INTERPRETATION
(1972) · Studies on Semantics in Generative Grammar · doi:10.1515/9783110867589-004 · Cited by 495
On not being led up the garden path: the use of context by the psychological syntax processor
Crain, S., & Steedman, M. (1985) · Natural Language Parsing · doi:10.1017/cbo9780511597855.011 · Cited by 460
A contextual binding theory of episodic memory: systems consolidation reconsidered
Yonelinas, A. P., Ranganath, C., Ekstrom, A. D., & Wiltgen, B. J. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0150-4 · Cited by 450
Altmann, G. T. M., & Kamide, Y. (2007) · Journal of Memory and Language · doi:10.1016/j.jml.2006.12.004 · Cited by 434 · cited 1×
“Just after hearing “ The man will drink, ” participants look at a full wine glass rather than an empty beer glass (47).”
Background(1988) · The MIT Press eBooks · doi:10.7551/mitpress/2103.001.0001 · Cited by 390
Implementations are not conceptualizations: Revising the verb learning model
MacWhinney, B., & Leinbach, J. (1991) · Cognition · doi:10.1016/0010-0277(91)90048-9 · Cited by 373 · cited 1×
“Protagonists argue it supports nuanced, context-and similarity-sensitive processing that is reflected in the quasiregular relationships between phrases and their sounds, spellings, and meanings (8, 9).”
BackgroundBybee, J., & McClelland, J. L. (2005) · The Linguistic Review · doi:10.1515/tlir.2005.22.2-4.381 · Cited by 362 · cited 1×
“Protagonists argue it supports nuanced, context-and similarity-sensitive processing that is reflected in the quasiregular relationships between phrases and their sounds, spellings, and meanings (8, 9).”
BackgroundSelf-citationThe Algebraic Mind: Integrating Connectionism and Cognitive Science
Marcus, G. F. (2001) · doi:10.7551/mitpress/1187.001.0001 · Cited by 341
Emergent linguistic structure in artificial neural networks trained by self-supervision
Manning, C. D., Clark, K., Hewitt, J., Khandelwal, U., & Levy, O. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1907367117 · Cited by 339
Taraban, R., & McClelland, J. L. (1988) · Journal of Memory and Language · doi:10.1016/0749-596x(88)90011-3 · Cited by 322 · cited 1×
“However, if “ beard ” replaces “ bat, ” then “ with a beard ” is an attribute of the man and is treated as part of a noun phrase headed by “ the man ” (4).”
Self-citationLearning and applying contextual constraints in sentence comprehension
St. John, M. F., & McClelland, J. L. (1990) · Artificial Intelligence · doi:10.1016/0004-3702(90)90008-n · Cited by 322
Self-citationThe impaired learning of semantic knowledge following bilateral medial temporal-lobe resection
Gabrieli, J. D. E., Cohen, N. J., & Corkin, S. (1988) · Brain and Cognition · doi:10.1016/0278-2626(88)90027-9 · Cited by 311 · cited 1×
“People exploit such information during language processing (64), and patients with MTL damage have difficulty understanding or producing extended narratives (65); they are also profoundly impaired in learning new words for later use (66).”
BackgroundThe spontaneous self-organization of an adaptive language
Steels, L. (2000) · Machine Intelligence 15 · doi:10.1093/oso/9780198538677.003.0011 · Cited by 301
Modelling the N400 brain potential as change in a probabilistic representation of meaning
Rabovsky, M., Hansen, S. S., & McClelland, J. L. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0406-4 · Cited by 290 · cited 1×
“Words and their sequencing serve as clues to meaning (43) that jointly constrain the understanding of the situation (40, 44).”
BackgroundSelf-citationZadbood, A., Chen, J., Leong, Y. C., Norman, K. A., & Hasson, U. (2017) · Cerebral Cortex · doi:10.1093/cercor/bhx202 · Cited by 281
Zadbood, A., Chen, J., Leong, Y. C., Norman, K. A., & Hasson, U. (2016) · doi:10.1101/081208 · Cited by 281
Bisk, Y., Holtzman, A., Thomason, J., Andreas, J., Bengio, Y., Chai, J., Lapata, M., Lazaridou, A., May, J., Nisnevich, A., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.703 · Cited by 274
Syntactic Structure from Deep Learning
Linzen, T., & Baroni, M. (2021) · Annual Review of Linguistics · doi:10.1146/annurev-linguistics-032020-051035 · Cited by 202 · cited 1×
“That said, careful analysis (27) indicates that the deep network ’ s sensitivity to grammatical structure is still imperfect and only partially understood.”
BackgroundLearning Video Representations using Contrastive Bidirectional Transformer
Sun, C., Baradel, F., Murphy, K. P., & Schmid, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.05743 · Cited by 192
Hasson, U., Egidi, G., Marelli, M., & Willems, R. M. (2018) · Cognition · doi:10.1016/j.cognition.2018.06.018 · Cited by 157 · cited 4×
“The approach was adopted in an early neural network model (40) and aligns with other current perspectives in cognitive science (41), cognitive neuroscience (42), and artificial intelligence (32).”
BackgroundResultInfluentialQuestion Answering on Knowledge Bases and Text using Universal Schema and Memory Networks
Das, R., Zaheer, M., Reddy, S., & McCallum, A. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p17-2057 · Cited by 154
Some problems with the notion of literal meanings
Rumelhart, D. E. (1993) · Cambridge University Press eBooks · doi:10.1017/cbo9781139173865.007 · Cited by 146
McClelland, J. L., McNaughton, B. L., & Lampinen, A. K. (2020) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2019.0637 · Cited by 136
Self-citationGrounded Language Learning in a Simulated 3D World
Hermann, K. M., Hill, F., Green, S., Wang, F., Faulkner, R., Soyer, H., Szepesvári, D., Czarnecki, W. M., Jaderberg, M., Teplyashin, D., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.06551 · Cited by 132 · cited 1×
“These considerations motivate recent work on agent-based language learning in simulated interactive three-dimensional environments (73 – 76).”
BackgroundSelf-citationGated-Attention Architectures for Task-Oriented Language Grounding
Chaplot, D. S., Mysore Sathyendra, K., Pasumarthi, R. K., Rajagopal, D., & Salakhutdinov, R. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11832 · Cited by 125
Interactive Activation and Mutual Constraint Satisfaction in Perception and Cognition
McClelland, J. L., Mirman, D., Bolger, D. J., & Khaitan, P. (2014) · Cognitive Science · doi:10.1111/cogs.12146 · Cited by 122
Self-citationMenenti, L., Petersson, K. M., Scheeringa, R., & Hagoort, P. (2009) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2008.21163 · Cited by 114
Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning
Oh, J., Singh, S. P., Lee, H., & Kohli, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05064 · Cited by 103 · cited 1×
“These considerations motivate recent work on agent-based language learning in simulated interactive three-dimensional environments (73 – 76).”
BackgroundIs relational reasoning dependent on language? A voxel-based lesion symptom mapping study
Baldo, J. V., Bunge, S. A., Wilson, S. M., & Dronkers, N. F. (2010) · Brain and Language · doi:10.1016/j.bandl.2010.01.004 · Cited by 97 · cited 1×
“Indeed, the frontal and parietal lobes are associated with representation of space and action (which causes change in relations), and patients with lesions to the frontal and parietal language-related areas have profound deficits in relational reasoning tasks (57).”
BackgroundQuasiregularity and Its Discontents: The Legacy of the Past Tense Debate
Seidenberg, M. S., & Plaut, D. C. (2014) · Cognitive Science · doi:10.1111/cogs.12147 · Cited by 86
Environmental drivers of systematicity and generalization in a situated agent
Hill, F., Lampinen, A., Schneider, R., Clark, S., Botvinick, M., McClelland, J. L., & Santoro, A. (2019) · arXiv · doi:10.48550/arxiv.1910.00571 · Cited by 81 · cited 1×
“77, an agent was trained to identify, lift, carry, and place objects relative to other objects in a virtual room, as specified by simplified language instructions.”
BackgroundSelf-citationLake, B. M., & Baroni, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00350 · Cited by 76
Word contexts enhance the neural representation of individual letters in early visual cortex
Heilbron, M., Richter, D., Ekman, M., Hagoort, P., & de Lange, F. P. (2020) · Nature Communications · doi:10.1038/s41467-019-13996-4 · Cited by 48 · cited 1×
“Brain regions for representing visual and auditory inputs are well established, and the evidence for their involvement in a mutual constraint satisfaction process with more integrative brain areas has been reviewed elsewhere (50, 51).”
BackgroundPhrase similarity in humans and machines.
Gershman, S. J., & Tenenbaum, J. B. (2015) · Cited by 30
Temporal integration of narrative information in a hippocampal amnesic patient
Zuo, X., Honey, C. J., Barense, M. D., Crombie, D., Norman, K. A., Hasson, U., & Chen, J. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116658 · Cited by 30
Bryson, J. J. (2008) · Mind & Society · doi:10.1007/s11299-007-0044-4 · Cited by 22
Weston, J., Chopra, S., & Bordes, A. (2014) · arXiv · doi:10.48550/arxiv.1410.3916 · cited 1×
“While a superpositional, distributed memory (68) will ultimately be the best model of this system, an initial implementation could store each such vector in a separate slot (69, 70).”
Background
Research works printed but not linked: 4
- Rumelhart D. E., “Toward an interactive model of reading” in Attention & Performance VI, Dornic S., Ed. (LEA, Hillsdale, NJ, 1977), pp. 573–603. [Google Scholar]
- Manning C. D., Schütze H., Foundations of Statistical Language Processing (MIT Press, Cambridge, MA, 1999). [Google Scholar]
- Morton J., Patterson K., “Little words–no!” in Deep Dyslexia, Coltheart M., Patterson K., Marshall J. C., Eds. (Routledge and Kegan Paul, London, UK, 1980), pp. 270–285. [Google Scholar]
- Russin J., O’Reilly R. C., Bengio Y., “Deep learning needs a prefrontal cortex” in ICLR Workshop on Bridging AI and Cognitive Science (ICLR, 2020). [Google Scholar]
Cited by
Using cognitive psychology to understand GPT-3
Binz, M., & Schulz, E. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2218523120 · Cited by 559
Shared computational principles for language processing in humans and deep language models
Goldstein, A., Zada, Z., Buchnik, E., Schain, M., Price, A., Aubrey, B., Nastase, S. A., Feder, A., Emanuel, D., Cohen, A., et al. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01026-4 · Cited by 447
“Furthermore, DLMs avoid the circularity built into many psycholinguistic language models that rely on linguistic terms to explain how language is encoded in neural substrates 19,62 .”
BackgroundEmergent analogical reasoning in large language models
Webb, T., Holyoak, K. J., & Lu, H. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01659-w · Cited by 404
A hierarchy of linguistic predictions during natural language comprehension
Heilbron, M., Armeni, K., Schoffelen, J.-M., Hagoort, P., & de Lange, F. P. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2201968119 · Cited by 343
Inductive biases for deep learning of higher-level cognition
Goyal, A., & Bengio, Y. (2022) · Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rspa.2021.0068 · Cited by 300
Metadata
- Authors
- McClelland, J. L., Hill, F., Rudolph, M., Baldridge, J., & Schütze, H.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- Robert Bosch (Germany) (DE)
- Google (United States) (US)
- Ludwig-Maximilians-Universität München (DE)
- Corresponding author
- McClelland, James L.
- Hill, Felix
- Rudolph, Maja
- Baldridge, Jason
- Schütze, Hinrich
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2020-09-28
- Volume
- 117
- Issue
- 42
- Pages
- 25966-25974
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC7585006
- OpenAlex
- W3088418428
- Semantic Scholar
- 00b1e962182c42949822821dc1a929bd132ec082
- Cited by
- 91
- References
- 88
- Citations by year
- 2026: 1 · 2025: 16 · 2024: 18 · 2023: 24 · 2022: 22 · 2021: 7 · 2020: 3
- FWCI
- 7.52
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Open access LMU (Ludwid Maxmilian's Universitat Munchen) (submittedVersion, other-oa)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- language comprehension, neural language models, distributed representations
- MeSH terms
- Artificial Intelligence, Brain, Computer Simulation, Humans, Intelligence, Language, Neural Pathways, Neural Networks, Computer, Comprehension
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.