Distributional Semantics: Meaning Through Culture and Interaction
Abstract
Mastering how to convey meanings using language is perhaps the main challenge facing any language learner. However, satisfactory accounts of how this is achieved, and even of what it is for a linguistic item to have meaning, are hard to come by. Nick Chater was one of the pioneers involved in the early development of one of the most successful methodologies within the cognitive science of language for discovering meaning: distributional semantics. In this article, we review this approach and discuss its successes and shortcomings in capturing semantic phenomena. In particular, we discuss what we dub the "distributional paradox:" how can models that do not implement essential dimensions of human semantic processing, such as sensorimotor grounding, capture so many meaning-related phenomena? We conclude by providing a preliminary answer, arguing that distributional models capture the statistical scaffolding of human language acquisition that allows for communication, which, in line with Nick Chater's more recent ideas, has been shaped by the features of human cognition on the timescale of cultural evolution.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLMs approximate data distributions rather than explicitly understanding knowledge.
“This view aligns with recent theoretical accounts positioning LLMs as probabilistic pattern recognizers that approximate data distributions rather than explicitly 'understanding' knowledge [21, 22].”
From Hallucinations in Bibliographic Recommendation: Citation Frequency as a Proxy for Training Data Redundancy (2025) · cites this paper as [22]
Cites
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 15,880 · see citing works
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · see citing works
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
Harnad, S. (1990) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(90)90087-6 · Cited by 4,235 · cited 3× · see citing works
“Indeed, this was the solution to the symbol grounding problem that Harnad (1990) envisaged, based on early connectionist modeling (for a discussion of the philosophical difficulties facing this approach, see Christiansen & Chater, 1992, 1993).”
Toward a mechanistic psychology of dialogue.
Pickering, M. J., & Garrod, S. (2004) · Behavioral and Brain Sciences · doi:10.1017/s0140525x04000056 · Cited by 2,678 · see citing works
Hierarchical Text-Conditional Image Generation with CLIP Latents
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06125 · Cited by 2,511 · cited 2× · see citing works
“Future work could explore how similar the linkages in these models representations of meaning are to the ones assumed to underlie human language use.”
Distributed representations, simple recurrent networks, and grammatical structure
Elman, J. L. (1991) · Machine Learning · doi:10.1007/bf00114844 · Cited by 1,354 · see citing works
CONSTRUCTING A LANGUAGE: A USAGE-BASED THEORY OF LANGUAGE ACQUISITION
Foster-Cohen, S. (2004) · Studies in Second Language Acquisition · doi:10.1017/s0272263104363059 · Cited by 969 · see citing works
Ethayarajh, K. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1006 · Cited by 841 · see citing works
Distributional Information: A Powerful Cue for Acquiring Syntactic Categories
Redington, M., Chater, N., & Finch, S. (1998) · Cognitive Science · doi:10.1207/s15516709cog2204_2 · Cited by 535 · cited 1× · see citing works
“Further distributional analyses in Redington, Chater, and Finch (1998) revealed that distributional information was more useful for classifying content words (nouns, verbs, etc.) than function words (determiners, prepositions, etc.).”
Dissecting Contextual Word Embeddings: Architecture and Representation
Peters, M., Neumann, M., Zettlemoyer, L., & Yih, W.-T. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1179 · Cited by 434 · see citing works
Cultural bias and cultural alignment of large language models
Tao, Y., Viberg, O., Baker, R. S., & Kizilcec, R. F. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae346 · Cited by 383 · cited 1× · see citing works
“…explains also why these interactions—and the “human-likeness” of LLMs more generally—tend to be mediated by cultural biases in relation to, specifically, American English, given the characteristics of their training data (Atari et al., 2023; Cao et al, 2023; Johnson et al., 2022; Tao et al, 2024).”
Incorporating Context into Language Encoding Models for fMRI
Jain, S., & Huth, A. G. (2018) · doi:10.1101/327601 · Cited by 155 · cited 1× · see citing works
“For example, LLMs are better at capturing the granularity of the brain’s semantic space (Sun et al., 2020) thanks to the inclusion of context (Goldstein et al., 2022; Jain & Huth, 2018) and the optimization of the space for in-context prediction (Caucheteux & King, 2022; Goldstein et al., 2022).”
Atari, M., Xue, M. J., Park, P. S., Blasi, D. E., & Henrich, J. (2023) · doi:10.31234/osf.io/5b26t · Cited by 151 · see citing works
Oh, B.-D., & Schuler, W. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00548 · Cited by 107 · see citing works
From Word Embeddings to Pre-Trained Language Models: A State-of-the-Art Walkthrough
Mars, M. (2022) · Applied Sciences · doi:10.3390/app12178805 · Cited by 91 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Niimi, J. (2025) · arXiv · doi:10.48550/arxiv.2510.25378
“This view aligns with recent theoretical accounts positioning LLMs as probabilistic pattern recognizers that approximate data distributions rather than explicitly “understanding” knowledge [21, 22].”
- Authors
- Contreras Kallens, P., & Christiansen, M. H.
- Affiliations
- Department of Language Science and Technology Saarland University
- Aarhus University (DK)
- Corresponding author
- Christiansen, Morten H.
- Venue
- Topics in Cognitive Science
- Publisher
- Wiley
- Published
- 2024-11-26
- Volume
- 17
- Issue
- 3
- Pages
- 739-769
- ISSN
- 1756-8757 (ISSN-L)
- 1756-8765
- OpenAlex
- W4404723816
- Cited by
- 17
- References
- 16
- Citations by year
- 2026: 5 · 2025: 11 · 2024: 1
- FWCI
- 1.26
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Topics in Cognitive Science (publishedVersion)
- PubMed (publishedVersion)
- Scopus (Elsevier) (submittedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Categorization, perception, and language
- Social Sciences › Cultural Studies › Language and cultural evolution
- Social Sciences › Arts and Humanities › Language and Linguistics › Syntax, Semantics, Linguistic Variation
- Keywords
- distributional semantics, meaning, sensorimotor grounding, language acquisition, statistical learning, cultural evolution, communication, cognitive linguistics, word meaning
- MeSH terms
- Culture, Humans, Language, Language Development, Psycholinguistics, Semantics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
