Do Language Models’ Words Refer?
Open accessThe DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
What do language models (LMs) do with language? They can produce sequences of (mostly) coherent strings closely resembling English. But do those sentences mean something, or are LMs simply babbling in a convincing simulacrum of language use? We address one aspect of this broad question: whether LMs’ words can refer, that is, achieve “word-to-world” connections. There is prima facie reason to think they do not, since LMs do not interact with the world in the way that ordinary language users do. Drawing on the externalist tradition in philosophy of language, we argue that those appearances are misleading: Even if the inputs to LMs are simply strings of text, they are strings of text with natural histories, and that may suffice for LMs’ words to refer.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Still, there are good reasons to resist the attribution of intentionality to AI models (Mandelkern & Linzen, 2024).”
From Are Large Language Models Intentional? The Limits of Referential Grounding (2026) · cites this paper as Mandelkern & Linzen, 2024
Citations
Cites
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“But some have argued that LMs are really nothing more than babbling “stochastic parrots” (a term created by Bender et al. 2021).”
BackgroundPutnam, H. (1981) · Cambridge University Press eBooks · doi:10.1017/cbo9780511625398 · Cited by 5,521
Clark, H. H. (1996) · doi:10.1017/cbo9780511620539 · Cited by 5,123
(2010) · Mind, Language and Reality · doi:10.1017/cbo9780511625251.014 · Cited by 3,767
Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data
Bender, E. M., & Koller, A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.463 · Cited by 1,082
Speaker's Reference and Semantic Reference
Kripke, S., & Philosophy Documentation Center (1977) · Midwest Studies in Philosophy · doi:10.1111/j.1475-4975.1977.tb00045.x · Cited by 855 · cited 1×
“A closely related question, which we will not tackle head-on, is whether the LM itself refers (whether it achieves speaker reference ), which can in principle come apart from the question of whether its words refer (Kripke 1977).”
BackgroundPatrick, P. L. (2008) · Wiley eBooks · doi:10.1002/9780470756591.ch23 · Cited by 511
Kaplan, D. (1990) · Aristotelian Society Supplementary Volume · doi:10.1093/aristoteliansupp/64.1.93 · Cited by 315
Making AI Intelligible: Philosophical Foundations
Cappelen, H., & Dever, J. (2021) · doi:10.1093/oso/9780192894724.001.0001 · Cited by 100
Symbols and grounding in large language models
Pavlick, E. (2023) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2022.0041 · Cited by 95
Language Models as Agent Models
Andreas, J. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.423 · Cited by 92
Language Models as Agent Models
Andreas, J. (2022) · arXiv · doi:10.48550/arxiv.2212.01681 · Cited by 92
Let’s Play Mono - Poly : BERT Can Reveal Words’ Polysemy Level and Partitionability into Senses
Soler, A. G., & Apidianaki, M. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00400 · Cited by 87 · cited 1×
“…contextualized vector representations of the instances of an ambiguous word can be clustered into different locations in vector space for each sense of the word (see also Gar´ı Soler and Apidianaki 2021); as such, the sense of a particular use of a word can often be inferred from its embedding.”
BackgroundDoes ChatGPT have semantic understanding? A problem with the statistics-of-occurrence strategy
Titus, L. M. (2024) · Cognitive Systems Research · doi:10.1016/j.cogsys.2023.101174 · Cited by 82
Meaning without reference in large language models
Piantadosi, S. T., & Hill, F. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.02957 · Cited by 65
What Are Words? Comments on Kaplan (1990), on Hawthorne and Lepore, and on the Issue
Bromberger, S. (2011) · The Journal of Philosophy · doi:10.5840/2011108925 · Cited by 58
Coelho Mollo, D., & Millière, R. (2026) · Philosophy and the Mind Sciences · doi:10.33735/phimisci.2026.12307 · Cited by 48
Mollo, D. C., & Millière, R. (2023) · arXiv · doi:10.48550/arxiv.2304.01481 · Cited by 47
Sharing Our Concepts with Machines
Butlin, P. (2023) · Erkenntnis · doi:10.1007/s10670-021-00491-w · Cited by 22 · cited 3×
“By contrast, we will argue that standard externalist accounts are at least prima facie compelling with respect to LMs. Butlin (2021) argues that externalism vindicates referentiality for chatbots but not for LMs. DeVault, Oved, and Stone (2006) argued that externalism did not vindicate referentiality for NLP as it exi…”
BackgroundSocietal grounding is essential to meaningful language use
Vault, D. D., Oved, I., & Stone, M. (2006) · National Conference on Artificial Intelligence · Cited by 18
Lederman, H., & Mahowald, K. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.04854 · Cited by 18 · cited 1×
“For instance, Lederman and Mahowald (2024) argue that LMs are not part of our speech communities, but their words nevertheless refer because they are causally sensitive to the intelligibility of their data.”
BackgroundEntailment Semantics Can Be Extracted from an Ideal Language Model
Merrill, W., Warstadt, A., & Linzen, T. (2022) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2022.conll-1.13 · Cited by 16
Self-citationLet's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into SensesPreprint
Soler, A. G., & Apidianaki, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.14694 · Cited by 16 · cited 1×
“…contextualized vector representations of the instances of an ambiguous word can be clustered into different locations in vector space for each sense of the word (see also Gar´ı Soler and Apidianaki 2021); as such, the sense of a particular use of a word can often be inferred from its embedding.”
BackgroundMorgan, M. H. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9781139151269.002 · Cited by 3
Research works printed but not linked: 3
- Coenen, 2019, Visualizing and measuring the geometry of BERT, Arxiv preprint arxiv 1906.02715 · arXiv
- Evans, 1982, The Varieties of Reference
- Kripke, 1980, Naming and Necessity
Cited by
Hicks, M. T., Humphries, J., & Slater, J. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09775-5 · Cited by 456
“And while there is considerable disagreement concerning whether ChatGPT has intentions, it’s widely agreed that the sentences it produces are (typically) meaningful (see e.g. Mandelkern and Linzen 2023).”
Still no lie detector for language models: probing empirical and conceptual roadblocks
Levinstein, B. A., & Herrmann, D. A. (2025) · Philosophical Studies · doi:10.1007/s11098-023-02094-3 · Cited by 50
Coelho Mollo, D., & Millière, R. (2026) · Philosophy and the Mind Sciences · doi:10.33735/phimisci.2026.12307 · Cited by 48
Mollo, D. C., & Millière, R. (2023) · arXiv · doi:10.48550/arxiv.2304.01481 · Cited by 47
Moderating Synthetic Content: the Challenge of Generative AI
Fisher, S. A., Howard, J. W., & Kira, B. (2024) · Philosophy & Technology · doi:10.1007/s13347-024-00818-9 · Cited by 37
Metadata
- Authors
- Mandelkern, M., & Linzen, T.
- Affiliations
- New York University (US)
- Corresponding author
- Mandelkern, Matthew
- Linzen, Tal
- Venue
- Computational Linguistics
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 50
- Issue
- 3
- Pages
- 1191-1200
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W4399492998
- Cited by
- 77
- References
- 22
- Citations by year
- 2026: 29 · 2025: 37 · 2024: 11
- FWCI
- 120.71
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Computational Linguistics (publishedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › Syntax, Semantics, Linguistic Variation
- Social Sciences › Arts and Humanities › Philosophy › Philosophy and Theoretical Science
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
- Keywords
- language models, reference, externalism, philosophy of language, meaning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.