Do Language Models’ Words Refer?
Open accessThe DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
What do language models (LMs) do with language? They can produce sequences of (mostly) coherent strings closely resembling English. But do those sentences mean something, or are LMs simply babbling in a convincing simulacrum of language use? We address one aspect of this broad question: whether LMs’ words can refer, that is, achieve “word-to-world” connections. There is prima facie reason to think they do not, since LMs do not interact with the world in the way that ordinary language users do. Drawing on the externalist tradition in philosophy of language, we argue that those appearances are misleading: Even if the inputs to LMs are simply strings of text, they are strings of text with natural histories, and that may suffice for LMs’ words to refer.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Still, there are good reasons to resist the attribution of intentionality to AI models (Mandelkern & Linzen, 2024).”
From Are Large Language Models Intentional? The Limits of Referential Grounding (2026) · cites this paper as Mandelkern & Linzen, 2024
Citations
Cites
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“But some have argued that LMs are really nothing more than babbling “stochastic parrots” (a term created by Bender et al. 2021).”
BackgroundPutnam, H. (1981) · Cambridge University Press eBooks · doi:10.1017/cbo9780511625398 · Cited by 5,521
Clark, H. H. (1996) · doi:10.1017/cbo9780511620539 · Cited by 5,123
(2010) · Mind, Language and Reality · doi:10.1017/cbo9780511625251.014 · Cited by 3,767
Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data
Bender, E. M., & Koller, A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.463 · Cited by 1,082
Speaker's Reference and Semantic Reference
Kripke, S., & Philosophy Documentation Center (1977) · Midwest Studies in Philosophy · doi:10.1111/j.1475-4975.1977.tb00045.x · Cited by 855 · cited 1×
“A closely related question, which we will not tackle head-on, is whether the LM itself refers (whether it achieves speaker reference ), which can in principle come apart from the question of whether its words refer (Kripke 1977).”
BackgroundPatrick, P. L. (2008) · Wiley eBooks · doi:10.1002/9780470756591.ch23 · Cited by 511
Kaplan, D. (1990) · Aristotelian Society Supplementary Volume · doi:10.1093/aristoteliansupp/64.1.93 · Cited by 315
Making AI Intelligible: Philosophical Foundations
Cappelen, H., & Dever, J. (2021) · doi:10.1093/oso/9780192894724.001.0001 · Cited by 100
Symbols and grounding in large language models
Pavlick, E. (2023) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2022.0041 · Cited by 95
Research works printed but not linked: 3
- Coenen, 2019, Visualizing and measuring the geometry of BERT, Arxiv preprint arxiv 1906.02715 · arXiv
- Evans, 1982, The Varieties of Reference
- Kripke, 1980, Naming and Necessity
Cited by
Hicks, M. T., Humphries, J., & Slater, J. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09775-5 · Cited by 456
“And while there is considerable disagreement concerning whether ChatGPT has intentions, it’s widely agreed that the sentences it produces are (typically) meaningful (see e.g. Mandelkern and Linzen 2023).”
Still no lie detector for language models: probing empirical and conceptual roadblocks
Levinstein, B. A., & Herrmann, D. A. (2025) · Philosophical Studies · doi:10.1007/s11098-023-02094-3 · Cited by 50
Coelho Mollo, D., & Millière, R. (2026) · Philosophy and the Mind Sciences · doi:10.33735/phimisci.2026.12307 · Cited by 48
Mollo, D. C., & Millière, R. (2023) · arXiv · doi:10.48550/arxiv.2304.01481 · Cited by 47
Moderating Synthetic Content: the Challenge of Generative AI
Fisher, S. A., Howard, J. W., & Kira, B. (2024) · Philosophy & Technology · doi:10.1007/s13347-024-00818-9 · Cited by 37
Metadata
- Authors
- Mandelkern, M., & Linzen, T.
- Affiliations
- New York University (US)
- Corresponding author
- Mandelkern, Matthew
- Linzen, Tal
- Venue
- Computational Linguistics
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 50
- Issue
- 3
- Pages
- 1191-1200
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W4399492998
- Cited by
- 77
- References
- 22
- Citations by year
- 2026: 29 · 2025: 37 · 2024: 11
- FWCI
- 120.71
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Computational Linguistics (publishedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › Syntax, Semantics, Linguistic Variation
- Social Sciences › Arts and Humanities › Philosophy › Philosophy and Theoretical Science
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
- Keywords
- language models, reference, externalism, philosophy of language, meaning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Do Language Models’ Words Refer? https://app.cochat.ai/papers/10.1162/coli_a_00522
- The paper
- Mandelkern, M., & Linzen, T. (2024). Do Language Models’ Words Refer? Computational Linguistics. https://doi.org/10.1162/coli_a_00522
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.