Do Language Models’ Words Refer?
Open accessThe DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
What do language models (LMs) do with language? They can produce sequences of (mostly) coherent strings closely resembling English. But do those sentences mean something, or are LMs simply babbling in a convincing simulacrum of language use? We address one aspect of this broad question: whether LMs’ words can refer, that is, achieve “word-to-world” connections. There is prima facie reason to think they do not, since LMs do not interact with the world in the way that ordinary language users do. Drawing on the externalist tradition in philosophy of language, we argue that those appearances are misleading: Even if the inputs to LMs are simply strings of text, they are strings of text with natural histories, and that may suffice for LMs’ words to refer.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Still, there are good reasons to resist the attribution of intentionality to AI models (Mandelkern & Linzen, 2024).”
From Are Large Language Models Intentional? The Limits of Referential Grounding (2026) · cites this paper as Mandelkern & Linzen, 2024
Citations
Cites
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“But some have argued that LMs are really nothing more than babbling “stochastic parrots” (a term created by Bender et al. 2021).”
BackgroundPutnam, H. (1981) · Cambridge University Press eBooks · doi:10.1017/cbo9780511625398 · Cited by 5,521
Clark, H. H. (1996) · doi:10.1017/cbo9780511620539 · Cited by 5,123
(2010) · Mind, Language and Reality · doi:10.1017/cbo9780511625251.014 · Cited by 3,767
Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data
Bender, E. M., & Koller, A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.463 · Cited by 1,082
Speaker's Reference and Semantic Reference
Kripke, S., & Philosophy Documentation Center (1977) · Midwest Studies in Philosophy · doi:10.1111/j.1475-4975.1977.tb00045.x · Cited by 855 · cited 1×
“A closely related question, which we will not tackle head-on, is whether the LM itself refers (whether it achieves speaker reference ), which can in principle come apart from the question of whether its words refer (Kripke 1977).”
BackgroundPatrick, P. L. (2008) · Wiley eBooks · doi:10.1002/9780470756591.ch23 · Cited by 511
Kaplan, D. (1990) · Aristotelian Society Supplementary Volume · doi:10.1093/aristoteliansupp/64.1.93 · Cited by 315
Making AI Intelligible: Philosophical Foundations
Cappelen, H., & Dever, J. (2021) · doi:10.1093/oso/9780192894724.001.0001 · Cited by 100
Symbols and grounding in large language models
Pavlick, E. (2023) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2022.0041 · Cited by 95
Research works printed but not linked: 3
- Coenen, 2019, Visualizing and measuring the geometry of BERT, Arxiv preprint arxiv 1906.02715 · arXiv
- Evans, 1982, The Varieties of Reference
- Kripke, 1980, Naming and Necessity
Cited by
Hicks, M. T., Humphries, J., & Slater, J. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09775-5 · Cited by 456
“And while there is considerable disagreement concerning whether ChatGPT has intentions, it’s widely agreed that the sentences it produces are (typically) meaningful (see e.g. Mandelkern and Linzen 2023).”
Still no lie detector for language models: probing empirical and conceptual roadblocks
Levinstein, B. A., & Herrmann, D. A. (2025) · Philosophical Studies · doi:10.1007/s11098-023-02094-3 · Cited by 50
Coelho Mollo, D., & Millière, R. (2026) · Philosophy and the Mind Sciences · doi:10.33735/phimisci.2026.12307 · Cited by 48
Mollo, D. C., & Millière, R. (2023) · arXiv · doi:10.48550/arxiv.2304.01481 · Cited by 47
Moderating Synthetic Content: the Challenge of Generative AI
Fisher, S. A., Howard, J. W., & Kira, B. (2024) · Philosophy & Technology · doi:10.1007/s13347-024-00818-9 · Cited by 37
Operationalising Representation in Natural Language Processing
Harding, J. (2023) · The British Journal for the Philosophy of Science · doi:10.1086/728685 · Cited by 53
“In the context of modern NLP, the characterisation of the task function D also bears on a foundational question: do ‘the internal representations and outputs of Large Language Models possess intrinsic meaning, enabling them to represent things in the world independently of human interpretation’ (Mollo and Milli`ere 20…”
BackgroundInfluentialLarge language models and linguistic intentionality
Grindrod, J. (2024) · Synthese · doi:10.1007/s11229-024-04723-8 · Cited by 50
Large language models and their big bullshit potential
Fisher, S. A. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09802-5 · Cited by 35
Standards for Belief Representations in LLMs
Herrmann, D. A., & Levinstein, B. A. (2024) · Minds and Machines · doi:10.1007/s11023-024-09709-6 · Cited by 83
Meaning and understanding in large language models
Havlík, V. (2024) · Synthese · doi:10.1007/s11229-024-04878-4 · Cited by 28
“All understanding rests on syntactic understanding.”
Nefdt, R. M. (2025) · Cambridge University Press eBooks · doi:10.1017/9781009491952 · Cited by 29
LLMs, Turing tests and Chinese rooms: the prospects for meaning in large language models
Borg, E. (2025) · Inquiry · doi:10.1080/0020174x.2024.2446241 · Cited by 30
Humans Learn Language from Situated Communicative Interactions. What about Machines?
Beuls, K., & Eecke, P. V. (2024) · Computational Linguistics · doi:10.1162/coli_a_00534 · Cited by 37
Dentella, V., Günther, F., & Leivada, E. (2025) · PLoS ONE · doi:10.1371/journal.pone.0327794 · Cited by 22
From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency
Ohmer, X. I., Bruni, E., & Hupkes, D. (2024) · Computational Linguistics · doi:10.1162/coli_a_00529 · Cited by 36
UTILIZING RETRIEVAL-AUGMENTED GENERATION IN LARGE LANGUAGE MODELS TO ENHANCE INDONESIAN LANGUAGE NLP
Tohir, H., Merlina, N., & Haris, M. (2024) · JITK (Jurnal Ilmu Pengetahuan dan Teknologi Komputer) · doi:10.33480/jitk.v10i2.5916 · Cited by 11
Reference without intentions in large language models
Pepp, J. (2025) · Inquiry · doi:10.1080/0020174x.2024.2448482 · Cited by 7
“…reasonable to think that it, too, can simply borrow the reference of names like ‘ Jimmy Carter ’ and produce tokens of that name that refer to Jimmy Carter (For this general line of thought, see, e.g. Koch (2023), Mandelkern and Linzen (2024), Grindrod (2024), and Lederman and Mahowald (2024)).”
Donner, S. (2025) · Journal of Documentation · doi:10.1108/jd-11-2024-0271 · Cited by 8
Do Vision and Language Models Share Concepts? A Vector Space Alignment Study
Li, J., Kementchedjhieva, Y., Fierro, C., & Søgaard, A. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00698 · Cited by 6
Are Large Language Models Intentional? The Limits of Referential Grounding
García-Valdecasas, M. (2026) · Philosophy & Technology · doi:10.1007/s13347-026-01079-4 · Cited by 6
“In the context of LLMs, the symbol grounding problem has been reframed as the “vector grounding problem” (Mollo & Millière, 2023), that is, the challenge of providing a non-circular account for how the entire set of the numerical and geometric properties of a high-dimensional vector space acquires meaning, and how the…”
InfluentialMachines built out of other people's words: Comment on Helen Kelly‐Holmes’ discussion article
Gershon, I., & Handman, C. (2024) · Journal of Sociolinguistics · doi:10.1111/josl.12684 · Cited by 6
Truth without belief: can LLM-generated content satisfy classical theories of truth?
Zhang, X., & Li, H. (2026) · AI and Ethics · doi:10.1007/s43681-026-01065-8 · Cited by 6
“This is compatible with Mandelkern and Linzen’s sug - gestion that reference may be inherited from the training data and anchored by downstream use [33].”
Lehtonen, E., Gröndahl, T., & Nordhoff, S. (2025) · IEEE Access · doi:10.1109/access.2025.3546565 · Cited by 5
“…though SBERT is specifically trained to track semantic similarity, this does not resolve more fundamental theoretical questions concerning semantic representation in LLMs – related to issues such as environmental grounding, reference, or representational format, e.g. [23], [43], [44], [45], [46].”
Three dimensions of speech coherence in people with early psychosis and their family members
Çokal, D., Aloraini, A., Palominos, C., Demirlek, C., Verim, B., Yalınçetin, B., Bora, E., & Hinzen, W. (2025) · Schizophrenia · doi:10.1038/s41537-025-00703-0 · Cited by 4
“Another issue is how semantic similarity is related to referential structure, and whether the former notion can illuminate the latter, regardless of the LLM used, given unclarity over whether LLMs capture referential meaning at all 24 .”
Keane, W. (2026) · Journal of the Royal Anthropological Institute · doi:10.1111/1467-9655.70133 · Cited by 5
AI Romance and Misogyny: A Speech Act Analysis
Holdier, A. G., & Weirich, K. (2025) · Oxford University Press eBooks · doi:10.1093/9780198945215.003.0074 · Cited by 6
Linguistic Relativity: An Essential Guide to Past Debates and Future Prospects
Pelletier, F. J., & Nefdt, R. M. (2025) · doi:10.1093/9780197799871.001.0001 · Cited by 6
Cohnitz, D., & Haukioja, J. (2025) · doi:10.1093/9780191915925.001.0001 · Cited by 6
Babbling stochastic parrots? A Kripkean argument for reference in large language modelsPreprint
Koch, S. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/pj6tg · Cited by 6
Language Models and Externalism: A Reply to Mandelkern and Linzen
Ostertag, G. (2025) · Computational Linguistics · doi:10.1162/coli_a_00551 · Cited by 2
Routledge, A. J. (2025) · Ethics and Information Technology · doi:10.1007/s10676-025-09869-8 · Cited by 4
ChatGPT is not bullshit, nor is it not not bullshit
Fitts, J. (2026) · Ethics and Information Technology · doi:10.1007/s10676-026-09903-3 · Cited by 1
“Hicks et al. ( 2024 ) cite Mandelkern & Linzen ( 2024 ) to support the claim that the utterances of ChatGPT are meaningful.”
Peirce's revenge on the Chinese Room
Loginov, E., Vasilyev, V. V., Kuznetsov, A. V., Volkov, D. B., Беседин, А. П., & Mertsalov, A. V. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1900073 · Cited by 1
Lan, Z., & Yang, J. (2026) · IEEE Access · doi:10.1109/access.2026.3740087 · Cited by 1
“Mandelkern and Linzen [20] apply this lesson to language models by arguing that corpora have causal histories that may support derivative reference, while Ostertag [21] challenges a quick move from such histories to model reference.”
InfluentialOyelade, O. N., Wang, H., & Rafferty, K. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127455 · Cited by 7
Bounding Reason: Inferentialism, Naturalism, and the Discursive Agency of LLMs
Malík, J., & Hubálek, M. (2025) · Global Philosophy · doi:10.1007/s10516-025-09761-6 · Cited by 1
Knowledge in Animals and Machines
Paul, L. A. (2026) · Journal of Neuroscience · doi:10.1523/jneurosci.0939-25.2025 · Cited by 1
Why Large Language Models Can Say but Cannot Imply
Özdil, M. S. (2026) · Turkish Academic Research Review - Türk Akademik Araştırmalar Dergisi [TARR] · doi:10.30622/tarr.1863625 · Cited by 1
Linguistic Relativity and Cognitive Science
Pelletier, F. J., & Nefdt, R. M. (2025) · Oxford University Press eBooks · doi:10.1093/9780197799871.003.0005 · Cited by 1
Basic Linguistic Relativity A Quick Sketch
Pelletier, F. J., & Nefdt, R. M. (2025) · Oxford University Press eBooks · doi:10.1093/9780197799871.003.0001
Conclusion: Whorf and Relativity—Yes? or No?
Pelletier, F. J., & Nefdt, R. M. (2025) · Oxford University Press eBooks · doi:10.1093/9780197799871.003.0006
Pelletier, F. J., & Nefdt, R. M. (2025) · Oxford University Press eBooks · doi:10.1093/9780197799871.002.0004
Pelletier, F. J., & Nefdt, R. M. (2025) · Oxford University Press eBooks · doi:10.1093/9780197799871.003.0007
Edward Sapir and Benjamin Lee Whorf: Lives, Research, and Whorfianism
Pelletier, F. J., & Nefdt, R. M. (2025) · Oxford University Press eBooks · doi:10.1093/9780197799871.003.0003
Three Interpretations of Linguistic Relativity
Pelletier, F. J., & Nefdt, R. M. (2025) · Oxford University Press eBooks · doi:10.1093/9780197799871.003.0004
Some Historical Antecedents of Linguistic Relativity
Pelletier, F. J., & Nefdt, R. M. (2025) · Oxford University Press eBooks · doi:10.1093/9780197799871.003.0002
Cohnitz, D., & Haukioja, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780191915925.003.0010
Dispositionalist Meta-Internalism
Cohnitz, D., & Haukioja, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780191915925.003.0005
Methodological Consequences for Philosophy of Language
Cohnitz, D., & Haukioja, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780191915925.003.0007
Cohnitz, D., & Haukioja, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780191915925.003.0004
Examples of Meta-Internalism and Meta-Externalism
Cohnitz, D., & Haukioja, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780191915925.003.0003
Cohnitz, D., & Haukioja, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780191915925.003.0009
Reference Failure, Reference Change, and Conceptual Engineering
Cohnitz, D., & Haukioja, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780191915925.003.0008
Cohnitz, D., & Haukioja, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780191915925.002.0004
Cohnitz, D., & Haukioja, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780191915925.003.0001
Cohnitz, D., & Haukioja, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780191915925.003.0006
Meta-Externalism and Meta-Internalism
Cohnitz, D., & Haukioja, J. (2025) · Oxford University Press eBooks · doi:10.1093/9780191915925.003.0002
A Two-Step, Multidimensional Account of Deception in Language Models
Dung, L. (2025) · Erkenntnis · doi:10.1007/s10670-025-01017-4 · Cited by 3
“…mindreading, introspection, and having intentions which are the focus of recent discussion on the cognitive capacities of AI (Kammerer & Frankish, 2023), and large-language models (LLMs) in particular (Bender et al., 2021; Binder et al. 2024; Mandelkern & Linzen, 2024; Pi et al. 2024; Ullman 2023).”
Karami, B., Zandi, F., & Hatami, J. (2025) · Research Square · doi:10.21203/rs.3.rs-8270539/v1 · Cited by 1
Grajales, J. F. S., Cardona-Román, D. M., & Roa, Á. A. C. (2025) · Investigación e Innovación en Ingenierías · doi:10.17081/invinno.13.2.8059
Beckmann, P. (2025) · Synthese · doi:10.1007/s11229-025-05174-5 · Cited by 4
Digital Minds: Issues in the Philosophy of Mind and Cognitive Science
Saad, B., & Mogensen, A. L. (2026) · Philosophy Compass · doi:10.1111/phc3.70141
Cojocariu, F. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6285878
The construct “renixa” in digital discourse and its recognition using LLM’s agents
Goryunova, N. (2026) · Journal of Psycholinguistic · doi:10.30982/2077-5911-2026-67-1-19-34
Stepping Into the Black Box: Opening Up LLMs to Public Exploration Through Discursive Design
Immel, S., Rajani, N., Verweij, R., Li, J., Nissen, B., Soares, L. L., & Taylor, A. S. (2026) · Designing Interactive Systems Conference · doi:10.1145/3800645.3812976 · Cited by 1
The case for humanities-based studies of AI production and effects
Asscher, O. (2026) · AI & Society · doi:10.1007/s00146-026-03170-y
“…GAI’s linguistic competence, as in the processing of syntactical structures, or logical-semantic abilities, as in comprehension of logical semantics (Mandelkern and Linzen 2024; Mahowald et al. 2024); probing constructions of self-consciousness (Tani and White 2020); or exploring how…”
The Iterated Social Construction of Words
Antonov, A. V. (2026) · Grazer Philosophische Studien · doi:10.1163/18756735-00000270
The invention of interlanguages
Handman, C. (2026) · Oxford University Press eBooks · doi:10.1093/oso/9780197762554.003.0008
Les grands modèles de langue : bilan et perspectives pour la linguistique
Poibeau, T. (2026) · Langages · doi:10.3917/lang.243.0073
Meleshchuk, A. (2026) · ГРААЛЬ НАУКИ · doi:10.36074/grail-of-science.21.08.2026.027
“Якщо мовна модель послідовно встановлює коректні логічні зв'язки, виявляє аналогії, інтегрує інформацію з різних контекстів і демонструє здатність прогнозувати раніше не спостережувані ситуації, це свідчить про наявність функціонально ефективної репрезентаційної системи [6; 10].”
InfluentialLan, Z. (2026) · Signs and Society · doi:10.1017/sas.2026.10076
A Scientific Theory of Consciousness
Goldstein, S., & Kirk‐Giannini, C. D. (2026) · Oxford University Press eBooks · doi:10.1093/9780197840375.003.0009
Goldstein, S., & Kirk‐Giannini, C. D. (2026) · Oxford University Press eBooks · doi:10.1093/9780197840375.003.0002
Goldstein, S., & Kirk‐Giannini, C. D. (2026) · Oxford University Press eBooks · doi:10.1093/9780197840375.003.0008
Goldstein, S., & Kirk‐Giannini, C. D. (2026) · Oxford University Press eBooks · doi:10.1093/9780197840375.003.0001
Goldstein, S., & Kirk‐Giannini, C. D. (2026) · Oxford University Press eBooks · doi:10.1093/9780197840375.003.0010
Goldstein, S., & Kirk‐Giannini, C. D. (2026) · Oxford University Press eBooks · doi:10.1093/9780197840375.003.0011
Goldstein, S., & Kirk‐Giannini, C. D. (2026) · Oxford University Press eBooks · doi:10.1093/9780197840375.003.0004
Goldstein, S., & Kirk‐Giannini, C. D. (2026) · Oxford University Press eBooks · doi:10.1093/9780197840375.003.0013
Goldstein, S., & Kirk‐Giannini, C. D. (2026) · Oxford University Press eBooks · doi:10.1093/9780197840375.003.0006
Goldstein, S., & Kirk‐Giannini, C. D. (2026) · Oxford University Press eBooks · doi:10.1093/9780197840375.003.0017
Goldstein, S., & Kirk‐Giannini, C. D. (2026) · Oxford University Press eBooks · doi:10.1093/9780197840375.003.0015
Goldstein, S., & Kirk‐Giannini, C. D. (2026) · Oxford University Press eBooks · doi:10.1093/9780197840375.003.0014
Belief and Desire in AI Systems
Goldstein, S., & Kirk‐Giannini, C. D. (2026) · Oxford University Press eBooks · doi:10.1093/9780197840375.003.0005
Goldstein, S., & Kirk‐Giannini, C. D. (2026) · Oxford University Press eBooks · doi:10.1093/9780197840375.003.0016
AI Welfare: Agency, Consciousness, Sentience
Goldstein, S., & Kirk‐Giannini, C. D. (2026) · doi:10.1093/9780197840375.001.0001 · Cited by 5
Metadata
- Authors
- Mandelkern, M., & Linzen, T.
- Affiliations
- New York University (US)
- Corresponding author
- Mandelkern, Matthew
- Linzen, Tal
- Venue
- Computational Linguistics
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 50
- Issue
- 3
- Pages
- 1191-1200
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W4399492998
- Cited by
- 77
- References
- 22
- Citations by year
- 2026: 29 · 2025: 37 · 2024: 11
- FWCI
- 120.71
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Computational Linguistics (publishedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › Syntax, Semantics, Linguistic Variation
- Social Sciences › Arts and Humanities › Philosophy › Philosophy and Theoretical Science
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
- Keywords
- language models, reference, externalism, philosophy of language, meaning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.