Are Large Language Models Intentional? The Limits of Referential Grounding
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index; 5 of the 15 sources its claims cite were checked for support.
Abstract
This paper reassesses Searle’s Chinese Room argument in light of Large Language Models (LLMs). While Searle criticized computational systems for being mere “syntax manipulators,” transformer models of the kind exemplified by LLMs complicate this characterization. Earlier computational systems relied on a more rigid logical foundation, based largely on pattern recognition and mechanical substitution. By tracking patterns of use, maintaining rich contextual dependencies, and exploiting implicit regularities in large datasets, LLMs show an unprecedented capacity for semantic modelling. And yet, LLMs remain limited in several respects. For instance, even the most advanced AI model suffers from impaired referential grounding and a good model of the external world. I argue that this is because they fail to successfully reproduce the spontaneous, nonsymbolic, biologically entrenched forms of semiosis that characterize the simplest biological organizations. After examining two plausible strategies to cope with these deficits, I conclude that the algorithmic implementation of formal symbol systems diverges in fundamental ways from experience-based intentional representations. This, in turn, warrants skepticism toward bald attributions of intentionality to LLMs and substantiates the core of Searle’s original argument against strong AI.
Overview
CoChat Summary
TL;DR
The paper argues that LLMs have genuine representational capacities and referential grounding through distributional semantics, so Searle’s “mere syntax manipulator” premise needs revision. However, because intentionality requires embodiment and sensorimotor/indexical grounding, LLMs still lack intentionality, vindicating Searle’s conclusion for different reasons.
Key findings
- LLMs are not mere syntactic pattern matchers: they manipulate tokens that are semantically structured and referentially grounded, preserving distributional regularities and modeling semantic relationships. The paper notes that by 2025 LLMs perform well on medical licensing exams such as NBME and JNME, and cites reported GPT-4 parameter counts of around 1.7–1.8 trillion versus GPT-3’s 175 billion, though the company has not confirmed these data.
- Referential grounding is not the same as intentionality. Using Peirce’s semiotics, the paper argues that symbols require embedded indexical relations and experiential contents, while formal symbol systems remain self-referential; LLM tokens preserve statistical patterns but do not produce meaningful recognitions.
- The metacognitive strategy (e.g., RAG, verifier layers) only adds more symbol manipulation and does not introduce new semantic resources. The multimodal strategy is better positioned to improve grounding, but it still lacks biological features such as light-to-neural transduction, selective attention, and saccadic vision.
- The conclusion is that LLMs have representational capacities but lack intentionality; the paper denies they are “stochastic parrots” while still upholding Searle’s conclusion that they are not intentional.
Methods
- Conceptual/philosophical analysis reassessing Searle’s Chinese Room argument in light of LLMs; no empirical data collection or sample is reported.
- The paper examines early symbolic AI, transformer/LLM architecture, Peircean semiotics, the symbol/vector grounding problem, internalist vs. externalist responses, and two proposed grounding strategies: metacognitive and multimodal.
- Main analysis is argument reconstruction: premises P1–P5 lead to conclusion C, with comparisons between LLM processing and biological semiosis/human vision.
Limitations
- The paper states caveats: it will not reassess the validity of the robot reply to Searle; it uses “representation” loosely; and it adopts a stricter concept of intentionality requiring embodiment.
- The paper states there is no consensus on the definition of intentionality or its implications, which partially obscures the positions of internalist and externalist strategies.
- Evident from the text: the argument is conceptual rather than experimental, relies on cited literature, and some cited empirical claims—such as GPT-4 parameter counts—are noted as unconfirmed by the company.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 8 Not valid · 11 Not checked · 69 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Emblematic of this is the fact that by 2025, LLMs perform well in many of the medical licensing exams, like NBME (Abbas et al., 2024), JNME (Liu et al., 2025), and other professional boards.”
§ Are Large Language Models Intentional? The Limits of Referential Grounding > 4 Pattern Recognition and Rule Execution vs. Statistical Reasoning
What the sources say
- Partly supportedcites Abbas et al., 2024 matched to “Comparing the Performance of Popular Large Language Models on the National Board of Medical Examiners Sample Questions” (2024) sourceExperimental — accuracy not yet measured
The Abstract reports high scores for four LLMs on 163 NBME clinical subject exam sample questions, supporting the NBME portion of the citing sentence. The Conclusions and Discussion caution that the questions were a limited, publicly available sample without image-based items, so the results do not establish performance on actual NBME exams. The other exams have separate citations and are not attributed to Abbas et al. (LLM, conf=0.86)
- Not checked — no full text of the sourcecites Liu et al., 2025 matched to “Evaluating the Effectiveness of advanced large language models in medical Knowledge: A Comparative study using Japanese national medical examination” (2025) source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Because of the limits of current models, focusing research efforts on increasing the size of LLMs by multiplying the number of internal parameters will scarcely contribute to solving the critical problem of the lack of situated perception and causal interaction with the physical world (Lake & Murphy, 2023).”
§ Are Large Language Models Intentional? The Limits of Referential Grounding > 7 Two Strategies for a better Grounding
What the sources say
- Supportedcites Lake & Murphy, 2023 matched to “Word meaning in minds and machines.” (2023) source
Section 8.1 (pp. 43–44) compares GPT-3’s much larger parameter count with GPT-2’s and says GPT-3 is nevertheless no closer to meeting the paper’s desiderata for psychological semantics because it remains trained on text alone and ungrounded. The paper also identifies grounding in perception and action as necessary for richer word representations (Section 7.6, p. 40). This supports the claim that increasing model size alone will do little to resolve the lack of grounding. (LLM, conf=0.92)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, LLMs frequently produce inaccurate or biased output.”
§ Are Large Language Models Intentional? The Limits of Referential Grounding > 3 The Rise of Large Language Models
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is one reason why, despite improvements, transformer models are conspicuously prone to "hallucination": the production of plausible-sounding sentences that are biased, incorrect, or outright nonsense.”
§ Are Large Language Models Intentional? The Limits of Referential Grounding > 3 The Rise of Large Language Models
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Unlike it, LLMs adapt their outputs to new informational contexts.”
§ Are Large Language Models Intentional? The Limits of Referential Grounding > 4 Pattern Recognition and Rule Execution vs. Statistical Reasoning
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
39 more claims
Create a free account to see all 92 claims and every source verdict.
- Probably not a substantial claim. 34
- It may state two separate claims as one. 22
Citations
Cites
Searle, J. R. (1980) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00005756 · Cited by 6,954 · cited 5×
“Whether or not Searle’s thought experiment applies directly to them, the sheer complexity of their operations weakens the force of the assumption that machines have “only syntax but no semantics” (Searle, 1980: 422).”
InfluentialHarnad, S. (1990) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(90)90087-6 · Cited by 4,235 · cited 1×
“The question of whether a formal system can itself interpret machine tokens highlights the centrality of the “symbol grounding problem” (Harnad, 1990 ), which Har - nad defined as the problem of how to make semantic interpretations intrinsic to the symbol system, rather than merely parasitic on the…”
Gorillas in our midst: sustained inattentional blindness for dynamic events
Simons, D. J., & Chabris, C. F. (1999) · Perception · doi:10.1068/p281059 · Cited by 3,181 · cited 1×
“The biological footprint of these processes defines human perception and determines what information is available to it (Simons & Chabris, 1999).”
Grice, H. P. (1957) · The Philosophical Review · doi:10.2307/2182440 · Cited by 2,824
Putnam, H. (2011) · The Pragmatism Reader · doi:10.1515/9781400838684-021 · Cited by 1,190
Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data
Bender, E. M., & Koller, A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.463 · Cited by 1,082 · cited 1×
“By relocating the missing element from semantics to embodiment and intentionality, my account attributes LLMs greater cognitive sophistication than is allowed by the Chinese Room argument, and denies, for this very reason, that they are “stochastic parrots” (Bender & Koller, 2020).”
Putnam, H. (1973) · The Journal of Philosophy · doi:10.2307/2025079 · Cited by 464
Bisk, Y., Holtzman, A., Thomason, J., Andreas, J., Bengio, Y., Chai, J., Lapata, M., Lazaridou, A., May, J., Nisnevich, A., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.703 · Cited by 274 · cited 1×
“Combining different source types, the system simulates the way the human brain integrates sight, sound, touch, and language to produce a coherent experience (Bisk et al., 2020).”
Word meaning in minds and machines.
Lake, B. M., & Murphy, G. L. (2023) · Psychological Review · doi:10.1037/rev0000297 · Cited by 131 · cited 1×
“…the limits of current models, focusing research efforts on increasing the size of LLMs by multiplying the number of internal parameters will scarcely contribute to solving the critical problem of the lack of situated perception and causal interaction with the physical world (Lake & Murphy, 2023).”
Symbols and grounding in large language models
Pavlick, E. (2023) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2022.0041 · Cited by 95 · cited 1×
“AI researchers have widely recognized this limitation (Pavlick, 2023).”
Deacon, T. W. (2021) · Biosemiotics · doi:10.1007/s12304-021-09453-9 · Cited by 81 · cited 1×
“Such capabilities would be more plausible under a new paradigm, one focused on capturing the underlying dynamics rather than the information biological organizations store (Deacon 2021, García-Valdecasas & Deacon 2024).”
Do Language Models’ Words Refer?
Mandelkern, M., & Linzen, T. (2024) · Computational Linguistics · doi:10.1162/coli_a_00522 · Cited by 77 · cited 5×
“In the context of LLMs, the symbol grounding problem has been reframed as the “vector grounding problem” (Mollo & Millière, 2023), that is, the challenge of providing a non-circular account for how the entire set of the numerical and geometric properties of a high-dimensional vector space acquires meaning, and how the…”
InfluentialLiu, M., Okuhara, T., Dai, Z., Huang, W., Gu, L., Okada, H., Furukawa, E., & Kiuchi, T. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105673 · Cited by 73 · cited 1×
“Emblematic of this is the fact that by 2025, LLMs perform well in many of the medical licensing exams, like NBME (Abbas et al., 2024), JNME (Liu et al., 2025), and other professional boards.”
Meaning without reference in large language models
Piantadosi, S. T., & Hill, F. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.02957 · Cited by 65
The body of knowledge: On the role of the living body in grounding embodied cognition
Ziemke, T. (2016) · Biosystems · doi:10.1016/j.biosystems.2016.08.005 · Cited by 62 · cited 1×
“Harnad believed that grounding the symbols used by computers in sensorimotor interactions with the environment might resolve the problem originated by the lack of an intrinsic meaning (Ziemke, 2016).”
Abbas, A., Rehman, M. S., & Rehman, S. S. (2024) · Cureus · doi:10.7759/cureus.55991 · Cited by 58 · cited 1×
“Emblematic of this is the fact that by 2025, LLMs perform well in many of the medical licensing exams, like NBME (Abbas et al., 2024), JNME (Liu et al., 2025), and other professional boards.”
Deacon, T. W. (2011) · Biosemiotics/Biosemiotics. Bookseries · doi:10.1007/978-94-007-2336-8_2 · Cited by 57
Coelho Mollo, D., & Millière, R. (2026) · Philosophy and the Mind Sciences · doi:10.33735/phimisci.2026.12307 · Cited by 48
Mollo, D. C., & Millière, R. (2023) · arXiv · doi:10.48550/arxiv.2304.01481 · Cited by 47 · cited 3×
“In the context of LLMs, the symbol grounding problem has been reframed as the “vector grounding problem” (Mollo & Millière, 2023), that is, the challenge of providing a non-circular account for how the entire set of the numerical and geometric properties of a high-dimensional vector space acquires…”
InfluentialGrounding the Vector Space of an Octopus: Word Meaning from Raw Text
Søgaard, A. (2023) · Minds and Machines · doi:10.1007/s11023-023-09622-4 · Cited by 46 · cited 1×
“In a similar vein, Søgaard (2023) argues that LLMs can be grounded in text alone because language universally reflects world structure.”
Connectionism and neural networks
Sun, R. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9781139046855.008 · Cited by 34
Block, N. (2018) · Routledge Encyclopedia of Philosophy · doi:10.4324/9780415249126-w037-1 · Cited by 33
Boden, M. A. (2014) · The Cambridge Handbook of Artificial Intelligence · doi:10.1017/cbo9781139046855.007 · Cited by 29
Meaning and understanding in large language models
Havlík, V. (2024) · Synthese · doi:10.1007/s11229-024-04878-4 · Cited by 28 · cited 3×
“Other internalist strategies (Piantadosi & Hill, 2022 , Havlík 2024) explore different paths, such as the possibility of reinterpreting the concept of reference itself.”
InfluentialSuppression of Unattended Features Is Independent of Task Relevance
Gundlach, C., Forschack, N., & Müller, M. M. (2021) · Cerebral Cortex · doi:10.1093/cercor/bhab351 · Cited by 19 · cited 1×
“Because this process is embedded in vested interests and temporary needs, attention is inherently biased and cannot be determined by current task goals or physical salience (Gundlach et al., 2022).”
The bewitching AI: The Illusion of Communication with Large Language Models
Bottazzi Grifoni, E., & Ferrario, R. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00893-6 · Cited by 15
Origins of biological teleology: how constraints represent ends
García-Valdecasas, M., & Deacon, T. W. (2024) · Synthese · doi:10.1007/s11229-024-04705-w · Cited by 13 · cited 2×
“(Deacon, 2024:73).”
Self-citationA Categorical Analysis of Large Language Models and Why LLMs Circumvent the Symbol Grounding ProblemPreprint
Floridi, L., Jia, Y., & Tohmé, F. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5894082 · Cited by 13
I. It ain't the meat, it's the motion
Sharvy, R. (1983) · Inquiry · doi:10.1080/00201748308601987 · Cited by 11
Neural Depictions of ‘World’ and ‘Self’: Bringing Computational Understanding to the Chinese Room
Aleksander, I. (2002) · doi:10.1093/oso/9780198250579.003.0013 · Cited by 2
Preston, J. (2002) · doi:10.1093/oso/9780198250579.003.0001 · Cited by 2
A Categorical Analysis of Large Language Models and Why LLMs Circumvent the Symbol Grounding ProblemPreprint
Floridi, L., Jia, Y., & Tohmé, F. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.09117 · Cited by 2 · cited 1×
“Floridi et al. (2025) argue that when a human considers the proposition “snow is white,” there is a causal connection between the individual’s neural states and the snow itself, rather than to some descriptors of it.”
Semiotic Ground and the Hierarchic Nature of Information
Deacon, T. W. (2024) · doi:10.1002/9781119865667.ch4 · Cited by 1
Bisk, Y., Holtzman, A., Thomason, J., et al. (2020) · arXiv · doi:10.48550/arxiv.2004.10151
Research works printed but not linked: 7
- Fodor, J. A. (1975). The language of thought (Vol. 5). Harvard University Press.
- Kurzweil, R. (2000). The Age of Spiritual Machines: When Computers Exceed Human Intelligence, Penguin.
- Patel, R., & Pavlick, E. (2022). Mapping language models to grounded conceptual spaces. In Int. Conf. on Learning Representations, Online, 25–29 April 2022. https://openreview.net/forum?id=gJcEM8sxHK
- Putnam, H. (1975). The meaning of ‘meaning’. Minnesota Studies in the Philosophy of Science, 7, 131–193.
- Searle, J. (1984). Minds, Brains and Science. Harvard University Press.
- Vaswani, A. (2017). Attention is All You Need. In Guyon, I. et al. (Eds.), 31st Conference on Neural Information Processing Systems (NIPS). Advances in Neural Information Processing Systems. Vol. 30. Curran Associates, Inc.
- Wittgenstein, L. (2009). Philosophical Investigations. Wiley-Blackwell. Edited by G. E. M. Anscombe.
Cited by
Self-referential consistency in stateless language models: a behavioral perspective
Prestes, J. A. D. L. (2026) · AI & Society · doi:10.1007/s00146-026-03292-3
When AI becomes a bridge across expertise: Human–AI collaboration and the limits of agentic AI
Marozzo, F. (2026) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2026.100377
Colleague Skill: an attempt at distillation and mind uploading
Wang, Y. S. (2026) · AI and Ethics · doi:10.1007/s43681-026-01334-6
“We still lack definitive evidence that LLMs possess such things [2, 10].”
Do the critics of J. Searle take everything into account when leaving the Chinese Room?
Андреевич, К. О. (2026) · Omsk Scientific Bulletin Series Society History Modernity · doi:10.25206/2542-0488-2026-11-3-110-120
Metadata
- Authors
- García-Valdecasas, M.
- Affiliations
- Universidad de Navarra (ES)
- Corresponding author
- García-Valdecasas, Miguel
- Venue
- Philosophy & Technology
- Published
- 2026-03-25
- Volume
- 39
- Issue
- 2
- ISSN
- 2210-5433 (ISSN-L)
- 2210-5441
- OpenAlex
- W7140189339
- Cited by
- 6
- References
- 31
- Citations by year
- 2026: 6
- FWCI
- 47.01
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Philosophy & Technology (publishedVersion, cc-by)
- Funders
- Universidad de Navarra
- Topics
- Social Sciences › Arts and Humanities › Philosophy › Philosophy and Theoretical Science
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
- Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
- Keywords
- large language models, intentionality, artificial general intelligence, semiosis, semantic modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.