Talking about Large Language Models
Open accessAbstract
Interacting with a contemporary LLM-based conversational agent can create an illusion of being in the presence of a thinking creature. Yet, in their very nature, such systems are fundamentally not like us.
AI snapshot
Written from the paper’s full text.
TL;DR
This conceptual article argues that large language models should be understood primarily as next-token predictors, so anthropomorphic descriptions of them can mislead users and developers. It recommends caution with terms such as “believes” and “knows,” while allowing that belief-like descriptions may become less misleading when LLMs are embedded in richer systems with tools, multimodal input, or embodiment.
Key findings
- A bare-bones LLM models statistical distributions of tokens and generates likely continuations; it does not itself distinguish truth from falsehood or possess beliefs in the full sense.
- Anthropomorphic language can be harmless as shorthand under the intentional stance, but with powerful LLMs it risks encouraging overestimation or underestimation of capabilities.
- Embedding an LLM in a larger system that consults external sources, processes images, or controls a robot can make talk of knowledge or belief less problematic, but this depends on the system’s architecture and how language relates to the world.
- Reasoning-like behavior from LLMs, including chain-of-thought prompting, is better described as pattern completion than as guaranteed faithful logical inference.
Methods
- Conceptual and philosophical analysis rather than an empirical study; no dataset, sample, or statistical test is reported.
- The argument proceeds through illustrative examples, including factual question answering, fictional-world prompts, dialogue agents, vision-language models, and the SayCan robotic system.
- It compares bare-bones LLMs with embedded systems and examines related topics such as emergence, external information sources, embodiment, reasoning, generalization, and fine-tuning with human feedback.
Limitations
- The author states that the article does not advocate any particular account of belief or intention and takes no position on meaning in LLMs, focusing instead on how words are used.
- The author notes that a theoretical understanding of LLMs’ inner workings remains pending, despite some relevant work.
- Evident from the text: the discussion is conceptual, relies on illustrative or fictional examples, and its conclusions about belief-like capacities depend heavily on the specific architecture and embodiment of the system being considered.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
18 of 24 cited sources checked · 2 supported · 0 flagged · accuracy not yet measured
- Interpretationcitation§ What about Embodiment?
One way to build a trustworthy reasoning system using LLMs is to embed them in an algorithm that is faithful to logic because it realizes the same causal structure.
“One way to build a trustworthy reasoning system using LLMs is to embed them in an algorithm that is similarly faithful to logic because it realizes the same causal structure.8,9”
Quote in the paperNot a substantial claim2 of 2 sources checked- cites [8] 8. Lake, B.M. and Murphy, G.L. Word meaning in minds and machines. Psychological Rev. 130, 2 (2023), 401-431. source
Section 7.5 (pp. 39–40) says that explicit relations are needed for correct inferences and suggests combining neural networks with a knowledge base to encourage consistency. It does not propose embedding LLMs in a logic-faithful algorithm, or say that such an algorithm would be trustworthy because it realizes the same causal structure. (LLM, conf=0.81)
Experimental — accuracy not yet measured - cites [9] 9. Nye, M. et al. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114 (2021). source
Section 2 (Method; page unavailable) describes training a language model to write an algorithm’s intermediate steps to a scratchpad, which supports the general idea of using an algorithmic procedure to aid reasoning. The paper does not say that this makes a system trustworthy or faithful to logic by realizing the same causal structure. (LLM, conf=0.82)
Experimental — accuracy not yet measured
- Premisecitation§ What about Embodiment?
A deductive theorem prover can be trusted because its generated sequences are faithful to logic, with a computational causal structure mirroring the truth-preserving inferential structure of the problem.
“We can trust a deductive theorem prover because the sequences of sentences it generates are faithful to logic, in the sense that they are the result of an underlying computational process whose causal structure mirrors the truth-preserving inferential structure of the problem.8”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [8] 8. Lake, B.M. and Murphy, G.L. Word meaning in minds and machines. Psychological Rev. 130, 2 (2023), 401-431. source
Section 7.5 (p. 39) discusses how semantic models need explicit relations to draw correct inferences, but it does not address deductive theorem provers. Searches for “theorem prover,” “truth-preserving,” and deductive logical inference found no passage explaining why a theorem prover’s outputs can be trusted or describing the claimed mirroring of causal and inferential structure. (LLM, conf=0.76)
Experimental — accuracy not yet measured
- Interpretationreasoning§ Do LLMs Really Know Anything?
A simple LLM-based question-answering system like BOT will generate more appropriate responses if they conform to script-like dialogue patterns than to actual human conversation patterns.
“BOT will be much better at generating appropriate responses if they conform to this pattern rather than to the pattern of actual human conversation.”
Quote in the paperSubstantial claim - Interpretationreasoning§ What LLMs Do and How They Work
When an LLM answers a factual prompt, it is not answering about the world but producing a statistically likely continuation.
“In an important sense, we are not really asking who the first person was to walk on the Moon.We are asking the model the following question: Given the statistical distribution of words in the vast public corpus of (English) text, what words are most likely to follow the sequence,”
Quote in the paperSubstantial claim - Interpretationreasoning§ LLMs and the Intentional Stance
Knowing which word is likely to follow another is not the same as knowing the corresponding fact.
“knowing that the word”
Quote in the paperNot a substantial claim
- Premisereasoning§ What LLMs Do and How They Work
A bare-bones LLM's core operation is to predict likely next tokens in a sequence according to its statistical model.
“Given the statistical distribution of words in the vast public corpus of (English) text, what words are most likely to follow the sequence,”
- Also stated in: § LLMs and the Intentional Stance · plain
“all it does, at a fundamental level, is sequence prediction.”
Quote in the paperSubstantial claim - Also stated in: § LLMs and the Intentional Stance · plain
- Interpretationreasoning§ What about Embodiment?
For a VLM based on frozen joint text-image distributions, if it says an image contains a dog, there is no guarantee its words relate to the dog rather than to a spuriously correlated feature such as a kennel.
“If the user presents the VLM with a picture of a dog, and the VLM says”
- Also stated in: § What about Embodiment? · plain
“This is a picture of a dog,”
- Also stated in: § What about Embodiment? · plain
“there is no guarantee that its words relate to the dog in particular, rather than some other feature of the image that is spuriously correlated with dogs (such as the presence of a kennel).”
Quote in the paperSubstantial claim - Also stated in: § What about Embodiment? · plain
- Interpretationreasoning§ LLMs and the Intentional Stance
A bare-bones LLM does not know anything because it only performs sequence prediction.
“A bare-bones LLM does not really know anything because all it does, at a fundamental level, is sequence prediction.”
Quote in the paperNot a substantial claim - Interpretationreasoning§ What LLMs Do and How They Work
The distinctions among real-world, fictional, and rhyme prompts are invisible to the bare-bones LLM's operation.
“These distinctions are invisible at the level of what the bare-bones LLM itself-the core component of any LLM-based system-actually does, which is to generate statistically likely sequences of words.”
Quote in the paperSubstantial claim - Premisenone given§ Abstract
A sufficiently capable model can handle many tasks that would require intelligence in people by predicting the next token.
“a great many tasks that demand intelligence in humans can be reduced to next-token prediction with a sufficiently performant model.”
Quote in the paperNot a substantial claim - Interpretationreasoning§ What about Embodiment?
The only way to fully trust arguments generated by a pure LLM prompted to reason would be to reverse-engineer it and discover an emergent mechanism conforming to faithful reasoning.
“By contrast, the only way to fully trust the arguments generated by a pure LLM, one that has been coaxed into performing reasoning by prompt engineering alone, would be to reverse-”
- Also stated in: § Can Language Models Reason? · plain
“engineer it and discover an emergent mechanism that conformed to the faithful reasoning prescription.”
Quote in the paperNot a substantial claim - Also stated in: § Can Language Models Reason? · plain
- Interpretationreasoning§ What about Embodiment?
For most contemporary VLM-based systems, the relationship between a user-provided image and the words the VLM generates is mere correlation rather than causal.
“However, most contemporar y VLM-based systems do not work this way.Rather, they depend on frozen models of the joint distribution of text and images.”
- Also stated in: § What about Embodiment? · plain
“Importantly, the former relationship is mere correlation, while the latter is causal.”
Quote in the paperSubstantial claim- Rests on an assumption: The relationship between the world shared by humans and human words is causal.
- Rests on an assumption: Most contemporary VLM-based systems rely on frozen models that capture the joint distribution of text and images.
- Also stated in: § What about Embodiment? · plain
- Interpretationreasoning§ What about Embodiment?
When an LLM is prompted, the question it is really answering is what words are likely to follow the given sequence according to how words are distributed in the public corpus.
“Given the statistical distribution of words in the public corpus, what words are likely to follow the sequence S”
- Also stated in: § What about Embodiment? · plain
“Given the statistical distribution of words in the public corpus, what words are likely to follow the sequence”
Quote in the paperNot a substantial claim - Also stated in: § What about Embodiment? · plain
- Interpretationreasoning§ External Information Sources
The intentional stance becomes more compelling only with respect to the whole system, not the bare language model.
“It is only with respect to the whole system that the intentional stance becomes more compelling in such a case.”
Quote in the paperSubstantial claim - Interpretationreasoning§ Do LLMs Really Know Anything?
A basic dialog system is unable to take full part in the human truth language game, since it does not live in the shared world of human language users.
“It cannot participate fully in the human language game of truth because it does not inhabit the world we human language users share.”
Quote in the paperNot a substantial claim - Interpretationreasoning§ Do LLMs Really Know Anything?
It is misleading to say a basic dialog system really has beliefs, because that implies answerability to external reality not obtainable through textual exchanges alone.
“it would be misleading, if pressed, to say that a basic dialog system”
Quote in the paperNot a substantial claim - Interpretationreasoning§ Do LLMs Really Know Anything?
At the heart of every LLM application is a model generating statistically likely continuations of word sequences.
“at the heart of every such application is a model doing just that one thing-generating statistically likely continuations of word sequences.”
Quote in the paperSubstantial claim - Interpretationreasoning§ How Do LLMs Generalize?
RLHF and other fine-tuning do not fundamentally change the account of what LLMs do: the result is still a model of the distribution of tokens in human language, albeit slightly skewed.
“To what extent do RLHF and other forms of fine-tuning muddy our account of what LLMs really do?Well, not so much.The result is still a model of the distribution of tokens in human language, albeit one that has been slightly skewed.”
Quote in the paperNot a substantial claim - Premisecitation§ Can Language Models Reason?
The symbols an LLM manipulates are not grounded through perception, except indirectly and parasitically via the humans who produced the training data.
“None of the symbols manipulated by an LLM are grounded in the sense of Harnad, 16 through perception, except indirectly and parasitically through the humans who generated the original training data.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Harnad, 16
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationreasoning§ Can Language Models Reason?
The most likely continuation of a reasoning problem will sometimes be the correct answer, not because it is a likely individual human response but because it is a likely collective human response.
“Moreover, because the vast corpus of published human text contains numerous examples of reasoning problems accompanied by correct answers, the most likely continuation will sometimes be the correct answer.When this occurs, it is not because the correct answer is a likely individual human response but because it is a likely collective human response.”
Quote in the paperSubstantial claim- Rests on an assumption: Published human text contains many examples of reasoning problems with correct answers.
- May combine two claims. 26
- Classified as probably not a substantial claim. 22
- The quote only partly states this claim. 4
May combine two claims.
Interpretationcitation§ Do LLMs Really Know Anything?A simple LLM-based question-answering system has no communicative intent and does not know that questions come from a person or that a person receives its answers.
“a simple LLM-based question-answering system, such as BOT, has no communicative intent.3 In no meaningful sense, even under the license of the intentional stance, does it know that the questions it is asked come from a person or that a person is on the receiving end of its answers.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [3] 3. Kojima, T. et al. Large language models are zero-shot reasoners. arXiv preprint arXiv:2205.11916 (2022). source
The Background (p. 3) describes LLMs as models that estimate probabilities over text, and the Two-stage prompting section (p. 4) describes how they generate answers to questions. Neither passage addresses communicative intent or whether a system knows that people ask and receive its answers; searches for those concepts found no relevant passage. (LLM, conf=0.76)
Experimental — accuracy not yet measured
May combine two claims.
Findingcitation§ Do LLMs Really Know Anything?An LLM encodes knowledge in the way an encyclopedia does, and one emergent property is that it encodes everyday-world knowledge that no encyclopedia captures.
“it can reasonably be claimed that one emergent property of an LLM is that it encodes kinds of knowledge of the everyday world and the way it works that no encyclopedia captures.18”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [18] 18. Shanahan, M. and Mitchell, M. Abstraction for deep reinforcement learning. In Proceedings of the 31 st Intern. Joint Conf. on Artificial Intelligence (2022), 5588-5596. source
Section 3.5, “Abstraction Through Language” (page unavailable), discusses large language models as a possible route to abstraction and notes GPT-3’s few-shot generalisation. It does not say that LLMs encode everyday-world knowledge unavailable in encyclopedias. I searched our copy for the encyclopedia comparison and for LLMs’ commonsense knowledge. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Classified as probably not a substantial claim.
Premisecitation§ What LLMs Do and How They WorkLarge language models are probabilistic generators of token distributions in big public text corpora, with tokens being words, word fragments, or characters.
“LLMs are generative mathematical models of the statistical distribution of tokens in the vast public corpus of human-generated text, where the tokens in question include words, parts of words, or individual charactersincluding punctuation marks.”
Quote in the paperSubstantial claim3 of 3 sources checked- cites [3] 3. Kojima, T. et al. Large language models are zero-shot reasoners. arXiv preprint arXiv:2205.11916 (2022). source
The Background (p. 3) describes a language model as estimating a probability distribution over text and says larger web-text corpora have helped enable LLMs. The Discussion (p. 10) also says LLMs have been trained on large corpora from various web sources. Our copy does not specify that the corpora are public or define tokens as words, word fragments, characters, or punctuation marks. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites [4] 4. Li, Z., Nye, M., and Andreas, J. Implicit representations of meaning in neural language models. In Proceedings of the 59 th Annual Meeting of the Assoc. for Computational Linguistics and the 11 th Intern. Joint Conf. on Natural Language Processing 1, Long Papers (2021). source
The Introduction (p. 1) says neural language models place probability distributions over sequences of words, and §4.1 (p. 5) names text corpora used to pretrain the models. The paper does not specify that tokens include word fragments, individual characters, or punctuation, or characterize the training material as one vast public corpus of human-generated text. (LLM, conf=0.85)
Experimental — accuracy not yet measured - cites [20] 20. Stiennon, N. et al. Learning to summarize from human feedback. In Advances in Neural Information Processing Systems (2020), 3008-3021. source
Section 3.4 (p. 5) says the models predict the next token in a large text corpus, and Appendix B.1 (p. 17) identifies the corpus sources and says the models use byte-pair encoding. The paper does not establish that the entire corpus is public or explicitly describe tokens as words, word fragments, individual characters, and punctuation marks. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S2 (checkable).
Premisecitation§ Vision-Language ModelsFor a system to have beliefs, it must genuinely engage with the world as its representations depict it.
“it must”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [33] 33. Chan, S.C. et al. Data distributional properties drive emergent in-context learning in transformers. In Advances in Neural Information Processing Systems (2022). source
The Discussion (p. 9) addresses in-context learning in transformers, not whether a system must engage with the world to have beliefs. Searches for beliefs, representations, and engagement with the world found no passage addressing that requirement. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Interpretationcitation§ What LLMs Do and How They WorkWhen an LLM improves reasoning when told to 'think step by step', the temptation to see it as human-like is almost overwhelming.
“When an LLM can be made to improve its performance on reasoning tasks simply by being told to”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [17] 17. Schick, T. et al. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761 (2023). source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisecitation§ Vision-Language ModelsVision-language models combine a language model with an image encoder and are trained on text-image pairs, enabling prediction of word continuations in the context of an image.
“vision-language models (VLMs) such as VilBERT 19 and Flamingo 2 combine a language model with an image encoder and are trained on a multi-modal corpus of text-image pairs.This enables them to predict how a given sequence of words will continue in the context of a given image.”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [19] 19. Smith, B.C. The Promise of Artificial Intelligence: Reckoning and Judgment. MIT Press (2019). source
The Introduction (p. 2) and Section 2.2 (p. 3) describe ViLBERT’s visual and language streams and training on paired images and text. But Section 2.2 describes reconstructing masked words, not predicting word continuations; Related Work (p. 8) says decoding output text from this bidirectional model remains unclear. The continuation-prediction part is therefore not supported for ViLBERT. (LLM, conf=0.90)
Experimental — accuracy not yet measured - cites [2] 2. Harnad, S. The symbol grounding problem. Physica D: Nonlinear Phenomena 42, 1-3 (1990), 335-346. source
May combine two claims.
Premisecitation§ What about Embodiment?Chain-of-thought prompting submits a prompt prefix containing examples of multi-step reasoning before the user's query, which encourages the model to show its workings and improves reasoning performance.
“In chain-of-thought prompting, for example, a prompt prefix is submitted to the model, before the user's query, containing a few examples of multi-step reasoning, with all the intermediate steps explicitly spelled out.23,38 Doing this encourages the model to”
- Also stated in: § What about Embodiment? · plain
“show its workings,”
- Also stated in: § What about Embodiment? · plain
“which improves reasoning performance.”
Quote in the paperNot a substantial claim1 of 2 sources checked- Also stated in: § What about Embodiment? · plain
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationcitation§ Conclusion: Why This MattersToday's LLMs and applications are so powerful and convincingly intelligent that the intentional stance's license to use terms like belief and knowledge can no longer safely be applied without risk of misleading about their character and capabilities.
“However, today's LLMs, and the applications that use them, are so powerful, so convincingly intelligent, that such license can no longer safely be applied.30,39”
Quote in the paperNot a substantial claim2 of 2 sources checked- cites [30] 30. Bender, E. and Koller, A. Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proceedings of the 58 th Annual Meeting of the Assoc. for Computational Linguistics (2020), 5185-5198. source
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured - cites [39] 39. Devlin, J., Chang, M-W., Lee, K., and Toutanova, K. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018). source
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
May combine two claims. The quote only partly states this claim.
Interpretationcitation§ How Do LLMs Generalize?An LLM with enough parameters trained on a sufficiently large dataset with the right statistical properties can acquire a pattern-completion mechanism with a degree of generality, which is an emergent capability that can solve reasoning problems in a chain-of-thought prompt, though without guarantee of faithfulness to logic.
“This is a powerful, emergent capability with many useful modes of application, one of which is to solve reasoning problems in the context of a chain-of-thought prompt (although there is no guarantee of faithfulness to logic here, no guarantee that, in the case of deductive reasoning, pattern completion will be truth-preserving).8,9”
Quote in the paperSubstantial claim2 of 2 sources checked- cites [8] 8. Lake, B.M. and Murphy, G.L. Word meaning in minds and machines. Psychological Rev. 130, 2 (2023), 401-431. source
Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured - cites [9] 9. Nye, M. et al. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114 (2021). source
Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
May combine two claims.
Premisecitation§ How Do LLMs Generalize?Fine-tuning a model from human feedback at scale using preference data is a potent technique that can shape responses to reflect user norms, filter toxic language, improve factual accuracy, and mitigate fabrication.
“Fine-tuning a model from human feedback at scale, using preference data from paid raters or drawn from a large and willing user base, is an especially potent technique.It has the potential not only to shape a model's responses to better reflect user norms (for better or worse), but also to filter out toxic language, improve factual accuracy, and mitigate the tendency to fabricate information.”
Quote in the paperNot a substantial claim2 of 3 sources checked- cites [14] 14. Radford, A. et al. Language models are unsupervised multitask learners. (2019). source
Sections 2.3 (p. 7) and 2.7 (p. 11) describe collecting human response preferences and fine-tuning on preferred responses. The paper also reports reduced rule violations (p. 3), evidence of improved correctness (section 3.3, p. 18), and an aim to reduce hallucinations (section 2.4, p. 9). It supports the technique’s potential, but does not establish the alternative of drawing preference data from a large, willing user base; its evaluations use paid annotators (section 3.1, p. 14). (LLM, conf=0.84)
Experimental — accuracy not yet measured - cites [26] 26. Wittgenstein, L. Philosophical Investigations. Basil Blackwell (1953). source
- cites [34] 34. Chowdhery, S. et al. PaLM: Scaling language modeling with pathways. arXiv preprint arxiv:2204.02311 (2022). source
The Introduction (p. 5) and Section 10.2 (p. 43) assess toxicity in PaLM’s generated text, but do not assess fine-tuning with human preference data. Searches for “human feedback,” “preference data,” and factual accuracy or fabrication found no passage connecting that technique to the claimed benefits; “human feedback” appears in a reference title (p. 58), not a discussion of those effects. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Premisenone given§ What about Emergence?Talk of beliefs belongs most naturally to people, and the expectations tied to it rest on shared understanding inherited through evolution.
“Humans are the natural home of talk of beliefs and the like, and the behavioral expectations that go hand in hand with such talk are grounded in our mutual understanding, which is itself the product of a common evolutionary heritage.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationreasoning§ LLMs and the Intentional StanceA bare-bones LLM has no notion of truth or falsehood and cannot be sensitive to the truth of the sequences it predicts.
“The model itself has no notion of truth or falsehood because it lacks the means to exercise these concepts in anything like the way we do.”
- Also stated in: § Do LLMs Really Know Anything? · plain
“The internal mechanisms it uses to do this, whatever they are, cannot in themselves be sensitive to the truth or otherwise of the word sequences it predicts.”
Quote in the paperNot a substantial claim- Also stated in: § Do LLMs Really Know Anything? · plain
Probably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationreasoning§ Do LLMs Really Know Anything?The basic function of an LLM-generating statistically likely continuations-is extraordinarily versatile.
“the basic function of an LLM, namely to generate statistically likely continuations of word sequences, is extraordinarily versatile.”
Quote in the paperNot a substantial claimMay combine two claims.
Premisereasoning§ Do LLMs Really Know Anything?A bare-bones LLM is not by itself a conversational agent; it must be embedded in a larger system to manage turn-taking and be prompted to produce conversation-like behavior.
“a bare-bones LLM is, by itself, not a conversational agent.d For a start, the LLM must be embedded in a larger system to manage the turn-taking in the dialog.But it will also need to be coaxed into producing conversation-like behavior.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Premisereasoning§ External Information SourcesA belief about the shared world requires the capacity to revise it when evidence from that world warrants.
“Nothing can count as a belief about the world we share-in the largest sense of the term-unless it is against the backdrop of the ability to update beliefs appropriately in light of the evidence from that world, an essential aspect of the capacity to distinguish truth from falsehood.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationreasoning§ What about Embodiment?An isolated LLM cannot triangulate on objective reality, but when embedded in a larger system that interacts with an external world, it becomes more human-like than a disembodied language model.
“In isolation, an LLM is not the sort of thing that can do this, but in application, LLMs are embedded in larger systems.”
- Also stated in: § What about Embodiment? · plain
“When such a system inhabits a world like our own-a world populated with 3D objects, some of which are other agents, some of whom are language-users-it is, in this important respect, a lot more human-like than a disembodied language model.”
Quote in the paperSubstantial claim- Also stated in: § What about Embodiment? · plain
May combine two claims. The quote only partly states this claim.
Interpretationreasoning§ LLMs and the Intentional StanceA simple LLM-based question-answering system such as BOT does not have beliefs or knowledge.
“a simple LLM-based question-answering system like BOT does not, properly speaking, have beliefs.”
- Also stated in: § LLMs and the Intentional Stance · plain
“BOT does not really know that Burundi is south of Rwanda”
Quote in the paperSubstantial claim- Also stated in: § LLMs and the Intentional Stance · plain
Probably not a substantial claim: may not pass S2 (checkable), S4 (based).
Premisenone given§ Do LLMs Really Know Anything?Belief in its fullest sense can be ascribed only when a system can distinguish truth from falsehood.
“Only in the context of a capacity to distinguish truth from falsehood can we legitimately speak of belief in its fullest sense.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationreasoning§ Do LLMs Really Know Anything?A bare-bones LLM has no access to external reality against which its words might be measured, nor means to apply external criteria of truth.
“the LLM itself has no access to any external reality against which its words might be measured, nor the means to apply any other external criteria of truth, such as agreement with other language users.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable).
Premisecitation§ What LLMs Do and How They WorkAnthropomorphic language about artifacts is harmless when understood as shorthand via the intentional stance.
“These examples of what Dennett calls the intentional stance are harmless and useful forms of shorthand for complex processes whose details we don't know or care about.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [11] 11. OpenAI. GPT-4 technical report. arXiv preprint arXiv:2303.08774 (2023). source
Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.
Interpretationreasoning§ What about Emergence?A dialog system's access to external information sources can confer a form of access to external reality, making the word 'belief' less misleading because it could seek evidence and change its mind.
“the ability to consult external information sources does indeed confer on a dialog system a form of access to an external reality against which its words can be measured.Used in this context, the word”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationreasoning§ What about Embodiment?In chain-of-thought prompting, the token sequences most likely to follow the prompt prefix resemble the prefix and include multiple reasoning steps, so the model generates them.
“The sequences of tokens most likely to follow S will have a similar form to sequences found in the prompt prefix, which is to say they will include multiple steps of reasoning, so these are what the model generates.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Premisenone given§ AbstractAI systems operate in ways fundamentally different from humans.
“they work in ways that are fundamentally different than the way humans work.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationreasoning§ External Information SourcesThe shift from the language model itself to the larger system is crucial; the language model remains a sequence predictor with no more access to the external world.
“this line of thinking depends on the shift from the language model itself to the larger system of which the language model is a part.The language model itself is still just a sequence predictor and has no more access to the external world than it ever did.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationreasoning§ What about Embodiment?Non-trivial reasoning problems require multiple inference steps, and LLMs can be applied to multi-step reasoning without further training through prompt engineering.
“But non-trivial reasoning problems require multiple inference steps.LLMs can be effectively applied to multi-step reasoning, without further training, thanks to clever prompt engineering.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
Interpretationnone given§ What about Embodiment?Because logic is content neutral, one cannot criticize talk of reasoning in LLMs on the grounds that they lack access to an external reality against which truth or falsehood can be measured.
“The content neutrality of logic means that we cannot criticize talk of reasoning in LLMs on the grounds that they have no access to an external reality against which truth or falsehood can be measured.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone given§ What about Embodiment?A suitably prompted LLM often produces multi-step arguments that are valid and final answers that are correct, but when it appears to reason correctly it does so by mimicking well-formed arguments in its training set and/or prompt.
“It is remarkable that, not only do the model's responses take the form of an argument with multiple steps, but also the argument in question is often (but not always) valid, and the final answer is often (but not always) correct.”
- Also stated in: § What about Embodiment? · hedged
“But to the extent that a suitably prompted LLM appears to reason correctly, it does so by mimicking well-formed arguments in its training set and/or in the prompt.”
Quote in the paperNot a substantial claim- Also stated in: § What about Embodiment? · hedged
May combine two claims.
Interpretationreasoning§ Do LLMs Really Know Anything?The idea of a whole dialog system having beliefs is not a category error, but a simple dialog agent like BOT still lacks beliefs.
“At least the very idea of the whole system having beliefs makes sense.There is no category error here.However, for a simple dialog agent like BOT, the answer is surely still”
Quote in the paperSubstantial claimMay combine two claims. The quote only partly states this claim.
Interpretationreasoning§ Do LLMs Really Know Anything?Neither an encyclopedia nor a bare-bones LLM literally knows or believes anything in the way a human does.
“An encyclopedia does not literally”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationreasoning§ What about Embodiment?Humans are fundamentally different from LLMs because humans belong to a shared-world community of language users.
“Humans are members of a community of language users inhabiting a shared world, and this primal fact makes them essentially different than LLMs.”
- Also stated in: § What about Embodiment? · plain
“Human language users can consult the world to settle their disagreements and update their beliefs.”
Quote in the paperNot a substantial claim- Rests on an assumption: Human language users can consult the world to settle disagreements and update beliefs.
- Also stated in: § What about Embodiment? · plain
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning§ What about Embodiment?Whether it is appropriate to attribute communicative intent, knowledge, or belief to an embodied LLM system depends on how the system works, what model it uses, and how the model is embedded in the architecture.
“But whether it is appropriate to speak of communicative intent in the context of such a system, or of knowledge and belief in their fullest sense, depends”
- Also stated in: § What about Embodiment? · plain
“Whether these concerns apply to any specific VLM-based system depends on exactly how that system works, what sort of model it uses, and how that model is embedded in the system's overall architecture.”
Quote in the paperNot a substantial claim- Also stated in: § What about Embodiment? · plain
Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Interpretationreasoning§ Conclusion: Why This MattersContemporary LLM-based conversational agents can create a compelling illusion of a thinking creature, but they are fundamentally not like humans and can be inscrutable, presenting a patchwork of less-than-human and superhuman capacities.
“Interacting with a contemporary LLM-based conversational agent can create a compelling illusion of being in the presence of a thinking creature like us.Yet, in their very nature, such systems are fundamentally not like us.The shared”
- Also stated in: § Conclusion: Why This Matters · plain
“form of life”
- Also stated in: § Conclusion: Why This Matters · plain
“that underlies mutual understanding and trust among humans is absent, and these systems can be inscrutable as a result, presenting a patchwork of less-than-human with superhuman capacities, of uncannily human-like with peculiarly inhuman behavior.”
Quote in the paperNot a substantial claim- Rests on an assumption: The form of life shared by humans, which supports mutual understanding and trust, is missing in LLM-based systems.
- Also stated in: § Conclusion: Why This Matters · plain
May combine two claims.
Interpretationcitation§ What about Embodiment?In SayCan, the LLM's natural-language descriptions of low-level actions are grounded by serving as intermediaries between perception and action, but language learning and use in SayCan is very different from human language because the models are pre-trained on text-only data in a disembodied setting and have not learned language through interaction in a shared world.
“Nevertheless, despite being physically embodied and interacting with the real world, the way language is learned and used in a system such as SayCan is very different from the way it is learned and used by a human.”
- Also stated in: § What about Embodiment? · plain
“Language models incorporated in systems such as SayCan are pre-trained to perform sequence prediction in a disembodied setting from a text-only dataset.They have not learned language by talking to other language users while immersed in a shared world and engaged in joint activity.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Ahn et al. 1
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Also stated in: § What about Embodiment? · plain
Classified as probably not a substantial claim. May combine two claims.
Interpretationreasoning§ What about Embodiment?In embodied systems today, the role of language is very limited: users issue natural-language instructions and systems generate interpretable descriptions of actions, which hardly compares to the range of collective activity language supports in humans.
“But in such systems today, the role of language is very limited.The user issues instructions to the system in natural language, and the system generates interpretable natural-language descriptions of its actions.But this tiny repertoire of language use hardly bears comparison to the cornucopia of collective activity that language supports in humans.”
Quote in the paperSubstantial claimMay combine two claims. The quote only partly states this claim.
Interpretationreasoning§ Can Language Models Reason?Because of the prevalence of a common meta-pattern in the published human language corpus, token-level pattern completion often yields the most likely continuation in few-shot prompting, and chain-of-thought prompts transform reasoning problems into next-token prediction problems solvable by pattern completion.
“This is an example of a common meta-pattern in the published humanlanguage corpus: a series of sequences of tokens, wherein each sequence conforms to the same pattern.”
- Also stated in: § Can Language Models Reason? · plain
“Similarly, in the context of a suitable chain-of-thoughtstyle prompt, reasoning problems are transformed into next-token prediction problems, which can be solved by pattern completion.”
Quote in the paperSubstantial claim- Rests on an assumption: A common meta-pattern exists in the published human language corpus: token sequences that each conform to one pattern.
- Also stated in: § Can Language Models Reason? · plain
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning§ Conclusion: Why This MattersCareless use of philosophically loaded words such as
“The careless use of philosophically loaded words such as”
- Also stated in: § External Information Sources · plain
“believes”
- Also stated in: § Conclusion: Why This Matters · plain
“thinks”
Quote in the paperNot a substantial claim- Also stated in: § External Information Sources · plain
May combine two claims.
Interpretationreasoning§ How Do LLMs Generalize?Next-token prediction is sufficient for solving previously unseen reasoning problems, even if unreliably, because an LLM models a complex distribution and samples from it rather than cutting and pasting text fragments.
“The noteworthy thing is that next-token prediction is sufficient for solving previously unseen reasoning problems, even if unreliably.”
- Also stated in: § How Do LLMs Generalize? · plain
“Rather, an LLM models a distribution that is unimaginably complex and allows users and applications to sample from that distribution.”
Quote in the paperNot a substantial claim- Also stated in: § How Do LLMs Generalize? · plain
Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Interpretationreasoning§ How Do LLMs Generalize?A base LLM can be thought of as equivalent to a model trained from scratch with RLHF, in the limit of astronomical raters and training time; this thought experiment shows that what counts most for an LLM's functionality is the nature of the final product rather than the process by which it is produced.
“Conversely, in the limit, we can think of a conventionally trained base LLM as equivalent to a model trained completely from scratch with RLHF.”
- Also stated in: § How Do LLMs Generalize? · plain
“But the thought experiment illustrates that what counts most when we think about the functionality of an LLM is not so much the process by which it is produced (although this is important) but the nature of the final product.”
Quote in the paperNot a substantial claim- Also stated in: § How Do LLMs Generalize? · plain
May combine two claims.
Interpretationreasoning§ Can Language Models Reason?Few-shot prompting does not teach an LLM how to reason; rather, the LLM performs pattern completion, completing a partial sequence that conforms to the pattern of the prompt.
“It is tempting to say that the few-shot prompt teaches the LLM how to reason, but this would be a misleading characterization.What the LLM does is more accurately described in terms of pattern completion.”
- Also stated in: § Can Language Models Reason? · hedged
“The most likely continuation of this partial sequence in the context of the few-shot prompt is a sequence that completes the pattern.”
Quote in the paperSubstantial claim- Also stated in: § Can Language Models Reason? · hedged
Probably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationnone given§ Can Language Models Reason?The answer to whether LLM-based systems really have beliefs is usually no.
“While the answer to the question”
- Also stated in: § Can Language Models Reason? · plain
“Do LLM-based systems really have beliefs?”
- Also stated in: § Can Language Models Reason? · hedged
“is usually”
Quote in the paperNot a substantial claim- Also stated in: § Can Language Models Reason? · plain
The paper relies on this without support.
Anthropomorphic descriptions of LLM-based dialog agents can cause users and developers to misjudge their capabilities.
“As LLMs become more powerful, it becomes increasingly tempting to describe LLM-based dialog agents in human-like terms, which can lead users to overestimate (or underestimate) their capabilities.”
“One danger of anthropomorphism is that it can mislead users and developers alike to expect an AI system to exhibit human-level performance on tasks where it in fact cannot match humans, while exhibiting merely research human-level competence on other tasks where it can in fact outperform humans.”
The paper relies on this without support.
Human language users can consult the world to settle disagreements and update beliefs.
“Human language users can consult the world to settle their disagreements and update their beliefs.”
The paper relies on this without support.
The relationship between the world shared by humans and human words is causal.
“Importantly, the former relationship is mere correlation, while the latter is causal.”
The paper relies on this without support.
Most contemporary VLM-based systems rely on frozen models that capture the joint distribution of text and images.
“However, most contemporar y VLM-based systems do not work this way.Rather, they depend on frozen models of the joint distribution of text and images.”
The paper relies on this without support.
Published human text contains many examples of reasoning problems with correct answers.
“because the vast corpus of published human text contains numerous examples of reasoning problems accompanied by correct answers”
The paper relies on this without support.
A common meta-pattern exists in the published human language corpus: token sequences that each conform to one pattern.
“This is an example of a common meta-pattern in the published humanlanguage corpus: a series of sequences of tokens, wherein each sequence conforms to the same pattern.”
The paper relies on this without support.
The form of life shared by humans, which supports mutual understanding and trust, is missing in LLM-based systems.
“The shared”
“form of life”
“that underlies mutual understanding and trust among humans is absent”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLMs exhibit an inherent brittleness and unreliability in knowledge tasks, and struggle with real-world understanding and human-like reasoning tasks.
“While a growing number of studies have shown that LLMs exhibit an inherent brittleness (13) and unreliability (14, 15) in knowledge tasks, and that they struggle with real-world understanding (11, 16) and human-like reasoning tasks (9, 10, 14), others have shown that LLMs instead excel in mimicking human language and communicative abilities (17)”
From The benefits and dangers of anthropomorphic conversational agents (2025) · cites this paper as [16]
For the portions citing this paper, “How Do LLMs Generalize?” says LLMs can solve reasoning problems, but somewhat unreliably; “Can Language Models Reason?” likewise notes occasional mistakes and cautions against characterizing their output as reasoning in the human sense. “Do LLMs Really Know Anything?” questions whether a basic question-answering system can assess truth against external reality, but the passages do not establish unreliability in knowledge tasks as a measured finding. The other parts of the citing sentence have different citations. (LLM, conf=0.78)
Experimental — accuracy not yet measuredAnthropomorphic abilities bring with them the potential for deception and manipulation at scale and for mistaken beliefs by users that these machines understand human experience and existence in ways they cannot.
“This brings with it the potential for deception and manipulation at scale and for mistaken beliefs by users that these machines understand human experience and existence in ways they cannot (16, 33).”
From The benefits and dangers of anthropomorphic conversational agents (2025) · cites this paper as [16]
The Abstract says anthropomorphism can mislead users into attributing capabilities to AI systems that they lack. The Conclusion: Why This Matters says conversational agents can create an illusion of a thinking creature like us, although they lack the shared human experience underlying mutual understanding. Our copy does not establish the separate claim about deception and manipulation at scale. (LLM, conf=0.86)
Experimental — accuracy not yet measuredIn media discourse, AI chatbots are frequently described using psychological language and conversational norms in human-AI interaction invite treating chatbots as social agents.
“Since in media discourse, AI chatbots are frequently described using psychological language and conversational norms in human-AI interaction invite treating chatbots as social agents (Shanahan, 2024), in surveys, this may make consciousness attributions to chatbots seem like the expected or 'correct' response.”
From Are Attributions of Consciousness to AI Chatbots Epistemically Innocent? (2026) · cites this paper as Shanahan, 2024
The Conclusion: Why This Matters says that words such as “believes” and “thinks” encourage anthropomorphism, including when speaking to the media, and that interacting with a conversational agent can create the illusion of a thinking creature. It does not establish that media descriptions frequently use such language or specifically attribute the effect to conversational norms. The available passage has no reliable page number. (LLM, conf=0.82)
Experimental — accuracy not yet measuredA generative system supplies the signals of credibility while bypassing the process that those signals are meant to indicate.
“A generative system thus supplies the signals of credibility while bypassing the process that, in the classroom and in inquiry generally, those signals are meant to indicate [2,45].”
From Hallucination as epistemic pollution: epistemic injustice and credibility in futures-oriented science education (2026) · cites this paper as [45]
Calls cautioning against anthropomorphizing LLMs and AI systems are not new.
“Calls that caution against researchers, the media or the general public anthropomorphizing LLMs and AI systems are not new (16, 28).”
From The benefits and dangers of anthropomorphic conversational agents (2025) · cites this paper as [16]
Language models are mostly built upon deep neural networks, with the majority based on transformers or close variations, and are pre-trained at scale with massive unlabeled corpus then fine-tuned with task-oriented data.
“Nowadays, language models are mostly built upon deep neural networks, where two features need to be emphasized. First of all, the majority of language models are based on transformers or their close variations [28, 29].”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [30]
The dialogical behavior exhibited by LLMs can generate in us the experience of being in the presence of a human-like interlocutor.
“Shanahan (2024) also emphasises that the dialogical behavior exhibited by LLMs can generate in us the experience of being in the presence of a human-like interlocutor.”
From A phenomenology and epistemology of large language models: transparency, trust, and trustworthiness (2024) · cites this paper as Shanahan, 2024
Anthropomorphizing technical systems has been criticized for setting the wrong expectations and causing over-reliance in technology.
“This phenomenon - anthropomorphizing 1 technical systems - has been criticized for setting the wrong expectations and causing over-reliance in technology [21, 45, 46].”
From From "AI" to Probabilistic Automation: How Does Anthropomorphization of Technical Systems Descriptions Influence Trust? (2024) · cites this paper as [45]
Numerous academic papers have appeared which warn about the risks of overusing anthropomorphic language to describe LLMs and generative probabilistic automation technology.
“With the blight of publicly-available Large Language Models (LLMs) and generative probabilistic automation technology, numerous academic papers have appeared which warn about the risks of overusing anthropomorphic language to describe such technology [1, 12, 21, 43, 45, 46].”
From From "AI" to Probabilistic Automation: How Does Anthropomorphization of Technical Systems Descriptions Influence Trust? (2024) · cites this paper as [45]
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 16,037 · see citing works
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881 · see citing works
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,695 · see citing works
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · see citing works
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505 · see citing works
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · see citing works
Harnad, S. (1990) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(90)90087-6 · Cited by 4,235 · cited 1× · see citing works
“…follows from the premises here irrespective of the meaning (if any) 12 None of the symbols manipulated by an LLM are grounded in the sense of Harnad (1990), that is to say through perception, except indirectly and parasitically through the humans who generated the original training data. of…”
BackgroundTraining language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · see citing works
Training Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117 · see citing works
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · see citing works
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109 · see citing works
Large Language Models are Zero-Shot Reasoners
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · arXiv · doi:10.48550/arxiv.2205.11916 · Cited by 2,013 · see citing works
The Unreasonable Effectiveness of Data
Halevy, A. Y., Norvig, P., & Pereira, F. M. B. (2009) · IEEE Intelligent Systems · doi:10.1109/mis.2009.36 · Cited by 1,776 · cited 2× · see citing works
“new meaning to the phrase “unreasonable effectiveness of data” (Halevy et al., 2009).”
BackgroundFlamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555 · cited 2× · see citing works
“, 2019) and Flamingo (Alayrac et al., 2022), for example, combine a language model with an image encoder, and are trained on a multi-modal corpus of text-image pairs.”
BackgroundMethodEmergent Abilities of Large Language Models
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.07682 · Cited by 1,264 · see citing works
Toolformer: Language Models Can Teach Themselves to Use Tools
Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023) · arXiv · doi:10.48550/arxiv.2302.04761 · Cited by 1,150 · see citing works
Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data
Bender, E. M., & Koller, A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.463 · Cited by 1,082 · cited 4× · see citing works
“A large language model is a very different sort of animal (Bender and Koller, 2020; Bender et al., 2021; Marcus and Davis, 2020).”
BackgroundResultInfluentialFlamingo: a visual language model for few-shot learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1723 · Cited by 966 · see citing works
LaMDA: Language Models for Dialog Applications
Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., Du, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.08239 · Cited by 778 · see citing works
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., Finn, C., Fu, C., Gopalakrishnan, K., Hausman, K., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.01691 · Cited by 624 · see citing works
ReAct: Synergizing Reasoning and Acting in Language Models
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022) · arXiv · doi:10.48550/arxiv.2210.03629 · Cited by 608 · see citing works
Learning to summarize from human feedback
Stiennon, N., Ouyang, L., Wu, J., Ziegler, D. M., Lowe, R., Voss, C., Radford, A., Amodei, D., & Christiano, P. (2020) · arXiv · doi:10.48550/arxiv.2009.01325 · Cited by 550 · see citing works
Davidson, D. (1982) · Dialectica · doi:10.1111/j.1746-8361.1982.tb01546.x · Cited by 531 · cited 1× · see citing works
“8 Davidson uses a similar argument to call into question whether belief is possible without language (Davidson, 1982).”
BackgroundEthical and social risks of harm from Language Models
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04359 · Cited by 477 · see citing works
The Promise of Artificial Intelligence: Reckoning and Judgment
Smith, B. C. (2019) · doi:10.7551/mitpress/12385.001.0001 · Cited by 271 · cited 1× · see citing works
“In the words of B.C.Smith, it must “authentically engage with the world’s being the way in which [its] representations represent it as being” (Smith, 2019).”
BackgroundScaling Language Models: Methods, Analysis & Insights from Training Gopher
Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.11446 · Cited by 235 · see citing works
Locating and Editing Factual Associations in GPT
Meng, K., Bau, D., Andonian, A., & Belinkov, Y. (2022) · arXiv · doi:10.48550/arxiv.2202.05262 · Cited by 230 · see citing works
Dennett, D. (2009) · The Oxford Handbook of Philosophy of Mind · doi:10.1093/oxfordhb/9780199262618.003.0020 · Cited by 182 · see citing works
Locating and Editing Factual Associations in GPT
Meng, K., Bau, D., Andonian, A., & Belinkov, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1262 · Cited by 149 · see citing works
Show Your Work: Scratchpads for Intermediate Computation with Language Models
Nye, M., Andreassen, A. J., Gur-Ari, G., Michalewski, H., Austin, J., Bieber, D., Dohan, D., Lewkowycz, A., Bosma, M., Luan, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.00114 · Cited by 144 · see citing works
Improving alignment of dialogue agents via targeted human judgements
Glaese, A., McAleese, N., Trębacz, M., Aslanides, J., Firoiu, V., Ewalds, T., Rauh, M., Weidinger, L., Chadwick, M., Thacker, P., et al. (2022) · arXiv · doi:10.48550/arxiv.2209.14375 · Cited by 134 · see citing works
Word meaning in minds and machines.
Lake, B. M., & Murphy, G. L. (2023) · Psychological Review · doi:10.1037/rev0000297 · Cited by 131 · cited 1× · see citing works
“BOT: This yields the full prompt to be submitted to the LLM, which will hopefully predict a con-4”
BackgroundSelection-Inference: Exploiting Large Language Models for Interpretable Logical ReasoningPreprint
Creswell, A., Shanahan, M., & Higgins, I. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.09712 · Cited by 122 · see citing works
Self-citationIn-context Learning and Induction Heads
Olsson, C., Elhage, N., Nanda, N., Joseph, N., DasSarma, N., Henighan, T., Mann, B., Askell, A., Bai, Y., Chen, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2209.11895 · Cited by 104 · see citing works
Implicit Representations of Meaning in Neural Language Models
Li, B. Z., Nye, M., & Andreas, J. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.143 · Cited by 84 · cited 2× · see citing works
“In chain-of-thought prompting, for example, a prompt prefix is submitted to the model, before the user’s query, containing a few examples of multi-step reasoning, with all the intermediate steps explicitly spelled out (Nye et al., 2021; Wei et al., 2022b).”
BackgroundMeaning without reference in large language modelsPreprint
Piantadosi, S. T., & Hill, F. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.02957 · Cited by 72 · see citing works
Data Distributional Properties Drive Emergent In-Context Learning in Transformers
Chan, S. C. Y., Santoro, A., Lampinen, A. K., Wang, J. X., Singh, A., Richemond, P. H., McClelland, J., & Hill, F. (2022) · arXiv · doi:10.48550/arxiv.2205.05055 · Cited by 69 · see citing works
Data Distributional Properties Drive Emergent In-Context Learning in Transformers
Chan, S., Santoro, A., Lampinen, A., Wang, J., Singh, A., Richemond, P., McClelland, J., & Hill, F. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1371 · Cited by 69 · see citing works
Faithful Reasoning Using Large Language Models
Creswell, A., & Shanahan, M. (2022) · arXiv · doi:10.48550/arxiv.2208.14271 · Cited by 37 · see citing works
Self-citationAbstraction for Deep Reinforcement Learning
Shanahan, M., & Mitchell, M. (2022) · International Joint Conference on Artificial Intelligence · doi:10.48550/arxiv.2202.05839 · Cited by 32
Self-citationAbstraction for Deep Reinforcement Learning
Shanahan, M., & Mitchell, M. (2022) · Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/780 · Cited by 17 · cited 2× · see citing works
“Plausibly, an LLM with enough parameters trained on a sufficiently large dataset with the right statistical properties can acquire a pattern completion mechanism with a degree of generality (Shanahan and Mitchell, 2022).”
BackgroundSelf-citationOpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
Research works printed but not linked: 2
- Marcus, G. and Davis, E. GPT-3, bloviator: OpenAI's language generator has no idea what it's talking about. MIT Technology Rev. (Aug. 2020).
- Ruane, E., Birhane, A. and Ventresque, A. Conversational AI: Social and ethical considerations. In Proceedings of the 27th AIAI Irish Conf. on Artificial Intelligence and Cognitive Science (2019), 104--115.
Cited by
Sorted by citations
Detecting hallucinations in large language models using semantic entropy
Farquhar, S., Kossen, J., Kuhn, L., & Gal, Y. (2024) · Nature · doi:10.1038/s41586-024-07421-0 · Cited by 934 · see citing works
“Although we agree that metaphor must be used carefully with LLMs 22 , the widespread adoption of the term hallucination reflects the fact that it points to an important phenomenon.”
Role play with large language models
Shanahan, M., McDonell, K., & Reynolds, L. (2023) · Nature · doi:10.1038/s41586-023-06647-8 · Cited by 491 · see citing works
Self-citationWhen large language models meet personalization: perspectives of challenges and opportunities
Chen, J., Liu, Z., Huang, X., Wu, C., Liu, Q., Jiang, G., Pu, Y., Lei, Y., Chen, X., Wang, X., et al. (2024) · World Wide Web · doi:10.1007/s11280-024-01276-1 · Cited by 367 · see citing works
“Secondly, the language models are pre-trained at scale with a massive amount of unlabeled corpus [3, 30–32].”
Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods
Cao, Y., Zhao, H., Cheng, Y., Shu, T., Chen, Y., Liu, G., Liang, G., Zhao, J., Yan, J., & Li, Y. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3497992 · Cited by 174 · see citing works
“LLMs typically refer to the Transformer-based language models [12] containing billions of parameters and being trained on massive text data (i.e., several terabytes (TBs) scale) [64], such as GPT-3 [28] and LLaMA [65].”
Waqas, A., Bui, M. M., Glassy, E. F., Naqa, I. E., Borkowski, P., Borkowski, A., & Rasool, G. (2023) · Laboratory Investigation · doi:10.1016/j.labinv.2023.100255 · Cited by 149 · see citing works
Large language models for structured reporting in radiology: past, present, and future
Busch, F., Hoffmann, L., dos Santos, D. P., Makowski, M. R., Saba, L., Prucker, P., Hadamitzky, M., Navab, N., Kather, J. N., Truhn, D., et al. (2024) · European Radiology · doi:10.1007/s00330-024-11107-6 · Cited by 134 · see citing works
“Eventually, the term large language model (LLM) was adopted to describe transformer-based models with billions of parameters that are trained on vast amounts of textual data, allowing them to learn more nuanced relationships and solve more NLP complex tasks [15, 26 – 28].”
Data-driven stock forecasting models based on neural networks: A review
Bao, W., Cao, Y., Yang, Y., Che, H., Huang, J., & Wen, S. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102616 · Cited by 145 · see citing works
The benefits and dangers of anthropomorphic conversational agents
Peter, S., Riemer, K., & West, J. D. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2415898122 · Cited by 110 · see citing works
How understanding large language models can inform the use of ChatGPT in physics education
Polverini, G., & Gregorcic, B. (2023) · European Journal of Physics · doi:10.1088/1361-6404/ad1420 · Cited by 131 · see citing works
“Shanahan [30] explains that because answers to questions rarely directly follow the question itself in most data sources, it is unlikely that “auto-completing” the text following the question mark would provide the answer to the question asked.”
BackgroundOn the creativity of large language models
Franceschelli, G., & Musolesi, M. (2025) · AI & SOCIETY · doi:10.1007/s00146-024-02127-3 · Cited by 113 · see citing works
“By relying only on given distributions and being trained on them, LLMs might at most express combinatorial or exploratory creativity.”
InfluentialPerils and opportunities in using large language models in psychological research
Abdurahman, S., Atari, M., Karimi-Malekabadi, F., Xue, M. J., Trager, J., Park, P. S., Golazizian, P., Omrani, A., & Dehghani, M. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae245 · Cited by 111 · see citing works
History, development, and principles of large language models: an introductory survey
Wang, Z., Chu, Z., Doan, T. V., Ni, S., Yang, M., & Zhang, W. (2025) · AI and Ethics · doi:10.1007/s43681-024-00583-7 · Cited by 109 · see citing works
“This survey distinguishes itself from other reviews on PLMs or LLMs [4, 12–14], which often concentrate on specific aspects using technical language; this survey aims to elucidate LLMs and their principles in a more accessible manner, intending to maximize its full potential.”
McCoy, R. T., Yao, S., Friedman, D., Hardy, M. D., & Griffiths, T. L. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2322420121 · Cited by 96 · see citing works
Scientific Large Language Models: A Survey on Biological & Chemical Domains
Zhang, Q., Ding, K., Lv, T., Wang, X., Yin, Q., Zhang, Y., Jing, Y., Wang, Y., Li, X., Xiang, Z., et al. (2025) · ACM Computing Surveys · doi:10.1145/3715318 · Cited by 155 · see citing works
A phenomenology and epistemology of large language models: transparency, trust, and trustworthiness
Heersmink, R., de Rooij, B., Clavel Vázquez, M. J., & Colombo, M. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09777-3 · Cited by 91 · see citing works
“Shanahan (2024) also emphasises that the dialogical behavior exhibited by LLMs can generate in us the experience of being in the presence of a human-like interlocutor.”
An overview of large AI models and their applications
Tu, X., He, Z., Huang, Y., Zhang, Z., Yang, M., & Zhao, J. C. (2024) · Visual Intelligence · doi:10.1007/s44267-024-00065-8 · Cited by 95 · see citing works
“Large foundational models, based on the type of input data they handle, can be further divided into three major types: LLMs [22], LVMs [23], and LMMs [24].”
Zhu, Y., Zhou, Y., Yan, L., et al. (2024) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2024.3437651 · Cited by 77 · see citing works
“Research from Google highlights that large models display dramatic improvements in complex tasksatcertainmodelscales[10].”
BackgroundA Survey of Robot Intelligence with Large Language Models
Jeong, H., Lee, H., Kim, C., & Shin, S. (2024) · Applied Sciences · doi:10.3390/app14198868 · Cited by 80 · see citing works
Wang, Y., Zhao, L., Li, Z., Xi, Y., Pan, Y., Zhao, G., & Zhang, L. (2025) · Nature Microbiology · doi:10.1038/s41564-025-02114-4 · Cited by 87 · see citing works
“Typically, LLMs refer to those built on Transformer-based architectures with hundreds of millions (or even billions) of trainable parameters, trained on extensive textual corpora 8 .”
Mirages. On Anthropomorphism in Dialogue Systems
Abercrombie, G., Curry, A., Dinkar, T., Rieser, V., & Talat, Z. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.290 · Cited by 83 · see citing works
“Terminology There is increasing awareness that the anthropomorphic language and jargon used to describe technologies such as language models contributes to inaccurate perceptions of their capabilities, particularly among the general public (Hunger, 2023; Salles et al., 2020; Shanahan, 2023).”
BackgroundConceptualizing generative AI as style engines: Application archetypes and implications
Riemer, K., & Peter, S. (2024) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2024.102824 · Cited by 63 · see citing works
Large language models for medicine: a survey
Zheng, Y., Gan, W., Chen, Z., Qi, Z., Liang, Q., & Yu, P. S. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02318-w · Cited by 81 · see citing works
“In recent years, the emergence of large language models (LLMs) [8, 9] has represented a significant breakthrough in the fields of artificial intelligence [10] and data sciences.”
Kim, S. S. Y., Vaughan, J. W., Liao, Q. V., Lombrozo, T., & Russakovsky, O. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714020 · Cited by 58 · see citing works
Systematic literature review on bias mitigation in generative AI
Afreen, J., Mohaghegh, M., & Doborjeh, M. (2025) · AI and Ethics · doi:10.1007/s43681-025-00721-9 · Cited by 66 · see citing works
Liu, C., Arulappan, A., Naha, R. K., Mahanti, A., Kamruzzaman, J., & Ra, I.-H. (2024) · IEEE Access · doi:10.1109/access.2024.3445413 · Cited by 61 · see citing works
“Pre-trained language models (PLMs) have proven highly effective in various natural language processing (NLP) tasks, as evidenced in several studies [33], [34].”
BackgroundThe role of large language models in agriculture: harvesting the future with LLM intelligence
Shaikh, T. A., Rasool, T., Veningston, K., & Yaseen, S. M. (2024) · Progress in Artificial Intelligence · doi:10.1007/s13748-024-00359-4 · Cited by 61 · see citing works
“This is a prominent differentiating characteristic between LLMs and PLMs. OpenAI researchers found that bigger models would consistently demonstrate superior performance and significantly enhanced sampling efficiency compared to previous iterations [28].”
Guo, Z., Tang, F., Luo, L., Zhao, M., & Kato, N. (2025) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2025.3568637 · Cited by 52 · see citing works
“LLMs based on deep learning techniques possess capabilities in understanding, generating, and reasoning with natural language [30], representing a milestone in the advancement of natural language processing (NLP) technologies.”
Li, S., Azfar, T., & Ke, R. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3508471 · Cited by 56 · see citing works
“The term ‘large language model’ generally refers to transformer architecture based language models consisting of billions of parameters and trained on extensive text data [16].”
Nie, T., Sun, J., & Ma, W. (2025) · Artificial Intelligence for Transportation · doi:10.1016/j.ait.2025.100003 · Cited by 56 · see citing works
“…capabilities such as language understanding, in-context learning, multimodal reasoning, and human-like decision making, LLMs have evolved beyond their initial functionality as text generators, becoming versatile and general problem solvers (Zhao et al., 2023; Minaee et al., 2024; Shanahan, 2024).”
Is AI my co-author? The ethics of using artificial intelligence in scientific publishing
Moffatt, B., & Hall, A. (2024) · Accountability in Research · doi:10.1080/08989621.2024.2386285 · Cited by 42 · see citing works
“More importantly, it cannot provide a justification because it did not in any meaningful way understand the output it generated (Bender and Koller 2020; Lake and Murphy 2023; Maggiori 2023; Mitchell 2019; Saba 2023; Shanahan 2024).”
InfluentialAchieving Peak Performance for Large Language Models: A Systematic Review
Rostam, Z. R. K., Szénási, S., & Kertész, G. (2024) · IEEE Access · doi:10.1109/access.2024.3424945 · Cited by 39 · see citing works
“As a result, LLM is the term used to refer to large-sized PLMs [48], [51], [52].”
A Primer for Evaluating Large Language Models in Social-Science Research
Abdurahman, S., Ziabari, A. S., Moore, A. K., Bartels, D. M., & Dehghani, M. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251325174 · Cited by 51 · see citing works
“There is an emerging literature discussing issues with anthropomorphism of LLM and other artificial-intelligence systems and how this can lead to a skewed perception of LLMs and their behaviors (e.g., see discussions in Abdurahman et al., 2024; Messeri & Crockett, 2024; Shanahan, 2024).”
Inie, N., Druga, S., Zukerman, P., & Bender, E. M. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3659040 · Cited by 39 · see citing works
“In cases of error or mal-function, determining responsibility can be challenging, especially when users have been led to view these systems as “intelligent” entities.”
BackgroundAutomatic Construction Accident Report Analysis Using Large Language Models (LLMs)
Ahmadi, E., Muley, S., & Wang, C. (2025) · Journal of Intelligent Construction · doi:10.26599/jic.2024.9180039 · Cited by 35 · see citing works
van Es, K., & Nguyen, D. (2025) · AI & SOCIETY · doi:10.1007/s00146-024-02108-6 · Cited by 37 · see citing works
“If the aim is to assess the trustworthiness and safety of AI systems, focus should shift to how these technologies work, and it becomes imperative to avoid attributing capacities they lack by anthropomorphizing them (Shanahan 2024).”
Privacy Preserving Prompt Engineering: A Survey
Edemacu, K., & Wu, X. (2025) · ACM Computing Surveys · doi:10.1145/3729219 · Cited by 48 · see citing works
“The term large language models (LLMs) is employed to delineate these large-sized PLMs [113, 125, 140].”
The influence of mental state attributions on trust in large language models
Colombatto, C., Birch, J., & Fleming, S. M. (2025) · Communications Psychology · doi:10.1038/s44271-025-00262-1 · Cited by 34 · see citing works
“Thetendency toassignmentalstatestoAIsystemsisindependentfromwhetherthese systemstrulypossessthem — and indeed, the extent to which current AI systems possess consciousness remains contentious in scienti fi c analysis 1 – 5 .”
Future Shock: Generative AI and the International AI Policy and Governance Crisis
Leslie, D., & Perini, A. M. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.88b4cc98 · Cited by 33 · see citing works
Is artificial consciousness achievable? Lessons from the human brain
Farisco, M., Evers, K., & Changeux, J. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106714 · Cited by 36 · see citing works
A question-answering framework for automated abstract screening using large language models
Akinseloyin, O., Jiang, X., & Palade, V. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae166 · Cited by 31 · see citing works
“46,47 LLMs have shown impressive generalizability across diverse domains and strong zero-/few-shot reasoning capabilities.”
BackgroundAI-Testimony, Conversational AIs and Our Anthropocentric Theory of Testimony
Freiman, O. (2024) · Social Epistemology · doi:10.1080/02691728.2024.2316622 · Cited by 33 · see citing works
“In a nutshell, the models are trained on a vast amount of texts to learn patterns and relationships (Shanahan 2022).”
MethodConsciousness science: where are we, where are we going, and what if we get there?
Cleeremans, A., Mudrik, L., & Seth, A. K. (2025) · Frontiers in Science · doi:10.3389/fsci.2025.1546279 · Cited by 32 · see citing works
Zhou, Y., Ni, T., Lee, W., & Zhao, Q. (2025) · Transactions on Artificial Intelligence · doi:10.53941/tai.2025.100003 · Cited by 34 · see citing works
“LLMs are AI systems trained on massive amounts of textual data to understand and generate human language [8,9].”
Lemke, C., Kirchner, K., Anandarajah, L., & Herfurth, F. (2023) · European Conference on e-Learning · doi:10.34190/ecel.22.1.1828 · Cited by 31 · see citing works
Generative AI and the Metaverse: A Scoping Review of Ethical and Legal Challenges
Tabassum, A., Elmahjub, E., Padela, A. I., Zwitter, A., & Qadir, J. (2025) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2025.3536082 · Cited by 26 · see citing works
“By powering the creation of content, enabling seamless interaction, and driving personalization, GenAI addresses some of the metaverse’s most significant technological challenges [10], [36].”
Joffe, I., Felobes, G., Elgouhari, Y., Talebi-Kalaleh, M., Mei, Q., & Chui, Y. H. (2025) · Journal of Computing in Civil Engineering · doi:10.1061/jccee5.cpeng-6037 · Cited by 25 · see citing works
Simulacra as conscious exotica
Shanahan, M. (2026) · Inquiry · doi:10.1080/0020174x.2024.2434860 · Cited by 25 · see citing works
“The position adopted in this paper, following (Shanahan, 2024), is that we should be wary of taking too seriously talk of beliefs in simple LLM-based conversational agents, despite their impressive conversational skills, since they lack the means to “participate fully in the human language game of…”
Self-citationDevelopment and Comparative Evaluation of a Reinstructed GPT ‐4o Model Specialized in Periodontology
Fanelli, F., Saleh, M. H. A., Santamaria, P., Zhurakivska, K., Nibali, L., & Troiano, G. (2024) · Journal Of Clinical Periodontology · doi:10.1111/jcpe.14101 · Cited by 23 · see citing works
Towards a mixed human–machine creativity
Farina, M., Pedrycz, W., & Lavazza, A. (2024) · Journal of Cultural Cognitive Science · doi:10.1007/s41809-024-00146-6 · Cited by 27 · see citing works
Leveraging large language models for peptide antibiotic design
Guan, C., Fernandes, F. C., Franco, O. L., & de la Fuente‐Núñez, C. (2024) · Cell Reports Physical Science · doi:10.1016/j.xcrp.2024.102359 · Cited by 22 · see citing works
Nfaoui, E. H., & Elfaik, H. (2024) · Journal of theoretical and applied electronic commerce research · doi:10.3390/jtaer19020058 · Cited by 20 · see citing works
“…large language models (LLMs) refer to transformer language models obtained by scaling model size (hundreds of billions or more of parameters) [58], pretraining corpus and computation. by scaling model size (hundreds of billions or more of parameters) [58], pretraining corpus and computation.”
BackgroundMetin, U., & Göymen, M. (2025) · American Journal of Orthodontics and Dentofacial Orthopedics · doi:10.1016/j.ajodo.2025.04.008 · Cited by 22 · see citing works
A RAG-Based Question-Answering Solution for Cyber-Attack Investigation and Attribution
Rajapaksha, S., Rani, R., & Karafili, E. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-82362-6_15 · Cited by 24 · see citing works
The role of Large Language Models in addressing IoT challenges: A systematic literature review
Vito, G. D., Palomba, F., & Ferrucci, F. (2025) · Future Generation Computer Systems · doi:10.1016/j.future.2025.107829 · Cited by 22 · see citing works
Jaeger, J. (2024) · Neurons Behavior Data analysis and Theory · doi:10.51628/001c.94404 · Cited by 17 · see citing works
Towards Incremental Learning in Large Language Models: A Critical Review
Jovanovic, M., & Voss, P. (2025) · Expert Systems · doi:10.1111/exsy.70127 · Cited by 24 · see citing works
“Otherwise, we should be able to reverse-engineer their responses to understand an internal process that leads to the reasoning-driven response [204].”
Enriching automatic test case generation by extracting relevant test inputs from bug reports
Ouédraogo, W. C., Plein, L., Kaboré, K., Habib, A., Klein, J., Lo, D., & Bissyandé, T. F. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10635-z · Cited by 24 · see citing works
Hou, R., Wang, X., Zhang, W., Song, Z., Wang, K., Chen, Y., Liu, J., & Ruan, T. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3529682 · Cited by 15 · see citing works
“Large language model: The LLM refers to a language model that contains hundreds of billions (or more) of parameters and is trained on extensive text data [19].”
Six Fallacies in Substituting Large Language Models for Human Participants
Lin, Z. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251357566 · Cited by 20 · see citing works
“Such attribution gaps can lead researchers and the public to either overestimate or underestimate these systems’ capabilities (Crockett & Messeri, 2023; Shanahan, 2024), potentially shaping AI policy in ways disconnected from technological reality (Lin, 2025a).”
InfluentialShan, R., Wang, Y., Guo, S., Cui, Y., Zhao, L., Li, J., & Wang, Z. (2025) · Energies · doi:10.3390/en18133542 · Cited by 15 · see citing works
LLMs for legal reasoning: A unified framework and future perspectives
Nguyen, H.-T., Fungwacharakorn, W., Zin, M. M., Goebel, R. G., Toni, F., Stathis, K., & Satoh, K. (2025) · Computer law & security review · doi:10.1016/j.clsr.2025.106165 · Cited by 15 · see citing works
Medical multimodal large language models: A systematic review
Hu, Y., Xu, C., Lin, B., Yang, W., & Tang, Y. Y. (2025) · Intelligent Oncology · doi:10.1016/j.intonc.2025.09.005 · Cited by 14 · see citing works
Dual-process theory and decision-making in large language models
Brady, O., Nulty, P., Zhang, L., Ward, T. E., & McGovern, D. P. (2025) · Nature Reviews Psychology · doi:10.1038/s44159-025-00506-1 · Cited by 16 · see citing works
Li, X., Zheng, Z., Zhao, M., Zhao, Y., Shi, L., & Wang, B. (2025) · Computer Networks · doi:10.1016/j.comnet.2025.111341 · Cited by 14 · see citing works
Compacting AI: In Search of the Small Language Model
Jovanovic, M., & Campbell, M. (2024) · Computer · doi:10.1109/mc.2024.3404674 · Cited by 14 · see citing works
Tang, B., Liang, J., Hu, W., & Luo, H. (2025) · Systems · doi:10.3390/systems13070555 · Cited by 16 · see citing works
With Friends Like These: Love and Friendship with AI Agents
Lott, M., & Hasselberger, W. (2025) · Topoi · doi:10.1007/s11245-025-10247-8 · Cited by 11 · see citing works
Gjorgjevikj, A., Martinc, M., Cenikj, G., Stojanov, R., Drole, J., Ispirova, G., Menichetti, G., Ogrinc, N., Trajanov, D., Džeroski, S., et al. (2026) · Current Research in Food Science · doi:10.1016/j.crfs.2026.101351 · Cited by 11 · see citing works
“Recent advances in Artificial Intelligence (AI), particularly Large Language Models (LLMs) (Shanahan, 2024; Luo et al., 2025a; Luo et al., 2025b), provide powerful tools for tackling complex challenges in food systems.”
Future Shock: Grappling With the Generative AI Revolution
Leslie, D., & Meng, X. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.fad6d25c · Cited by 11 · see citing works
Humans Learn Language from Situated Communicative Interactions. What about Machines?
Beuls, K., & Eecke, P. V. (2024) · Computational Linguistics · doi:10.1162/coli_a_00534 · Cited by 15 · see citing works
Dove, G. O. (2024) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2023.0149 · Cited by 12 · see citing works
Code Clone Detection Techniques Based on Large Language Models
Almatrafi, A. A., Eassa, F. E., & Sharaf, S. A. (2025) · IEEE Access · doi:10.1109/access.2025.3549780 · Cited by 11 · see citing works
“Ongoing research continues to expand the capabilities of both transformers and LLMs, emphasizing their dynamic nature in addressing evolving challenges in natural language processing [18].”
Road of Large Language Model: Source, Challenge, and Future Perspectives
Zhao, W., Yang, X. S., Lv, Z., Xu, C. X., & Guan, Z. (2025) · Research · doi:10.34133/research.0655 · Cited by 21 · see citing works
“Through scaling PLM, in terms of both model size and amount of training data, research ers found that larger PLM exhibits distinct behaviors compared to smaller ones and achieves amazing performance, leading to the development of large language models (LLMs) [ 4 ].”
Dong, X., Wang, Z., & Han, S. (2025) · Informatics · doi:10.3390/informatics12020051 · Cited by 15 · see citing works
Levich, S., & Knust, L. (2025) · International Journal of Accounting Information Systems · doi:10.1016/j.accinf.2025.100750 · Cited by 11 · see citing works
AI Mimicry and Human Dignity: Chatbot Use as a Violation of Self‐Respect
van der Rijt, J., Mollo, D. C., & Vaassen, B. (2025) · Journal of Applied Philosophy · doi:10.1111/japp.70037 · Cited by 13 · see citing works
“…do not have propositional attitudes nor a belief-desire psychology (and, arguably, no psychology at all), do not experience emotions (or anything else, for that matter), and are far from fulfilling the requirements for personhood (Shevlin and Halina, 2019; Zimmerman et al., 2023; Shanahan, 2024).”
Large Language Models for Automated Web-Form-Test Generation: An Empirical Study
Li, T., Cui, C., Huang, R., Towey, D., & Ma, L. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3735553 · Cited by 17 · see citing works
From intelligence to autopoiesis: rethinking artificial intelligence through systems theory
Zönnchen, B., Dzhimova, M., & Socher, G. (2025) · Frontiers in Communication · doi:10.3389/fcomm.2025.1585321 · Cited by 11 · see citing works
“Likewise, Shanahan’s argument concerning LLMs’ lack of embodiment (Shanahan, 2024) is valid: without structural coupling to a living body, i.e., a living system, LLMs lack key perturbation that shape human sense-making.”
InfluentialMachine learning: from perovskite materials to devices
Chen, J., Wang, M. T., Jiang, C., Dou, X., Yin, X., & Yue, Y. (2025) · Solar Energy · doi:10.1016/j.solener.2025.113684 · Cited by 10 · see citing works
Security of Language Models for Code: A Systematic Literature Review
Chen, Y., Sun, W., Fang, C., Chen, Z., Ge, Y., Han, T., Zhang, Q., Liu, Y., Chen, Z., & Xu, B. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3735554 · Cited by 10 · see citing works
Spiegler, S. V., Hoda, R., & Pant, A. (2025) · Technology in Society · doi:10.1016/j.techsoc.2025.103109 · Cited by 11 · see citing works
Knowledge graph and large language model synergy for food safety: Approaches and perspectives
An, H., Dong, Q., Doulgeraki, A. I., Nychas, G.-J. E., Zhou, Z., & Liu, Y. (2026) · Trends in Food Science & Technology · doi:10.1016/j.tifs.2026.105592 · Cited by 11 · see citing works
Phelps, S., & Russell, Y. I. (2025) · Journal of Physics Complexity · doi:10.1088/2632-072x/ada711 · Cited by 35 · see citing works
“…intelligence has been a recurring debate in AI [7; 61; 148], with many researchers emphasizing that GPT (and similar LLMs) are merely giving us an illusion of intelligence as it performs “linguistic acts” rather than actually understanding the language that it is using [50; 148; 149; 150; 151].”
BackgroundInfluentialYin, H., Aryani, A., Lambert, G. W., Wu, Z., Nambiar, N. M., White, M. R., Salvador‐Carulla, L., Sadiq, S. W., Sojli, E., Boddy, J., et al. (2025) · Array · doi:10.1016/j.array.2025.100419 · Cited by 9 · see citing works
Estimation of disciplinary similarity with large language models
Cantone, G. G., Zheng, E.-T., Tomaselli, V., & Nightingale, P. (2025) · Scientometrics · doi:10.1007/s11192-025-05385-0 · Cited by 9 · see citing works
“They are capable of communicating with a human agent, behaving similarly to so‑called chatbots , but being capable of generalising and generating a much wider range of answers (Mei et al., 2024; Noy & Zhang, 2023; Nejjar et al., 2024; Ray, 2023; Shanahan, 2024).”
Chalehchaleh, R., Farahbakhsh, R., & Crespi, N. (2025) · Social Network Analysis and Mining · doi:10.1007/s13278-025-01505-z · Cited by 9 · see citing works
“These transformer-based models, characterized by hundreds of billions of parameters trained on extensive corpora (Shanahan 2024), have demonstrated strong capabilities in language understanding and text generation (Yang et al. 2024).”
Ilagan, J. B., & Ilagan, J. R. (2023) · doi:10.35542/osf.io/azhf9 · Cited by 9 · see citing works
AI Consciousness: A Centrist Manifesto
Birch, J. (2025) · doi:10.31234/osf.io/af7c9_v1 · Cited by 10 · see citing works
Using large language models for grading in education: an applied test for physics
Mok, R., Akhtar, F., Clare, L., Li, C., Ida, J., Ross, L., & Campanelli, M. (2025) · Physics Education · doi:10.1088/1361-6552/adb92b · Cited by 24 · see citing works
Efficient Dynamic Indexing for Range Filtered Approximate Nearest Neighbor Search
Zhang, F., Jiang, M., Hou, G., Shi, J., Fan, H. T., Zhou, W., Li, F., & Wang, S. (2025) · Proceedings of the ACM on Management of Data · doi:10.1145/3725401 · Cited by 9 · see citing works
Data Contracts in Data Mesh: A Systematic Gray Literature Review
Wasser, J., Kumara, I., Monsieur, G., van den Heuvel, W., & Tamburri, D. A. (2025) · Lecture notes in business information processing · doi:10.1007/978-3-031-98033-6_2 · Cited by 8 · see citing works
Unlocking the Potentials of Large Language Models in Orthodontics: A Scoping Review
Zheng, J., Ding, X., Pu, J. J., Chung, S. M., Ai, Q. Y. H., Hung, K. F., & Shan, Z. (2024) · Bioengineering · doi:10.3390/bioengineering11111145 · Cited by 7 · see citing works
“LLMs have demonstrated remarkable abilities in understanding human language and generating human-like dialogue, leading to their widespread use [22,23].”
HumanEvalComm: Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agent
Wu, J. J., & Fard, F. H. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3715109 · Cited by 10 · see citing works
Multimodal stock market emotion recognition model trained with a large language model
Liu, C., Miao, Y., Zhao, Q., Wang, C., & Zhu, X. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111035 · Cited by 8 · see citing works
Do Multimodal Large Language Models and Humans Ground Language Similarly?
Jones, C. R., Bergen, B. K., & Trott, S. (2024) · Computational Linguistics · doi:10.1162/coli_a_00531 · Cited by 9 · see citing works
Vatavu, R.-D. (2024) · doi:10.1145/3678957.3678958 · Cited by 7 · see citing works
Artificial and Post-Artificial Texts
Bajohr, H. (2023) · Basel Media Culture and Cultural Techniques Working Papers · doi:10.12685/bmcct.2023.007 · Cited by 7 · see citing works
Qorich, M., & El Ouazzani, R. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128405 · Cited by 6 · see citing works
Çetinkaya, İ. O., Büyüktahtakın, İ. E., Shojaee, P., et al. (2025) · Computers & Operations Research · doi:10.1016/j.cor.2025.107325 · Cited by 6 · see citing works
“Large Language Models (LLMs) have emerged as powerful tools capable of completing complex tasks (Zhao et al. (2023); Shanahan (2024)), however, they are still far away from achieving algorithmic innovation by themselves.”
Zeng, T., & Zhang, Y. (2025) · Ocean Engineering · doi:10.1016/j.oceaneng.2025.123628 · Cited by 6 · see citing works
- Authors
- Shanahan, M.
- Affiliations
- Imperial College London (GB)
- Corresponding author
- Shanahan, Murray
- ORCID
- Shanahan, Murray
- Venue
- Communications of the ACM
- Published
- 2024-01-25
- Volume
- 67
- Issue
- 2
- Pages
- 68-79
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W4391215636
- Cited by
- 381
- References
- 26
- Citations by year
- 2026: 156 · 2025: 147 · 2024: 69 · 2023: 9
- FWCI
- 175.84
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Communications of the ACM (publishedVersion)
- Topics
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- large language models, chatbots, artificial intelligence, human-AI interaction, anthropomorphism, machine cognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read under definition version 3 (extract-v3) on 2026-09-28.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-28.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
