The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index; 19 of the 25 sources its claims cite were checked for support.
ChatGPT, extended: large language models and the extended mind
Open accessAbstract
Recent research has relied on the use of fine-tuning techniques to incorporate philosophical knowledge into Large Language Models (LLMs). The present paper outlines an alternative approach to the development of such systems—one that is rooted in a technique known as Retrieval-Augmented Generation (RAG). In contrast to fine-tuning, RAG does not seek to adjust the internal parameters (or internal memory) of an LLM. Instead, RAG relies on the retrieval of information from an externally-situated store, which functions as a form of non-parametric (or external) memory. Applying this technique to the works of the contemporary philosopher Andy Clark yields Digital Andy: an LLM that is able to respond to questions about the extended mind. This serves as a practical demonstration of RAG-based techniques, highlighting how philosophical knowledge can be ‘incorporated’ into an LLM without the need for additional machine learning. But Digital Andy’s reliance on extra-systemic resources also raises questions about the scope of active externalist theorizing, encouraging us to consider Digital Andy’s status as an extended cognitive/computational system. Addressing these questions reveals some interesting points of convergence between the philosophical effort to understand the extended mind and the technological effort to build the next generation of LLMs.
Overview
CoChat Summary
TL;DR
The paper builds Digital Andy, a Retrieval-Augmented Generation (RAG) LLM that uses Andy Clark’s published works as external memory, and uses it to ask whether such a system qualifies as extended AI. It finds RAG can tailor responses without fine-tuning, but concludes Digital Andy fails as an EXAI system because the LLM does not trigger the retrieval loop; the Digital Andy App does.
Key findings
- Digital Andy’s RAG conditions (DA PARAGRAPHS and DA SECTIONS) produced better responses than GPT DEFAULT for Clark-specific queries: GPT DEFAULT did not recognize “wideware” and treated “Otto” as a name of Germanic origin, while the RAG conditions correctly connected these to Clark’s work.
- Paragraph chunking sometimes outperformed section chunking for fine-grained queries: for a query about Andy’s cat Lolo, DA SECTIONS failed to retrieve any chunks mentioning Lolo, while DA PARAGRAPHS retrieved a paragraph from Clark (2003) that explicitly mentioned Lolo; the authors attribute this to retrieval, not generation.
- GPT MODIFIED (no retrieval, same system prompt as Digital Andy) was unexpectedly better than GPT DEFAULT and sometimes approximated the RAG conditions, suggesting the base GPT model already had some Clark-related information in internal memory; this raises the “redundancy problem.”
- The authors conclude Digital Andy fails to qualify as an EXAI system because of the “agency problem”: the retrieval loop is triggered by the Digital Andy App, not by the LLM, and the LLM is a passive recipient of retrieved information. They suggest active RAG and agentic LLMs may address this.
Methods
- Built a RAG system using 58 first-authored Andy Clark publications (7 books, 34 journal articles, 17 contributions to edited works; 979,769 words). Text was extracted from PDFs, stripped of footnotes/endnotes, figures, captions, keywords, acknowledgements, content listings, and references, then chunked into paragraph chunks (7,931 chunks) and section chunks (1,086 chunks) with a 3,000-token maximum. Chunks were embedded with OpenAI’s text-embedding-ada-002 into 1,536-dimensional vectors and stored in a Supabase PostgreSQL vector database.
- At inference, a user query was embedded and used for vector similarity search (L2 distance), retrieving 20 paragraph chunks or 5 section chunks. The Digital Andy App constructed a prompt with an instruction statement, context window, and query, then posted it to gpt-3.5-turbo-1106 with a system prompt encouraging impersonation of Andy Clark.
- Evaluation used 20 queries on aspects of Clark’s work, recorded in four conditions: DA PARAGRAPHS, DA SECTIONS, GPT DEFAULT (no retrieval, default system prompt), and GPT MODIFIED (no retrieval, same system prompt as Digital Andy). Responses were compared qualitatively, with a tracer facility listing retrieved chunks; the authors selected the first response per query from a single batch run.
Limitations
- Paper-stated: the authors make no claims about the optimality of the prompt; they selected only the first response per query and did not invoke the model multiple times; they note that lack of information about an LLM’s training regime makes it difficult to gauge what the model knows; footnotes/endnotes were removed even though a reviewer noted they may contain important information; the chunking strategies used are not the only possible ones; and the agency-problem interpretation is not shared by all authors.
- Paper-stated: the redundancy problem is contingent on the focus on Andy Clark, whose popularity and extensive publication history make it likely that ChatGPT was exposed to his work during training; it might not arise with a lesser-known philosopher or non-public-domain material.
- Evident from the text: the evaluation is informal and qualitative, based on 20 queries and illustrative examples rather than systematic metrics or statistical tests; the base model is an older GPT version; and possible training-data overlap with Clark’s work complicates claims about the role of external memory.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
13 Valid · 3 Not valid · 4 Not checked · 105 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“In contrast to fine-tuning, RAG does not seek to adjust the internal parameters (or internal memory) of an LLM.”
What the sources say
- Not matchedcites Gao et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Disputedcites Lewis et al., 2020 sourceExperimental — accuracy not yet measured
The claim says RAG does not seek to adjust an LLM’s internal parameters, but the paper describes RAG as a fine-tuning approach (Introduction, p. 2). Its Training section (p. 3) explicitly says the authors fine-tune the BART generator and query encoder. The paper does show that its retrieval index can be replaced without retraining (Discussion, p. 9), but that does not mean RAG leaves model parameters unchanged during training. (LLM, conf=0.94)
Checked by a reasoning model — may be wrong.
- Partly supportedcites Ram et al., 2023 sourceExperimental — accuracy not yet measured
Section 3.1 says the paper’s specific In-Context RALM method does not alter LM weights, supporting the contrast with fine-tuning for that method. But the Introduction and Discussion say other retrieval-augmented approaches modify LM architectures or rely on fine-tuning, so the source does not support the claim about RAG in general. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
Every source the paper cites for this claim was checked and supports it.
In the paper
“In this case, the model fails to report the connection between the Dark Room Problem and predictive processing, which is a prominent feature of Clark's work (Clark, 2018; Friston et al., 2012).”
What the sources say
- Supportedcites Clark, 2018 source
Section 2 presents the Darkened Room scenario as a prominent challenge to the predictive-processing framework. Section 7 returns to that connection in its conclusions. The statement about what the GPT model failed to report is the citing authors’ observation, not a finding attributed to Clark. (LLM, conf=0.94)
- Supportedcites Friston et al., 2012 source
The paper’s Abstract identifies the Dark-Room Problem as a puzzle for a theory of brain function centered on minimizing surprise and names Clark as a participant in the discussion. A later passage in the same indexed section explains the problem through model-based expectations and predicted sensory signals, supporting the connection to predictive processing. The claim about what the GPT model failed to report is the citing authors’ observation, not a finding attributed to this source. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Instead, RAG relies on the retrieval of information from an externally-situated store, which functions as a form of non-parametric (or external) memory.”
What the sources say
- Not matchedcites Gao et al., 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites Lewis et al., 2020 source
The Introduction (p. 2) identifies RAG’s non-parametric memory as a dense vector index of Wikipedia accessed by a neural retriever. The Methods (p. 2) confirms that RAG retrieves text documents and uses them as context for generation. (LLM, conf=0.94)
- Supportedcites Ram et al., 2023 source
Section 3.1 (page not supplied) states that retrieval-augmented language models retrieve documents from an external corpus and condition predictions on them. This supports the description of the corpus as an external, non-parametric source of information, although the paper does not use the word “memory” in the quoted passage. (LLM, conf=0.90)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
Every source the paper cites for this claim was checked and supports it.
In the paper
“we do not doubt the more general point that Floridi and Chiriatti (2020) are making here, namely, that LLMs exhibit errors when confronted with tasks that rely on specialized skills and knowledge (in this case, a capacity for mathematical reasoning).”
What the sources say
- Supportedcites Floridi and Chiriatti (2020) source
In §3, “Three Tests: Mathematics, Semantics, and Ethics,” the authors report that GPT-3 gives an incorrect answer to a mathematical problem and describe errors on questions requiring semantic understanding. These examples support the attributed point that an LLM can make errors on such tasks. (LLM, conf=0.90)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“To be sure, LLMs are perhaps ill-equipped to solve certain sorts of problems, such as those involving complex mathematical calculations.”
What the sources say
- Partly supportedcites Floridi and Chiriatti (2020) sourceExperimental — accuracy not yet measured
In “Three Tests: Mathematics, Semantics, and Ethics,” the paper shows GPT-3 answering a simple algebra problem correctly but failing when larger numbers are substituted, and advises against relying on it for maths. This supports the concern about mathematical reliability, but the example does not test complex calculations or establish the claim for LLMs generally. (LLM, conf=0.83)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
47 more claims · 1 with their cited sources checked
Create a free account to see all 125 claims and every source verdict.
- It may state two separate claims as one. 23
- Probably not a substantial claim. 56
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“…of topics, with issues such as understanding, meaning, intelligence, value alignment, and more general ethical concerns forming the basis of recent work (Bender et al., 2021 ; Floridi, 2023 ; Floridi & Chiriatti, 2020 ; Heersmink et al., 2024; Kasirzadeh & Gabriel, 2023 ; Pezzulo et al., 2024 ).”
Clark, A., & Chalmers, D. (1998) · Analysis · doi:10.1093/analys/58.1.7 · Cited by 6,333 · cited 5×
“…the more salient by the focus on Andy Clark, for Andy Clark is a leading exponent of active externalism, championing the idea that extra-organismic resources can, on occasion, form part of the material fabric that realizes human mental states and processes (Clark & Chalmers, 1998 ; Clark, 2008 ).”
InfluentialSelf-citationRetrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · cited 1×
“While Schwitzgebel et al. ( 2024 ) rely on fine-tuning to assimilate knowledge into the internal parameters of a GPT model, we rely on a technique, called Retrieval-Augmented Generation (RAG) (Gao et al., 2024 ; Lewis et al., 2020 ; Ram et al., 2023 ).”
GPT-3: Its Nature, Scope, Limits, and Consequences
Floridi, L., & Chiriatti, M. (2020) · Minds and Machines · doi:10.1007/s11023-020-09548-1 · Cited by 2,519 · cited 5×
“(Floridi and Chiriatti, 2020 , p. 688) Although we were not able to reproduce this particular problem using the current version of ChatGPT, we do not doubt the more general point that Floridi and Chiri-atti ( 2020 ) are making here, namely, that LLMs exhibit errors when confronted with tasks that…”
InfluentialSupersizing the Mind: Embodiment, Action, and Cognitive Extension
Clark, A. (2008) · doi:10.1093/acprof:oso/9780195333213.001.0001 · Cited by 2,201
Self-citationToolformer: Language Models Can Teach Themselves to Use Tools
Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023) · arXiv · doi:10.48550/arxiv.2302.04761 · Cited by 1,150 · cited 1×
“Notable examples of such systems include the likes of LaMDA (Thoppilan et al., 2022 ), Toolformer (Schick et al., 2023 ), and Mind’s Eye (Liu et al., 2022 ).”
Retrieval-Augmented Generation for Large Language Models: A Survey
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2312.10997 · Cited by 876 · cited 1×
“While Schwitzgebel et al. ( 2024 ) rely on fine-tuning to assimilate knowledge into the internal parameters of a GPT model, we rely on a technique, called Retrieval-Augmented Generation (RAG) (Gao et al., 2024 ; Lewis et al., 2020 ; Ram et al., 2023 ).”
LaMDA: Language Models for Dialog Applications
Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., Du, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.08239 · Cited by 778 · cited 1×
“Notable examples of such systems include the likes of LaMDA (Thoppilan et al., 2022 ), Toolformer (Schick et al., 2023 ), and Mind’s Eye (Liu et al., 2022 ).”
Research works printed but not linked: 1
- Frankish, K. (2024). What are large language models doing? In A. Strasser (Ed.), Anna’s AI anthology: How to live with smart machines? (pp. 53–79). xenomoi.
Cited by
Fleischmann, K. (2026) · Arts and Humanities in Higher Education · doi:10.1177/14740222261420495 · Cited by 4
“Researchers describe AI as part of a distributed cognitive system in which aspects of thinking and memory move from the human mind to external systems (Clark and Chalmers, 1998; Smart et al., 2025).”
The coalescent architecture of agency: normative directionality as the key to human–AI integration
Torres–Martínez, S. (2026) · AI & Society · doi:10.1007/s00146-026-02979-x · Cited by 3
“Smart et al. (2025) argue that large language models cannot qualify as genuine extensions of human cognition because they lack the requisite agential properties—the “agency problem” that arises when retrieval loops are triggered externally rather than by the system itself.”
The Mind-Technology Problem in the Age of GenAI: Introduction to the Special Issue
Clowes, R. W., Gärtner, K., & Theiner, G. (2026) · Social Epistemology · doi:10.1080/02691728.2025.2574296 · Cited by 7
“…produced either by retraining nets on material such as text messages or writings of the individual to be duplicated (Strasser, Crosby, and Schwitzgebel 2022), or by using the same material by other means such as Retrieval Augmented Generation (Gao et al. 2023; P. Smart, Clowes, and Clark 2025).”
Self-citationPhilosophical Significance of Artificial Intuitions
Inarimori, K., Takeshita, M., Matsuda, A., & Miyazono, K. (2026) · Erkenntnis · doi:10.1007/s10670-026-01136-6
The Story of Your Life: Large Language Models and Personal Memory
Smart, P. R., Clowes, R. W., Krueger, J. W., & Boniface, M. (2026) · Review of Philosophy and Psychology · doi:10.1007/s13164-026-00831-1 · Cited by 1
Self-citation
Metadata
- Authors
- Smart, P., Clowes, R., & Clark, A.
- Affiliations
- University of Southampton (GB)
- Universidade Nova de Lisboa (PT)
- University of Sussex (GB)
- Corresponding author
- Smart, Paul
- Venue
- Synthese
- Published
- 2025-06-06
- Volume
- 205
- Issue
- 6
- ISSN
- 0039-7857 (ISSN-L)
- 1573-0964
- OpenAlex
- W4411097919
- Semantic Scholar
- cee08db506a9134673e61599fb216de12cf11e63
- Cited by
- 13
- References
- 50
- Citations by year
- 2026: 13
- FWCI
- 5.36
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Synthese (publishedVersion, cc-by)
- ePrints Soton (University of Southampton) (acceptedVersion)
- Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, public-domain)
- Figshare (submittedVersion, cc-by)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
- Social Sciences › Arts and Humanities › Philosophy › Philosophy and Theoretical Science
- Keywords
- extended mind, retrieval-augmented generation, large language models, Andy Clark, external memory, externalism, fine-tuning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: ChatGPT, extended: large language models and the extended mind. https://app.cochat.ai/papers/10.1007/s11229-025-05046-y
- The paper
- Smart, P., Clowes, R., & Clark, A. (2025). ChatGPT, extended: large language models and the extended mind. Synthese. https://doi.org/10.1007/s11229-025-05046-y
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.