Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index; 19 of the 25 sources its claims cite were checked for support.

ChatGPT, extended: large language models and the extended mind

Open access
Smart, Paul, Clowes, Robert, Clark, Andy · 2025 · Synthese

Abstract

Recent research has relied on the use of fine-tuning techniques to incorporate philosophical knowledge into Large Language Models (LLMs). The present paper outlines an alternative approach to the development of such systems—one that is rooted in a technique known as Retrieval-Augmented Generation (RAG). In contrast to fine-tuning, RAG does not seek to adjust the internal parameters (or internal memory) of an LLM. Instead, RAG relies on the retrieval of information from an externally-situated store, which functions as a form of non-parametric (or external) memory. Applying this technique to the works of the contemporary philosopher Andy Clark yields Digital Andy: an LLM that is able to respond to questions about the extended mind. This serves as a practical demonstration of RAG-based techniques, highlighting how philosophical knowledge can be ‘incorporated’ into an LLM without the need for additional machine learning. But Digital Andy’s reliance on extra-systemic resources also raises questions about the scope of active externalist theorizing, encouraging us to consider Digital Andy’s status as an extended cognitive/computational system. Addressing these questions reveals some interesting points of convergence between the philosophical effort to understand the extended mind and the technological effort to build the next generation of LLMs.

Overview

CoChat Summary

TL;DR

The paper builds Digital Andy, a Retrieval-Augmented Generation (RAG) LLM that uses Andy Clark’s published works as external memory, and uses it to ask whether such a system qualifies as extended AI. It finds RAG can tailor responses without fine-tuning, but concludes Digital Andy fails as an EXAI system because the LLM does not trigger the retrieval loop; the Digital Andy App does.

Key findings

  • Digital Andy’s RAG conditions (DA PARAGRAPHS and DA SECTIONS) produced better responses than GPT DEFAULT for Clark-specific queries: GPT DEFAULT did not recognize “wideware” and treated “Otto” as a name of Germanic origin, while the RAG conditions correctly connected these to Clark’s work.
  • Paragraph chunking sometimes outperformed section chunking for fine-grained queries: for a query about Andy’s cat Lolo, DA SECTIONS failed to retrieve any chunks mentioning Lolo, while DA PARAGRAPHS retrieved a paragraph from Clark (2003) that explicitly mentioned Lolo; the authors attribute this to retrieval, not generation.
  • GPT MODIFIED (no retrieval, same system prompt as Digital Andy) was unexpectedly better than GPT DEFAULT and sometimes approximated the RAG conditions, suggesting the base GPT model already had some Clark-related information in internal memory; this raises the “redundancy problem.”
  • The authors conclude Digital Andy fails to qualify as an EXAI system because of the “agency problem”: the retrieval loop is triggered by the Digital Andy App, not by the LLM, and the LLM is a passive recipient of retrieved information. They suggest active RAG and agentic LLMs may address this.

Methods

  • Built a RAG system using 58 first-authored Andy Clark publications (7 books, 34 journal articles, 17 contributions to edited works; 979,769 words). Text was extracted from PDFs, stripped of footnotes/endnotes, figures, captions, keywords, acknowledgements, content listings, and references, then chunked into paragraph chunks (7,931 chunks) and section chunks (1,086 chunks) with a 3,000-token maximum. Chunks were embedded with OpenAI’s text-embedding-ada-002 into 1,536-dimensional vectors and stored in a Supabase PostgreSQL vector database.
  • At inference, a user query was embedded and used for vector similarity search (L2 distance), retrieving 20 paragraph chunks or 5 section chunks. The Digital Andy App constructed a prompt with an instruction statement, context window, and query, then posted it to gpt-3.5-turbo-1106 with a system prompt encouraging impersonation of Andy Clark.
  • Evaluation used 20 queries on aspects of Clark’s work, recorded in four conditions: DA PARAGRAPHS, DA SECTIONS, GPT DEFAULT (no retrieval, default system prompt), and GPT MODIFIED (no retrieval, same system prompt as Digital Andy). Responses were compared qualitatively, with a tracer facility listing retrieved chunks; the authors selected the first response per query from a single batch run.

Limitations

  • Paper-stated: the authors make no claims about the optimality of the prompt; they selected only the first response per query and did not invoke the model multiple times; they note that lack of information about an LLM’s training regime makes it difficult to gauge what the model knows; footnotes/endnotes were removed even though a reviewer noted they may contain important information; the chunking strategies used are not the only possible ones; and the agency-problem interpretation is not shared by all authors.
  • Paper-stated: the redundancy problem is contingent on the focus on Andy Clark, whose popularity and extensive publication history make it likely that ChatGPT was exposed to his work during training; it might not arise with a lesser-known philosopher or non-public-domain material.
  • Evident from the text: the evaluation is informal and qualitative, based on 20 queries and illustrative examples rather than systematic metrics or statistical tests; the base model is an older GPT version; and possible training-data overlap with Clark’s work complicates claims about the role of external memory.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s11229-025-05046-y

Cites

55 of 55 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

13 Valid · 3 Not valid · 4 Not checked · 105 Own results and interpretations

47 more claims · 1 with their cited sources checked

Create a free account to see all 125 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 23
  • Probably not a substantial claim. 56

Create a free account to see these claims.

Citations

Cites

Show all 50 references

Research works printed but not linked: 1

  1. Frankish, K. (2024). What are large language models doing? In A. Strasser (Ed.), Anna’s AI anthology: How to live with smart machines? (pp. 53–79). xenomoi.

Cited by

See all 13

Metadata

Authors
Smart, P., Clowes, R., & Clark, A.
Affiliations
  • University of Southampton (GB)
  • Universidade Nova de Lisboa (PT)
  • University of Sussex (GB)
Corresponding author
Smart, Paul
Venue
Synthese
Published
2025-06-06
Volume
205
Issue
6
ISSN
  • 0039-7857 (ISSN-L)
  • 1573-0964
OpenAlex
W4411097919
Cited by
13
References
50
Citations by year
2026: 13
FWCI
5.36
Type
journal article
Language
en
Open access
Licence
Locations
  • Synthese (publishedVersion, cc-by)
  • ePrints Soton (University of Southampton) (acceptedVersion)
  • Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, public-domain)
  • Figshare (submittedVersion, cc-by)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
  • Social Sciences › Arts and Humanities › Philosophy › Philosophy and Theoretical Science
Keywords
extended mind, retrieval-augmented generation, large language models, Andy Clark, external memory, externalism, fine-tuning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: ChatGPT, extended: large language models and the extended mind. https://app.cochat.ai/papers/10.1007/s11229-025-05046-y
The paper
Smart, P., Clowes, R., & Clark, A. (2025). ChatGPT, extended: large language models and the extended mind. Synthese. https://doi.org/10.1007/s11229-025-05046-y

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.