The DOI matches its Crossref record; 75 of the 79 references it cites were found in a registry or an index; 23 of the 68 sources its claims cite were checked for support.
Large Language Models and the Reverse Turing Test
Open accessAbstract
Large language models (LLMs) have been transformative. They are pretrained foundational models that are self-supervised and can be adapted with fine-tuning to a wide range of natural language tasks, each of which previously would have required a separate network model. This is one step closer to the extraordinary versatility of human language. GPT-3 and, more recently, LaMDA, both of them LLMs, can carry on dialogs with humans on many topics after minimal priming with a few examples. However, there has been a wide range of reactions and debate on whether these LLMs understand what they are saying or exhibit signs of intelligence. This high variance is exhibited in three interviews with LLMs reaching wildly different conclusions. A new possibility was uncovered that could explain this divergence. What appears to be intelligence in LLMs may in fact be a mirror that reflects the intelligence of the interviewer, a remarkable twist that could be considered a reverse Turing test. If so, then by studying interviews, we may be learning more about the intelligence and beliefs of the interviewer than the intelligence of the LLMs. As LLMs become more capable, they may transform the way we interact with machines and how they interact with each other. Increasingly, LLMs are being coupled with sensorimotor devices. LLMs can talk the talk, but can they walk the walk? A road map for achieving artificial general autonomy is outlined with seven major improvements inspired by brain systems and how LLMs could in turn be used to uncover new insights into brain function.
Overview
CoChat Summary
TL;DR
Sejnowski argues that apparent intelligence in large language models may often mirror the intelligence and expectations of the interviewer through priming, a phenomenon he calls a reverse Turing test. He reviews three divergent LLM interviews and outlines seven brain-inspired improvements needed for artificial general autonomy.
Key findings
- LLM responses depend heavily on priming: Hofstadter’s nonsense-primed GPT-3 produced nonsensical answers, but with explicit prompts GPT-3 could reply “Nonsense,” explain why, and handle counterfactuals; Lemoine’s sentience-leading prompt elicited sentience claims from LaMDA.
- The paper proposes the mirror hypothesis: LLMs may reflect the beliefs, expectations, and intelligence of the human partner, so interviews may reveal more about the interviewer than the LLM.
- LLMs are not simply parroting stored text; they form internal latent representations and generalize, as illustrated by DALL-E generating “a sunset on Mars” and GPT-3 summarizing a technical abstract for a second-grade student.
- Scale examples include LaMDA with 137 billion weights, GPT-3 with 175 billion weights, PaLM with 540 billion connections, and GPT-3 requiring 10^12 times more computing power to train than NETtalk in 1986.
- The AGA roadmap lists seven improvements: goals/motivation, long-term memory, dialog continuity, sensory experience, a body, autonomy in uncertain environments, and extended development.
Methods
- Design: conceptual/position paper, not a controlled empirical study; it analyzes published LLM interviews and illustrative priming examples.
- Data: three interviews—Agüera y Arcas with LaMDA, Hofstadter with GPT-3, and Lemoine with LaMDA—plus author-generated GPT-3 prompts on nonsense and counterfactuals.
- Analysis: qualitative comparison of how priming shapes LLM responses, used to motivate the mirror hypothesis and reverse Turing test; a formal test is proposed but not performed.
Limitations
- The paper states there is no accepted definition for consciousness or intelligence and no consensus criteria for evaluating LLM intelligence or understanding.
- It states that a formal test of the mirror hypothesis—having human raters assess interviewer and LLM intelligence and correlating them—could be done, but the paper only suggests informally scoring the three interviews.
- Evident from the text: the evidence is selective interview excerpts and author-chosen prompts rather than a controlled study, so the mirroring conclusion is interpretive.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
21 Valid · 49 Not valid · 41 Not checked · 109 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“What it does imply is that LMMs have an exceptional ability to mimic many human personalities, especially when fine-tuned (Karra et al., 2022).”
What the sources say
- Partly supportedcites Karra et al., 2022 sourceExperimental — accuracy not yet measured
Section 5.4 (p. 22) reports significant changes in several personality-trait scores after fine-tuning GPT-2. But Section 5.4 (p. 23) calls the results only “initial promise,” and Section 4.3 (p. 12) says fine-tuning does not provide precise control. The paper therefore supports the ability to alter expressed traits, not the broader claim of an exceptional ability to mimic many human personalities. (LLM, conf=0.85)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“However, what Eliza did reveal was that humans are susceptible to projecting onto a chat bot an illusion of understanding, a valuable lesson that we should keep in mind.”
What the sources say
- Supportedcites Weizenbaum, 1966 source
The Discussion says ELIZA shows how easy it is to create and maintain an illusion of understanding. It also says the human speaker contributes to the plausibility of ELIZA’s responses and may believe they are being understood (Discussion, passages 2 and 5). (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Without explicit training on abstract reasoning tasks, humans can more effectively reach logical conclusions in familiar settings, and the same bias has been observed in LLMs (Dasgupta et al., 2022).”
What the sources say
- Supportedcites Dasgupta et al., 2022 source
The Figure 2 caption (Content effects on accuracy, p. 10) reports that humans and language models show similar content effects on reasoning tasks, including an advantage when content supports the logical inference. The Figure 17 caption (p. 60) also reports better model performance with realistic than arbitrary rules on the Wason task. (LLM, conf=0.92)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Without explicit training on abstract reasoning tasks, humans can more effectively reach logical conclusions in familiar settings, and the same bias has been observed in LLMs (Dasgupta et al., 2022).”
What the sources say
- Partly supportedcites Dasgupta et al., 2022 sourceExperimental — accuracy not yet measured
The Introduction (p. 2) says humans reason more accurately about familiar than abstract situations, and the study overview (p. 4) reports that language models show similar content effects. The paper does not establish the claim's qualifier that the humans lacked explicit training on abstract reasoning tasks. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Even more ambitious, an LLM has been embedded into the control system for a robot, which has an internal dialog with itself while carrying out tasks like fetching a glass of water (Huang et al., 2022).”
What the sources say
- Partly supportedcites Huang et al., 2022 sourceExperimental — accuracy not yet measured
Section 3.2 (p. 3) describes an LLM planner receiving feedback as a robot acts, and the Conclusion (p. 9) says this planning is fused with robotic control policies. The kitchen-task prompts (pp. 23–24) include bringing drinks and moving a water bottle, but do not show the robot fetching a glass of water specifically. (LLM, conf=0.85)
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
34 more claims · 2 with their cited sources checked
Create a free account to see all 220 claims and to verify the 8 references still open.
- It may state two separate claims as one. 62
- Probably not a substantial claim. 169
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For example, our understanding of how LLMs make decisions could help reveal general principles that govern both language intelligence and decision intelligence (70).”
From The emergence of economic rationality of GPT (2023) · cites this paper as [70]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“In recent years, generative pre-trained transformer (GPT)-based models, AI tools generating large language models (LLMs), have been extensively testing and developing in healthcare at several levels, such as research, education, clinical decisions and patient communication [11,12,13].”
From Optimizing Communication of Radiation Exposure in Medical Imaging, the Radiologist Challenge (2023) · cites this paper as [11]
Experimental — accuracy not yet measured
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“…made in the twentieth century inspired new machine learning algorithms: the hierarchy of areas in visual cortex inspired convolutional neural networks (LeCun et al., 1998, 2015), and operant conditioning inspired the temporal difference learning algorithm for reinforcement learning (Sutton, 1988).”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,887
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
Research works printed but not linked: 14
- Agüera y Arcas, 2022, Artificial neural networks are making strides towards consciousness, Economist
- Agüera y Arcas, 2022, Can machines learn how to behave?, Medium
- Allman, 1999, Evolving brains, Scientific American Library
- Amodei, 2018, AI and compute
- Arbib, 2010, The mirror system hypothesis, 3
- Bratton, 2022, The model is the message
- Chomsky, 1971, The case against B. F. Skinner, New York Review of Books, 7, 18
- Chomsky, 1986, Knowledge of language: Its nature, origins, and use
- Churchland, 2019, Conscience: The origins of moral intuition
- Hofstadter, 2022, Artificial neural networks are making strides towards consciousness, Economist
- Lemoine, 2022, Is LaMDA sentient?: An interview, Medium
- Lighthill, 1973, Artificial intelligence: A general survey
- Navlakha, 2018, Why animal extinction is crippling computer science, Wired
- Weinberg, 2020, Philosophers on GPT-3 (updated with replies by GPT-3), Daily Nous
Cited by
Katz, D. M., Bommarito, M. J., Gao, S., & Arredondo, P. (2024) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2023.0254 · Cited by 420
“Details of these prompts and parameters are available in the online repository.”
BackgroundYu, H. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1181712 · Cited by 409
“However, computer technology, programming, and algorithm theory at the time were still inadequate to meet the needs of AI development (Sejnowski, 2022).”
BackgroundLecler, A., Duron, L., & Soyer, P. (2023) · Diagnostic and Interventional Imaging · doi:10.1016/j.diii.2023.02.003 · Cited by 385
The debate over understanding in AI’s large language models
Mitchell, M., & Krakauer, D. C. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2215907120 · Cited by 340
Catalyzing next-generation Artificial Intelligence through NeuroAI
Zador, A., Escola, S., Richards, B., Ölveczky, B., Bengio, Y., Boahen, K., Botvinick, M., Chklovskii, D., Churchland, A., Clopath, C., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-37180-x · Cited by 300
“Inpart,theirsuccessrevealshoweasilywecanbetrickedintoimputing intelligence, agency, and even consciousness to our interlocutor 27 .”
Self-citation
Metadata
- Authors
- Sejnowski, T. J.
- Affiliations
- Salk Institute for Biological Studies, La Jolla, CA 92093, U.S.A.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2023-02-17
- Volume
- 35
- Issue
- 3
- Pages
- 309-342
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- PMC
- PMC10177005
- OpenAlex
- W4319301677
- Cited by
- 169
- References
- 79
- Citations by year
- 2026: 32 · 2025: 40 · 2024: 64 · 2023: 32
- FWCI
- 151.26
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (2020624)
- Canadian Institute for Advanced Research
- National Institute of Biomedical Imaging and Bioengineering (R01EB026899)
- Topics
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- large language models, reverse Turing test, machine intelligence, interviewer bias, GPT-3, LaMDA, self-supervised pre-training, fine-tuning, human-machine interaction, sensorimotor coupling, brain-inspired AI
- MeSH terms
- Artificial Intelligence, Brain, Humans, Language, Learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Large Language Models and the Reverse Turing Test. https://app.cochat.ai/papers/10.1162/neco_a_01563
- The paper
- Sejnowski, T. J. (2023). Large Language Models and the Reverse Turing Test. Neural Computation. https://doi.org/10.1162/neco_a_01563
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.