The DOI matches its DataCite record.
People cannot distinguish GPT-4 from a human in a Turing test
Open accessAbstract
We evaluated 3 systems (ELIZA, GPT-3.5 and GPT-4) in a randomized, controlled, and preregistered Turing test. Human participants had a 5 minute conversation with either a human or an AI, and judged whether or not they thought their interlocutor was human. GPT-4 was judged to be a human 54% of the time, outperforming ELIZA (22%) but lagging behind actual humans (67%). The results provide the first robust empirical demonstration that any artificial system passes an interactive 2-player Turing test. The results have implications for debates around machine intelligence and, more urgently, suggest that deception by current AI systems may go undetected. Analysis of participants' strategies and reasoning suggests that stylistic and socio-emotional factors play a larger role in passing the Turing test than traditional notions of intelligence.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLMs consistently pass the Turing test.
“they consistently pass the Turing test (20) as a result (14, 21, 22)”
From The benefits and dangers of anthropomorphic conversational agents (2025) · cites this paper as [21]
The phenomenon of anthropomorphism changes significantly when the technology already exhibits highly human-like qualities, indistinguishable from the 'real deal.'
“However, this phenomenon changes significantly when the technology already exhibits highly human-like (anthropomorphic) qualities, indistinguishable from the “real deal,” as is evidenced by the Turing Test (14, 21, 22).”
From The benefits and dangers of anthropomorphic conversational agents (2025) · cites this paper as [21]
GPT-4 passes an interactive two-player Turing Test and was judged to be human 54% of the time.
“GPT -4 passes an interactive two -player Turing Test and was judged to be human 54% of the time (Jones & Bergen, 2024).”
From Authenticity and Authorship Beyond the Algorithm: Action, Institution, and Epistemic Responsibility under Generative AI (2026) · cites this paper as Jones & Bergen, 2024
In many contexts, LLM responses are indistinguishable from human outputs, which makes behavioural performance an appealing indicator of understanding.
“In many contexts, their responses are indistinguishable from human outputs, which makes behavioural performance an appealing indicator of understanding (Jones & Bergen, 2024).”
From Authenticity and Authorship Beyond the Algorithm: Action, Institution, and Epistemic Responsibility under Generative AI (2026) · cites this paper as Jones & Bergen, 2024
Citations
Cited by
Sorted by citations
The benefits and dangers of anthropomorphic conversational agents
Peter, S., Riemer, K., & West, J. D. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2415898122 · Cited by 110
Jones, C. R., & Bergen, B. K. (2024) · arXiv · doi:10.48550/arxiv.2412.17128 · Cited by 7
Self-citationFrom Imitation to Introspection: Probing Self-Consciousness in Language Models
Chen, S., Yu, S., Zhao, S., & Lu, C. (2025) · Findings of the Association for Computational Linguistics: ACL 2025 · doi:10.18653/v1/2025.findings-acl.392 · Cited by 4
Kunwar, J. B. (2026) · Discourses on Culture · doi:10.2478/doc-2026-0006
Metadata
- Authors
- Jones, C. R., & Bergen, B. K.
- Affiliations
- University of California San Diego (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- Pages
- 1615-1639
- arXiv
- 2405.08007
- OpenAlex
- W4411541869
- Cited by
- 76
- References
- 0
- Citations by year
- 2026: 38 · 2025: 32 · 2024: 6
- FWCI
- 66.38
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Open Philanthropy Project
- University of California, San Diego
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- Turing test, GPT-4, GPT-4o, Eliza, GPT-3.5, human-AI interaction, AI deception, conversational AI, machine intelligence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
