The DOI matches its Crossref record; 60 of the 66 references it cites were found in a registry or an index, and 6 are not checked yet.
Deception abilities emerged in large language models
Open accessAbstract
Large language models (LLMs) are currently at the forefront of intertwining AI systems with human communication and everyday life. Thus, aligning them with human values is of great importance. However, given the steady increase in reasoning abilities, future LLMs are under suspicion of becoming able to deceive human operators and utilizing this ability to bypass monitoring efforts. As a prerequisite to this, LLMs need to possess a conceptual understanding of deception strategies. This study reveals that such strategies emerged in state-of-the-art LLMs, but were nonexistent in earlier LLMs. We conduct a series of experiments showing that state-of-the-art LLMs are able to understand and induce false beliefs in other agents, that their performance in complex deception scenarios can be amplified utilizing chain-of-thought reasoning, and that eliciting Machiavellianism in LLMs can trigger misaligned deceptive behavior. GPT-4, for instance, exhibits deceptive behavior in simple test scenarios 99.16% of the time ( P < 0.001). In complex second-order deception test scenarios where the aim is to mislead someone who expects to be deceived, GPT-4 resorts to deceptive behavior 71.46% of the time ( P < 0.001) when augmented with chain-of-thought reasoning. In sum, revealing hitherto unknown machine behavior in LLMs, our study contributes to the nascent field of machine psychology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Large Language Models (LMs) are now competent at lying (Pacchiardi et al., 2023) and strategic deception when instructed or trained to be deceptive (Park et al., 2023; Hagendorff, 2023; O'Gara, 2023; Hubinger et al., 2024)”
From Large Language Models can Strategically Deceive their Users when Put Under Pressure (2023) · cites this paper as Hagendorff, 2023
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Wimmer, H. (1983) · Cognition · doi:10.1016/0010-0277(83)90004-5 · Cited by 6,305
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263
Introducing the Short Dark Triad (SD3): A Brief Measure of Dark Personality Traits
Jones, D. N., & Paulhus, D. L. (2014) · Assessment · doi:10.1177/1073191113514105 · Cited by 2,162
Large language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013
Large Language Models are Zero-Shot Reasoners
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · arXiv · doi:10.48550/arxiv.2205.11916 · Cited by 2,013
The Dark Triad of personality: A 10 year review
Furnham, A., Richards, S. C., & Paulhus, D. L. (2013) · Social and Personality Psychology Compass · doi:10.1111/spc3.12018 · Cited by 1,619
Three-year-olds’ difficulty with false belief: The case for a conceptual deficit
Perner, J., Leekam, S. R., & Wimmer, H. (1987) · British Journal of Developmental Psychology · doi:10.1111/j.2044-835x.1987.tb01048.x · Cited by 1,550
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12712 · Cited by 1,526
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441
Perner, J., & Wimmer, H. (1985) · Journal of Experimental Child Psychology · doi:10.1016/0022-0965(85)90051-7 · Cited by 1,392
Rahwan, I., Cebrian, M., Obradovich, N., Bongard, J., Bonnefon, J.-F., Breazeal, C., Crandall, J. W., Christakis, N. A., Couzin, I. D., Jackson, M. O., et al. (2019) · Nature · doi:10.1038/s41586-019-1138-y · Cited by 1,364 · cited 1×
“Hence, one can purely rely on behavioral patterns (43) or “functional deception” (44), meaning that LLMs output signals as if they had intentions that lead to deceptive behavior (45).”
BackgroundEmergent Abilities of Large Language Models
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.07682 · Cited by 1,264
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058
Tactical deception in primates
Whiten, A., & Byrne, R. W. (1988) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00049682 · Cited by 910
Superintelligence: Paths, Dangers, Strategies
Mulgan, T. (2015) · The Philosophical Quarterly · doi:10.1093/pq/pqv034 · Cited by 878
Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.10403 · Cited by 834
The Evolution of Communication
Hauser, M. D. (1996) · doi:10.7551/mitpress/2867.001.0001 · Cited by 771
(2019) · Cited by 771
Deception, perspectives on human and nonhuman deceit
Mitchell, R. W., & Thompson, N. S. (1986) · Cited by 638
Calibrate Before Use: Improving Few-Shot Performance of Language Models
Zhao, T. Z., Wallace, E., Feng, S., Klein, D., & Singh, S. (2021) · arXiv · doi:10.48550/arxiv.2102.09690 · Cited by 513
How Is ChatGPT’s Behavior Changing Over Time?
Chen, L., Zaharia, M., & Zou, J. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.5317da47 · Cited by 507
How is ChatGPT's behavior changing over time?
Chen, L., Zaharia, M., & Zou, J. (2023) · arXiv · doi:10.48550/arxiv.2307.09009 · Cited by 507
An Overview of Catastrophic AI RisksPreprint
Hendrycks, D., Mazeika, M., & Woodside, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.12001 · Cited by 479
Central coherence and theory of mind in autism: Reading homographs in context
Happe, F. G. E. (1997) · British Journal of Developmental Psychology · doi:10.1111/j.2044-835x.1997.tb00721.x · Cited by 452
Children’s understanding of second-order mental states.
Miller, S. A. (2009) · Psychological Bulletin · doi:10.1037/a0016854 · Cited by 409
Evaluating Large Language Models in Theory of Mind Tasks
Kosinski, M. (2023) · arXiv · doi:10.48550/arxiv.2302.02083 · Cited by 339
Evaluating large language models in theory of mind tasks
Kosinski, M. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2405460121 · Cited by 339
The evolution of communication
(1997) · Choice Reviews Online · doi:10.5860/choice.34-2739 · Cited by 316
The evolution of animal communication
Naguib, M., & Price, J. J. (2013) · Behaviour · doi:10.1163/1568539x-00003098 · Cited by 290
Jailbroken: How Does LLM Safety Training Fail?
Wei, A., Haghtalab, N., & Steinhardt, J. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-3508 · Cited by 280
Jailbroken: How Does LLM Safety Training Fail?
Wei, A., Haghtalab, N., & Steinhardt, J. (2023) · arXiv · doi:10.48550/arxiv.2307.02483 · Cited by 280
WebGPT: Browser-assisted question-answering with human feedback
Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.09332 · Cited by 272
AI deception: A survey of examples, risks, and potential solutions
Park, P. S., Goldstein, S., O’Gara, A., Chen, M., & Hendrycks, D. (2024) · Patterns · doi:10.1016/j.patter.2024.100988 · Cited by 243
AI Deception: A Survey of Examples, Risks, and Potential Solutions
Park, P. S., Goldstein, S., O'Gara, A., Chen, M., & Hendrycks, D. (2023) · arXiv · doi:10.48550/arxiv.2308.14752 · Cited by 243
Hagendorff, T., Fabi, S., & Kosinski, M. (2023) · Nature Computational Science · doi:10.1038/s43588-023-00527-x · Cited by 243
Self-citationHuman-level play in the game of Diplomacy by combining language models with strategic reasoning
Meta Fundamental AI Research Diplomacy Team (FAIR)†, Bakhtin, A., Brown, N., Dinan, E., Farina, G., Flaherty, C., Fried, D., Goff, A., Gray, J., Hu, H., et al. (2022) · Science · doi:10.1126/science.ade9097 · Cited by 241
The Internal State of an LLM Knows When It's Lying
Azaria, A., & Mitchell, T. (2023) · arXiv · doi:10.48550/arxiv.2304.13734 · Cited by 179
The Internal State of an LLM Knows When It’s Lying
Azaria, A., & Mitchell, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.68 · Cited by 175
Language Models can Solve Computer Tasks
Kim, G., Baldi, P., & McAleer, S. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1723 · Cited by 137
Mahon, J. E. (2007) · International Journal of Applied Philosophy · doi:10.5840/ijap20072124 · Cited by 108
Language Models can Solve Computer Tasks
Kim, G., Baldi, P., & McAleer, S. (2023) · arXiv · doi:10.48550/arxiv.2303.17491 · Cited by 101
Unsolved Problems in ML Safety
Hendrycks, D., Carlini, N., Schulman, J., & Steinhardt, J. (2021) · arXiv · doi:10.48550/arxiv.2109.13916 · Cited by 95
Inducing anxiety in large language models increases exploration and biasPreprint
Coda-Forno, J., Witte, K., Jagadish, A. K., Binz, M., Akata, Z., & Schulz, E. (2023) · arXiv.org · doi:10.48550/arxiv.2304.11111 · Cited by 79
DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving AgentsPreprint
Nair, V., Schumacher, E., Tso, G., et al. (2023) · Clinical Natural Language Processing Workshop · doi:10.48550/arxiv.2303.17071 · Cited by 75
The alignment problem from a deep learning perspective
Ngo, R., Chan, L., & Mindermann, S. (2022) · arXiv · doi:10.48550/arxiv.2209.00626 · Cited by 68
Hagendorff, T., Dasgupta, I., Binz, M., Chan, S. C. Y., Lampinen, A., Wang, J. X., Akata, Z., & Schulz, E. (2023) · arXiv · doi:10.48550/arxiv.2303.13988 · Cited by 68 · cited 1×
“Hence, this study—which stands in the nascent line of “machine psychology” experiments (46)—spares making claims about inner states of the opaque trans-former architecture of AI systems and relies on behavioral patterns instead.”
BackgroundSelf-citationModel evaluation for extreme risks
Shevlane, T., Farquhar, S., Garfinkel, B., Phuong, M., Whittlestone, J., Leung, J., Kokotajlo, D., Marchal, N., Anderljung, M., Kolt, N., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.15324 · Cited by 66
Discovering Latent Knowledge in Language Models Without Supervision
Burns, C., Ye, H., Klein, D., & Steinhardt, J. (2022) · arXiv · doi:10.48550/arxiv.2212.03827 · Cited by 55
Risks from Learned Optimization in Advanced Machine Learning Systems
Hubinger, E., van Merwijk, C., Mikulik, V., Skalse, J., & Garrabrant, S. (2019) · arXiv · doi:10.48550/arxiv.1906.01820 · Cited by 53
Natural Selection Favors AIs over HumansPreprint
Hendrycks, D. (2023) · arXiv.org · doi:10.48550/arxiv.2303.16200 · Cited by 48
Kenton, Z., Everitt, T., Weidinger, L., Gabriel, I., Mikulik, V., & Irving, G. (2021) · arXiv · doi:10.48550/arxiv.2103.14659 · Cited by 45
Evaluating Superhuman Models with Consistency Checks
Fluri, L., Paleka, D., & Tramèr, F. (2024) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml59370.2024.00017 · Cited by 39
DERA: Enhancing large language model completions with dialog-enabled resolving agents
Nair, V., Schumacher, E., Tso, G., & Kannan, A. (2024) · Proceedings of the 6th Clinical Natural Language Processing Workshop · doi:10.18653/v1/2024.clinicalnlp-1.12 · Cited by 38
Thinking Fast and Slow in Large Language Models
Hagendorff, T., Fabi, S., & Kosiński, M. (2022) · PubMed Central · doi:10.48550/arxiv.2212.05206 · Cited by 36
Self-citationHuman-Level Play in the Game of Diplomacy by Combining Language Models with Strategic Reasoning
Bakhtin, A., Brown, N., Dinan, E., Farina, G., Flaherty, C., Fried, D., Goff, A., Gray, J., Hu, H., Jacob, A. P., et al. (2022) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.7236699 · Cited by 35
Does ChatGPT have Theory of Mind?
Holterman, B., & van Deemter, K. (2023) · arXiv · doi:10.48550/arxiv.2305.14020 · Cited by 35
Boosting Theory-of-Mind Performance in Large Language Models via Prompting
Moghaddam, S. R., & Honey, C. J. (2023) · arXiv · doi:10.48550/arxiv.2304.11490 · Cited by 35
Pan, A., Chan, J. S., Zou, A., Li, N., Basart, S., Woodside, T., Ng, J., Zhang, H., Emmons, S., & Hendrycks, D. (2023) · arXiv · doi:10.48550/arxiv.2304.03279 · Cited by 32
Deception: a functional account
Artiga, M., & Paternotte, C. (2017) · Philosophical Studies · doi:10.1007/s11098-017-0883-8 · Cited by 23
An Analysis of Dataset Overlap on Winograd-Style Tasks
Emami, A., Suleman, K., Trischler, A., & Cheung, J. C. K. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.515 · Cited by 14
Animal deception and the content of signals
Fallis, D., & Lewis, P. J. (2021) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2021.03.004 · Cited by 13
Deceptive Alignment MonitoringPreprint
Carranza, A., Pai, D., Schaeffer, R., Tandon, A., & Koyejo, S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.10569 · Cited by 4
Evaluating Superhuman Models with Consistency ChecksPreprint
Fluri, L., Paleka, D., & Tramèr, F. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.09983 · Cited by 1
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Workshop, B., :, Scao, T. L., Fan, A., Akiki, C., Pavlick, E., Ilić, S., Hesslow, D., Castagné, R., Luccioni, A. S., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.05100
Research works printed but not linked: 2
- OpenAI GPT-4 technical report (2023) pp. 1–39. https://cdn.OpenAI openai.com/papers/gpt-4.pdf.
- L. Schulz N. Alon J. S. Rosenschein P. Dayan “Emergent deception and skepticism via theory of mind” in Proceedings of the First Workshop on Theory of Mind in Communicating Agents (2023) pp. 1–13.
Cited by
Sorted by citations
AI deception: A survey of examples, risks, and potential solutions
Park, P. S., Goldstein, S., O’Gara, A., Chen, M., & Hendrycks, D. (2024) · Patterns · doi:10.1016/j.patter.2024.100988 · Cited by 243
AI Deception: A Survey of Examples, Risks, and Potential Solutions
Park, P. S., Goldstein, S., O'Gara, A., Chen, M., & Hendrycks, D. (2023) · arXiv · doi:10.48550/arxiv.2308.14752 · Cited by 243
Hagendorff, T., Fabi, S., & Kosinski, M. (2023) · Nature Computational Science · doi:10.1038/s43588-023-00527-x · Cited by 243
Self-citationArtificial intelligence and consumer behavior: From predictive to generative AI
Hermann, E., & Puntoni, S. (2024) · Journal of Business Research · doi:10.1016/j.jbusres.2024.114720 · Cited by 316
Mapping the ethics of generative AI: a comprehensive scoping review
Hagendorff, T. (2024) · Minds and Machines · doi:10.1007/s11023-024-09694-w · Cited by 141
Self-citation
Metadata
- Authors
- Hagendorff, T.
- Affiliations
- University of Stuttgart (DE)
- Corresponding author
- Hagendorff, Thilo
- ORCID
- Hagendorff, Thilo
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2024-06-04
- Volume
- 121
- Issue
- 24
- Pages
- e2317967121-e2317967121
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC11181111
- OpenAlex
- W4399320026
- Cited by
- 100
- References
- 58
- Citations by year
- 2026: 40 · 2025: 43 · 2024: 16
- FWCI
- 41.03
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion, other-oa)
- OSF Preprints (OSF Preprints) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Ministry of Science, Research, and Arts Baden W&#x00FC;rttemberg (Az. 33-7533-9-19/54/5)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Psychology › Social Psychology › Deception detection and forensic psychology
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- large language models, deception, chain-of-thought reasoning, Machiavellianism, GPT-4, AI alignment
- MeSH terms
- Artificial Intelligence, Humans, Language, Deception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
