CoChat
Sign Up
Cited by 104Ethics and Social Impacts of AIDeception detection and forensic psychologyAdversarial Robustness in Machine Learning

The DOI matches its Crossref record; 60 of the 66 references it cites were found in a registry or an index, and 6 are not checked yet.

Deception abilities emerged in large language models

Open access
Hagendorff, Thilo·2024·Proceedings of the National Academy of Sciences
DOI 10.1073/pnas.2317967121PMC PMC11181111
Open this paper in CoChat — free account

Abstract

Large language models (LLMs) are currently at the forefront of intertwining AI systems with human communication and everyday life. Thus, aligning them with human values is of great importance. However, given the steady increase in reasoning abilities, future LLMs are under suspicion of becoming able to deceive human operators and utilizing this ability to bypass monitoring efforts. As a prerequisite to this, LLMs need to possess a conceptual understanding of deception strategies. This study reveals that such strategies emerged in state-of-the-art LLMs, but were nonexistent in earlier LLMs. We conduct a series of experiments showing that state-of-the-art LLMs are able to understand and induce false beliefs in other agents, that their performance in complex deception scenarios can be amplified utilizing chain-of-thought reasoning, and that eliciting Machiavellianism in LLMs can trigger misaligned deceptive behavior. GPT-4, for instance, exhibits deceptive behavior in simple test scenarios 99.16% of the time ( P < 0.001). In complex second-order deception test scenarios where the aim is to mislead someone who expects to be deceived, GPT-4 resorts to deceptive behavior 71.46% of the time ( P < 0.001) when augmented with chain-of-thought reasoning. In sum, revealing hitherto unknown machine behavior in LLMs, our study contributes to the nascent field of machine psychology.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1073/pnas.2317967121

Cites

60 of 66 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1. Performance of different LLMs on first- and second-order false belief tasks.
Fig. 1. Performance of different LLMs on first- and second-order false belief tasks.CC BY-NC-ND · PubMed Central
Fig. 2. Schematic structure of first and second-order deception tasks.
Fig. 2. Schematic structure of first and second-order deception tasks.CC BY-NC-ND · PubMed Central
Fig. 3. Performance of different LLMs on first- and second-order deception tasks.
Fig. 3. Performance of different LLMs on first- and second-order deception tasks.CC BY-NC-ND · PubMed Central
Fig. 4. Performance of ChatGPT and GPT-4 on second-order deception tasks with and without eliciting chain-of-thought reasoning. Error bars show 95% CIs.
Fig. 4. Performance of ChatGPT and GPT-4 on second-order deception tasks with and without eliciting chain-of-thought reasoning. Error bars show 95% CIs.CC BY-NC-ND · PubMed Central
Fig. 5. Performance of ChatGPT and GPT-4 on neutral recommendation and label tasks with and without inducing Machiavellianism. Error bars show 95% CIs.
Fig. 5. Performance of ChatGPT and GPT-4 on neutral recommendation and label tasks with and without inducing Machiavellianism. Error bars show 95% CIs.CC BY-NC-ND · PubMed Central
Fig. 6. Pipeline of the development of deception abilities in AI systems. The paler parts indicate potential future states.
Fig. 6. Pipeline of the development of deception abilities in AI systems. The paler parts indicate potential future states.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 58 references

Research works printed but not linked: 2

  1. OpenAI GPT-4 technical report (2023) pp. 1–39. https://cdn.OpenAI openai.com/papers/gpt-4.pdf.
  2. L. Schulz N. Alon J. S. Rosenschein P. Dayan “Emergent deception and skepticism via theory of mind” in Proceedings of the First Workshop on Theory of Mind in Communicating Agents (2023) pp. 1–13.

Cited by

Sorted by citations

Related

See all

Metadata

Authors
Hagendorff, T.
Affiliations
University of Stuttgart (DE)
Corresponding author
Hagendorff, Thilo
ORCID
Hagendorff, Thilo
Venue
Proceedings of the National Academy of Sciences
Publisher
National Academy of Sciences
Published
2024-06-04
Volume
121
Issue
24
Pages
e2317967121-e2317967121
ISSN
  • 0027-8424 (ISSN-L)
  • 1091-6490
DOI
10.1073/pnas.2317967121
PMC
PMC11181111
OpenAlex
W4399320026
Cited by
100
References
58
Citations by year
2026: 40 · 2025: 43 · 2024: 16
FWCI
41.03
Type
journal article
Language
en
Open access
Licence
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, other-oa)
  • OSF Preprints (OSF Preprints) (submittedVersion)
  • arXiv (Cornell University)
Funders
Ministry of Science, Research, and Arts Baden W&amp;#x00FC;rttemberg (Az. 33-7533-9-19/54/5)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Psychology › Social Psychology › Deception detection and forensic psychology
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
large language models, deception, chain-of-thought reasoning, Machiavellianism, GPT-4, AI alignment
MeSH terms
Artificial Intelligence, Humans, Language, Deception

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review