CoChat

Human Performance in Deepfake Detection: A Systematic Review

Open access
Somoray, Klaire, Miller, Dan J., Holmes, Mary · 2025 · Human Behavior and Emerging Technologies

The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index; 79 of the 153 sources its claims cite were checked for support.

Abstract

Deepfakes refer to a wide range of computer‐generated synthetic media, in which a person’s appearance or likeness is altered to resemble that of another. This systematic review is aimed at providing an overview of the existing research into people’s ability to detect deepfakes. Five databases (IEEE, ProQuest, PubMed, Web of Science, and Scopus) were searched up to December 2023. Studies were included if they (1) were an original study; (2) were reported in English; (3) examined people’s detection of deepfakes; (4) examined the influence of an intervention, strategy, or variable on deepfake detection; and (5) reported relevant data needed to evaluate detection accuracy. Forty independent studies from 30 unique records were included in the review. Results were narratively summarized, with key findings organized based on the review’s research questions. Studies used different performance measures, making it difficult to compare results across the literature. Detection accuracy varies widely, with some studies showing humans outperforming AI models and others indicating the opposite. Detection performance is also influenced by person‐level (e.g., cognitive ability, analytical thinking) and stimuli‐level factors (e.g., quality of deepfake, familiarity with the subject). Interventions to improve people’s deepfake detection yielded mixed results. Humans and AI‐based detection models focus on different aspects when detecting, suggesting a potential for human–AI collaboration. The findings highlight the complex interplay of factors influencing human deepfake detection and the need for further research to develop effective strategies for deepfake detection.

Overview

CoChat Summary

TL;DR

This systematic review synthesizes 40 independent studies from 30 records on human deepfake detection, finding widely variable accuracy and mixed effects of person-level, stimuli-level, and intervention factors. Humans sometimes outperform AI models and sometimes underperform them, and most tested interventions were ineffective except interactive/media literacy training and AI prediction feedback.

Key findings

  • Across 30 records and 40 independent studies, sample sizes ranged from 19 to 11,088; direct detection accuracy averaged 57.6% to 75.43%, while indirect detection rates ranged from 0% to 83.5%. Metrics varied and included forced choices, Likert scales, open-ended questions, AUC, F1 score, precision, true positive rates, false positive rates, and false negative rates.
  • Person-level factors: cognitive ability and analytical thinking were positively associated with detection accuracy; truth bias was associated with miscategorizing deepfakes as real; gender, education, income, and profession had no effect; age and ethnicity showed mixed results. Stimuli-level factors: familiarity with the deepfaked subject improved detection; higher-quality deepfakes were harder to detect; face swaps were easier to detect than morphs; lip-syncs were harder to detect than face-swapped videos; audio deepfakes were poorly detected.
  • Human versus AI performance was mixed: Groh et al. found 82% of participants who saw at least 10 videos outperformed the winning DFDC model, and the crowd mean identified 74%–80% of videos versus the model’s 80%; Korshunov and Marcel found humans achieved average AUC 87.47% versus EfficientNet-trained-on-Google at 73.97%. In contrast, Chen et al. found the best machine learning model achieved AUC 0.985 versus human AUC 0.870, and Salini and HariKiran found approximately 70% of participants failed to detect fake videos while algorithms achieved over 90% accuracy.
  • Interventions: raising awareness about deepfakes and financial incentives had no effect; warnings had mixed results; written detection tips did not improve accuracy; media literacy training and interactive training with example videos and/or feedback improved detection; providing AI predictions improved human detection performance.

Methods

  • PRISMA systematic review registered on OSF; five databases (IEEE, ProQuest, PubMed, Web of Science, and Scopus) were searched in December 2022, with an updated search in December 2023; no time limits were imposed; unpublished works and dissertations were included; articles in languages other than English were excluded.
  • Inclusion criteria: original study, reported in English, examined people’s detection of deepfakes, examined the influence of an intervention, strategy, or variable on deepfake detection, and reported relevant data needed to evaluate detection accuracy. Exclusions included AI-only detection studies, general media literacy without specific deepfake detection tasks, lack of criteria to assess detection accuracy, and synthetic stimuli not mimicking an existing person or retouching only. Thirty records with 40 independent studies were included.
  • Narrative synthesis was conducted because substantial variability in design and outcomes made meta-analysis unsuitable; JBI Critical Appraisal tools were used for quasi-experimental, analytical cross-sectional, and qualitative studies.

Limitations

  • Stated in the paper: the review focused on deepfakes imitating an existing person, excluding face retouching and face or voice synthesis, which limits generalizability; rapid evolution of deepfake technology may limit applicability to newer techniques; there is little research on audio-based deepfakes; the English-language restriction limits global representation and may overlook cultural and societal factors; most included studies were cross-sectional or experimental, so longitudinal conclusions about intervention effectiveness or changing detection abilities cannot be drawn.
  • Evident from the text: performance metrics varied widely across studies, making comparison of human detection abilities difficult; risk-of-bias assessment found no experimental studies had follow-up, only two had pre-post measurements, and six lacked a control group or control condition; cross-sectional reporting issues included only one study having clear inclusion criteria, only half clearly describing subjects and research settings, and two not clearly identifying or controlling potential confounding factors.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1155/hbe2/1833228

Cites

60 of 60 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

55 Valid · 7 Not valid · 38 Not checked · 81 Own results and interpretations

82 more claims · 33 with their cited sources checked

Create a free account to see all 181 claims and to verify the 19 references still open.

See all claims — free account
  • Probably not a substantial claim. 64
  • It may state two separate claims as one. 48
  • Borderline: one of the tests for a substantial claim is unclear. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Research works printed but not linked: 2

  1. Wöhler L., 2020, Computer animation and social agents, 120
  2. Ciftci U. A., 2020, Proceedings of the IEEE Transactions on Pattern Analysis and Machine Intelligence

Cited by

See all 19

Metadata

Authors
Somoray, K., Miller, D. J., & Holmes, M.
Affiliations
James Cook University (AU)
Corresponding author
Somoray, Klaire
Venue
Human Behavior and Emerging Technologies
Publisher
Wiley
Published
2025-01
Volume
2025
Issue
1
ISSN
2578-1863 (ISSN-L)
OpenAlex
W4412881967
Cited by
19
References
41
Citations by year
2026: 17 · 2025: 2
FWCI
2.01
Type
journal article
Language
en
Open access
Licence
Locations
  • Human Behavior and Emerging Technologies (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Social Sciences › Psychology › Social Psychology › Deception detection and forensic psychology
Keywords
deepfake detection, synthetic media, human-AI comparison, detection accuracy, cognitive ability, analytical thinking, familiarity, human-AI collaboration

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.