CoChat

DFDT: An End-to-End DeepFake Detection Framework Using Vision Transformer

Open access
Khormali, Aminollah, Yuan, Jiann-Shiun · 2022 · Applied Sciences

The DOI matches its Crossref record; 94 of the 94 references it cites were found in a registry or an index; 19 of the 25 sources its claims cite were checked for support.

Abstract

The ever-growing threat of deepfakes and large-scale societal implications has propelled the development of deepfake forensics to ascertain the trustworthiness of digital media. A common theme of existing detection methods is using Convolutional Neural Networks (CNNs) as a backbone. While CNNs have demonstrated decent performance on learning local discriminative information, they fail to learn relative spatial features and lose important information due to constrained receptive fields. Motivated by the aforementioned challenges, this work presents DFDT, an end-to-end deepfake detection framework that leverages the unique characteristics of transformer models, for learning hidden traces of perturbations from both local image features and global relationship of pixels at different forgery scales. DFDT is specifically designed for deepfake detection tasks consisting of four main components: patch extraction & embedding, multi-stream transformer block, attention-based patch selection followed by a multi-scale classifier. DFDT’s transformer layer benefits from a re-attention mechanism instead of a traditional multi-head self-attention layer. To evaluate the performance of DFDT, a comprehensive set of experiments are conducted on several deepfake forensics benchmarks. Obtained results demonstrated the surpassing detection rate of DFDT, achieving 99.41%, 99.31%, and 81.35% on FaceForensics++, Celeb-DF (V2), and WildDeepfake, respectively. Moreover, DFDT’s excellent cross-dataset & cross-manipulation generalization provides additional strong evidence on its effectiveness.

Overview

CoChat Summary

TL;DR

DFDT is an end-to-end vision-transformer deepfake detector combining patch/position embeddings, multi-stream transformer blocks, attention-based patch selection, and a multi-scale classifier. It reports state-of-the-art intra-dataset accuracy and strong cross-dataset/cross-manipulation generalization on several benchmarks.

Key findings

  • Frame-level intra-dataset results: 99.41% ACC / 99.94% AUC on FaceForensics++ (raw), 99.31% ACC / 99.26% AUC on Celeb-DF (V2), and 81.35% ACC / 80.74% AUC on WildDeepfake.
  • Video-level cross-dataset generalization when trained on FaceForensics++: AUC 88.3% on Celeb-DF, 76.1% on DFDC, 97.8% on FaceShifter, 96.9% on DeeperForensics, average 89.7%; LipForensics is reported at 82.4%, 73.5%, 97.1%, 97.6%, average 87.8%.
  • Cross-manipulation generalization on FaceForensics++ subsets: DFDT average AUC 97.9% (Deepfakes 99.8%, FaceSwap 93.1%, Face2Face 99.6%, NeuralTextures 99.2%), compared with LipForensics average 97.1%.
  • Ablation: removing the re-attention mechanism decreases AUC by 1.7% on FaceForensics++, 0.9% on Celeb-DF (V2), and 1.4% on WildDeepfake.

Methods

  • DFDT consists of patch extraction & embedding, a multi-stream transformer block, attention-based patch selection, and a multi-scale classifier; its transformer layer uses a re-attention mechanism instead of traditional multi-head self-attention, with low-level and high-level branches using different patch sizes.
  • Preprocessing extracts 20% of frames per video, uses RetinaFace for facial landmarks, crops/resizes/aligns the face area, and applies overlapping patch extraction with positional embeddings.
  • Evaluation uses FaceForensics++, Celeb-DF (V2), and WildDeepfake for intra-dataset tests; DFDC, FaceShifter, and DeeperForensics for cross-dataset tests; 80% of videos are used for training, with the rest equally divided into test and validation. Metrics are ACC and AUC; implementation uses PyTorch, Adam with learning rate 10^-4 and tenfold decay every 40 steps, trained for 100 epochs.

Limitations

  • The paper does not include a dedicated limitations section; it states that performance on more challenging real-world datasets like WildDeepfake is “still far from perfect” and reports lower WildDeepfake scores (81.35% ACC, 80.74% AUC) than on FaceForensics++/Celeb-DF.
  • Evident from the reported results: cross-dataset DeeperForensics AUC is 96.9%, slightly below LipForensics’ 97.6%, so DFDT is competitive rather than superior there.
  • Evident from the text: no explicit discussion of computational cost, inference speed, or robustness beyond the tested benchmarks; the paper reports hardware setup but not efficiency comparisons.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/app12062953

Cites

94 of 94 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

13 Valid · 3 Not valid · 0 Not checked · 19 Own results and interpretations

13 more claims · 2 with their cited sources checked

Create a free account to see all 35 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 7
  • It may state two separate claims as one. 9
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Cited by

See all 88

Metadata

Authors
Khormali, A., & Yuan, J.-S.
Affiliations
University of Central Florida (US)
Corresponding author
Yuan, Jiann-Shiun
Venue
Applied Sciences
Publisher
MDPI AG
Published
2022-03-14
Volume
12
Issue
6
Pages
2953
ISSN
2076-3417 (ISSN-L)
OpenAlex
W4221045030
Cited by
88
References
75
Citations by year
2026: 20 · 2025: 31 · 2024: 22 · 2023: 13 · 2022: 2
FWCI
5.56
Type
journal article
Language
en
Open access
Locations
  • Applied Sciences (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
Keywords
deepfake detection, deepfake forensics, vision transformer, multi-scale classification, patch embedding, cross-dataset generalization, FaceForensics++, Celeb-DF

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.