CoChat
Sign Up
PreprintCited by 9Digital Media Forensic DetectionFace recognition and analysisGenerative Adversarial Networks and Image Synthesis

UniForensics: Face Forgery Detection via General Facial Representation

Open access
Fang, Ziyuan, Zhao, Hanqing, Wei, Tianyi, Zhou, Wenbo, Wan, Ming, Wang, Zhanyi, Zhang, Weiming, Yu, Nenghai·2024·arXiv
DOI 10.48550/arxiv.2407.19079arXiv 2407.19079
Open this paper in CoChat — free accountOpen PDF

Abstract

Previous deepfake detection methods mostly depend on low-level textural features vulnerable to perturbations and fall short of detecting unseen forgery methods. In contrast, high-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. Motivated by this, we propose a detection method that utilizes high-level semantic features of faces to identify inconsistencies in temporal domain. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video classification network, initialized with a meta-functional face encoder for enriched facial representation. In this way, we can take advantage of both the powerful spatio-temporal model and the high-level semantic information of faces. Furthermore, to leverage easily accessible real face data and guide the model in focusing on spatio-temporal features, we design a Dynamic Video Self-Blending (DVSB) method to efficiently generate training samples with diverse spatio-temporal forgery traces using real facial videos. Based on this, we advance our framework with a two-stage training approach: The first stage employs a novel self-supervised contrastive learning, where we encourage the network to focus on forgery traces by impelling videos generated by the same forgery process to have similar representations. On the basis of the representation learned in the first stage, the second stage involves fine-tuning on face forgery detection dataset to build a deepfake detector. Extensive experiments validates that UniForensics outperforms existing face forgery methods in generalization ability and robustness. In particular, our method achieves 95.3\% and 77.2\% cross dataset AUC on the challenging Celeb-DFv2 and DFDC respectively.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2407.19079

Cites

43 of 43 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Transformer-based architectures incorporating self-supervised and contrastive learning paradigms have demonstrated superior capabilities in capturing fine-grained inconsistencies across both spatial and temporal domains.

    “Additionally, transformer-based architectures incorporating self-supervised and contrastive learning paradigms, such as UniForensics [71], FakeSTormer [72], and CNN-Transformer [73], have further demonstrated superior capabilities in capturing fine-grained inconsistencies across both spatial and temporal domains.”

    From Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges (2024) · cites this paper as [71]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Fang, Z., Zhao, H., Wei, T., Zhou, W., Wan, M., Wang, Z., Zhang, W., & Yu, N.
Affiliations
University of Science and Technology of China (CN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
Volume
23
Issue
2
Pages
3176-3190
ISSN
1545-5971 (ISSN-L)
DOI
10.48550/arxiv.2407.19079
arXiv
2407.19079
OpenAlex
W4416286276
Cited by
8
References
43
Citations by year
2026: 6 · 2025: 2
FWCI
2.78
Type
journal article
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • IEEE Transactions on Dependable and Secure Computing (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Natural Science Foundation of China (62121002, 62072421, 62372423)
  • Fundamental Research Funds for the Central Universities (WK2100250070)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Keywords
face forgery detection, deepfake detection, high-level semantic features, time inconsistency, synthetic training data generation, self-supervised contrastive learning, cross-dataset generalization

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review