CoChat

Learning Features of Intra-Consistency and Inter-Diversity: Keys Toward Generalizable Deepfake Detection

Chen, Han, Lin, Yuzhen, Li, Bin, Tan, Shunquan · 2023 · IEEE Transactions on Circuits and Systems for Video Technology
Cited by 80

The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.

Abstract

Public concerns about deepfake face forgery are continually rising in recent years. Most deepfake detection approaches attempt to learn discriminative features between real and fake faces through end-to-end trained deep neural networks. However, the majorities of them suffer from the problem of poor generalization across different data sources, forgery methods, and/or post-processing operations. In this paper, following the simple but effective principle in discriminative representation learning, i.e., towards learning features of intra-consistency within classes and inter-diversity between classes, we leverage a novel transformer-based self-supervised learning method and an effective data augmentation strategy towards generalizable deepfake detection. Considering the differences between the real and fake images are often subtle and local, the proposed method firstly utilizes Self Prediction Learning (SPL) to learn rich hidden representations by predicting masked patches at a pre-training stage. Intra-class consistency clues in images can be mined without deepfake labels. After pre-training, the discrimination model is then fine-tuned via multi-task learning, including a deepfake classification task and a forgery mask estimation task. It is facilitated by our new data augmentation method called Adjustable Forgery Synthesizer (AFS), which can conveniently simulate the process of synthesizing deepfake images with various levels of visual reality in an explicit manner. AFS greatly prevents overfitting due to insufficient diversity in training data. Comprehensive experiments demonstrate that our method outperforms the state-of-the-art competitors on several popular benchmark datasets in terms of generalization to unseen forgery methods and untrained datasets.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tcsvt.2022.3209336

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 34 references

Research works printed but not linked: 21

  1. 10.1109/TCSVT.2021.3106047 · DOI
  2. 10.1109/TCSVT.2021.3089724 · DOI
  3. 10.1109/TCSVT.2021.3133859 · DOI · Paper page
  4. Chen, A simple framework for contrastive learning of visual representations, 1597
  5. Zhou, 2022, Self pre-training with masked autoencoders for medical image analysis, arXiv:2203.05573 · arXiv
  6. Vaswani, Attention is all you need, 30, 5998
  7. Dosovitskiy, An image is worth 16×16 words: Transformers for image recognition at scale, 1
  8. 10.1145/3474085.3475535 · DOI · Paper page
  9. 10.1109/TIFS.2022.3146781 · DOI · Paper page
  10. 10.1609/aaai.v36i1.19954 · DOI · Paper page
  11. 10.1109/TCSVT.2021.3074259 · DOI · Paper page
  12. 10.1109/CVPR52688.2022.00925 · DOI · Paper page
  13. 10.1109/CVPR42600.2020.00975 · DOI · Paper page
  14. 10.1109/CVPR52688.2022.00943 · DOI · Paper page
  15. Bao, BEiT: BERT pre-training of image transformers
  16. 10.1109/TBIOM.2021.3086109 · DOI · Paper page
  17. 10.1145/3306346.3323035 · DOI · Paper page
  18. 2022, Contributing Data to Deepfake Detection Research
  19. Tan, EfficientNet: Rethinking model scaling for convolutional neural networks, 6105
  20. 10.1109/ICCV.2017.74 · DOI · Paper page
  21. Van der Maaten, 2008, Visualizing data using t-SNE, J. Mach. Learn. Res., 9, 1

Cited by

See all 80

Metadata

Authors
Chen, H., Lin, Y., Li, B., & Tan, S.
Venue
IEEE Transactions on Circuits and Systems for Video Technology
Published
2023-03
Volume
33
Issue
3
Pages
1468-1480
ISSN
  • 1051-8215 (ISSN-L)
  • 1558-2205
OpenAlex
W4297095254
Cited by
80
References
34
FWCI
5.99
Type
journal article
Language
en
Open access
closed
Funders
  • NSFC (61872244, 62272314)
  • Basic and Applied Basic Research Foundation of Guangdong Province (2019B151502001)
  • Shenzhen Research and Development Program (JCYJ20200109105008228)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Learning Features of Intra-Consistency and Inter-Diversity: Keys Toward Generalizable Deepfake Detection. https://app.cochat.ai/papers/10.1109/tcsvt.2022.3209336
The paper
Chen, H., Lin, Y., Li, B., & Tan, S. (2023). Learning Features of Intra-Consistency and Inter-Diversity: Keys Toward Generalizable Deepfake Detection. IEEE Transactions on Circuits and Systems for Video Technology. https://doi.org/10.1109/tcsvt.2022.3209336

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.