CoChat
Sign Up
Cited by 155Digital Media Forensic DetectionFace recognition and analysisGenerative Adversarial Networks and Image Synthesis

Domain General Face Forgery Detection by Learning to Weight

Open access
Sun, Ke, Liu, Hong, Ye, Qixiang, Gao, Yue, Liu, Jianzhuang, Shao, Ling, Ji, Rongrong·2021·Proceedings of the AAAI Conference on Artificial Intelligence
DOI 10.1609/aaai.v35i3.16367
Open this paper in CoChat — free account

Abstract

In this paper, we propose a domain-general model, termed learning-to-weight (LTW), that guarantees face detection performance across multiple domains, particularly the target domains that are never seen before. However, various face forgery methods cause complex and biased data distributions, making it challenging to detect fake faces in unseen domains. We argue that different faces contribute differently to a detection model trained on multiple domains, making the model likely to fit domain-specific biases. As such, we propose the LTW approach based on the meta-weight learning algorithm, which configures different weights for face images from different domains. The LTW network can balance the model's generalizability across multiple domains. Then, the meta-optimization calibrates the source domain's gradient enabling more discriminative features to be learned. The detection ability of the network is further improved by introducing an intra-class compact loss. Extensive experiments on several commonly used deepfake datasets to demonstrate the effectiveness of our method in detecting synthetic faces. Code and supplemental material are available at https://github.com/skJack/LTW.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1609/aaai.v35i3.16367

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Optimizing each sample equally as in the traditional framework makes it difficult to uncover underlying forgery clues and is not conducive to generalization.

    “Moreover, the quality of forgery faces is different, and optimizing each sample equally as in the traditional framework makes it difficult to uncover the underlying forgery clues and is not conducive to generalization (Liu et al. 2021b; Sun et al. 2021).”

    From Dual Contrastive Learning for General Face Forgery Detection (2022) · cites this paper as Sun et al. 2021

  • High-quality samples are more conducive to improving the generalization of the model.

    “Meanwhile, Sun et al. (2021) shows high-quality samples are more conducive to improving the generalization of the model.”

    From Dual Contrastive Learning for General Face Forgery Detection (2022) · cites this paper as Sun et al. (2021)

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 23

  1. Afchar, D.; Nozick, V.; Yamagishi, J.; and Echizen, I. 2018. Mesonet: a compact facial video forgery detection network. In WIFS , 1-7. IEEE.
  2. Agarwal, S.; and Farid, H. 2017. Photo forensics from JPEG dimples. In WIFS , 1-6. IEEE.
  3. Chen, M.; Xu, Z.; Weinberger, K.; and Sha, F. 2012. Marginalized denoising autoencoders for domain adaptation. arXiv preprint arXiv:1206.4683 . · arXiv
  4. Cozzolino, D.; Thies, J.; R¨ ossler, A.; Riess, C.; Nießner, M.; and Verdoliva, L. 2018. Forensictransfer: Weaklysupervised domain adaptation for forgery detection. arXiv preprint arXiv:1812.02510 . · arXiv
  5. Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and FeiFei, L. 2009. Imagenet: A large-scale hierarchical image database. In CVPR , 248-255. Ieee.
  6. Dolhansky, B.; Bitton, J.; Pflaum, B.; Lu, J.; Howes, R.; Wang, M.; and Ferrer, C. C. 2020. The DeepFake Detection Challenge Dataset. arXiv preprint arXiv:2006.07397 . · arXiv
  7. Finn, C.; Abbeel, P.; and Levine, S. 2017. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. In ICML .
  8. Glorot, X.; Bordes, A.; and Bengio, Y. 2011. Domain adaptation for large-scale sentiment classification: A deep learning approach. In ICML .
  9. Guo, J.; Zhu, X.; Zhao, C.; Cao, D.; Lei, Z.; and Li, S. Z. 2020. Learning meta face recognition in unseen domains. In CVPR , 6163-6172.
  10. Kuang, H.; Ji, R.; Liu, H.; Zhang, S.; Sun, X.; Huang, F.; and Zhang, B. 2019. Multi-modal Multi-layer Fusion Network with Average Binary Center Loss for Face Anti-spoofing. In ACM MM , 48-56.
  11. Li, D.; Yang, Y.; Song, Y.-Z.; and Hospedales, T. M. 2018. Learning to generalize: Meta-learning for domain generalization. In AAAI .
  12. Li, L.; Bao, J.; Zhang, T.; Yang, H.; Chen, D.; Wen, F.; and Guo, B. 2020. Face x-ray for more general face forgery detection. In CVPR , 5001-5010.
  13. Li, Y.; Yang, X.; Sun, P.; Qi, H.; and Lyu, S. 2019. Celebdf: A new dataset for deepfake forensics. arXiv preprint arXiv:1909.12962 . · arXiv
  14. Matern, F.; Riess, C.; and Stamminger, M. 2019. Exploiting visual artifacts to expose deepfakes and face manipulations. In WACVW , 83-92. IEEE.
  15. McCloskey, S.; and Albright, M. 2018. Detecting gangenerated imagery using color cues. arXiv preprint arXiv:1812.08247 . · arXiv
  16. Mirsky, Y.; and Lee, W. 2020. The Creation and Detection of Deepfakes: A Survey. arXiv preprint arXiv:2004.11138 . · arXiv
  17. Nguyen, H. H.; Fang, F.; Yamagishi, J.; and Echizen, I. 2019. Multi-task learning for detecting and segmenting manipulated facial images and videos. arXiv preprint arXiv:1906.06876 . · arXiv
  18. Perera, P.; and Patel, V. M. 2019. Learning deep features for one-class classification. IEEE Transactions on Image Processing 28(11): 5450-5463.
  19. Qian, Y.; Yin, G.; Sheng, L.; Chen, Z.; and Shao, J. 2020. Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues. In ECCV , 86-103. Springer.
  20. Qiao, S.; Liu, C.; Shen, W.; and Yuille, A. L. 2018. Fewshot image recognition by predicting parameters from activations. In CVPR , 7229-7238.
  21. Rossler, A.; Cozzolino, D.; Verdoliva, L.; Riess, C.; Thies, J.; and Nießner, M. 2019. Faceforensics++: Learning to detect manipulated facial images. In ICCV , 1-11.
  22. Vinyals, O.; Blundell, C.; Lillicrap, T.; Wierstra, D.; et al. 2016. Matching networks for one shot learning. In NeurIPS , 3630-3638.
  23. Zhou, P.; Han, X.; Morariu, V. I.; and Davis, L. S. 2017. Two-stream neural networks for tampered face detection. In CVPRW , 1831-1839. IEEE.

Cited by

Sorted by citations

Authors
Sun, K., Liu, H., Ye, Q., Gao, Y., Liu, J., Shao, L., & Ji, R.
Affiliations
  • Xiamen University (CN)
  • National Institute of Informatics (JP)
  • University of Chinese Academy of Sciences (CN)
  • Tsinghua University (CN)
  • Huawei Technologies (China) (CN)
  • Inception Institute of Artificial Intelligence (AE)
ORCID
Venue
Proceedings of the AAAI Conference on Artificial Intelligence
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Published
2021-05-18
Volume
35
Issue
3
Pages
2638-2646
ISSN
  • 2159-5399 (ISSN-L)
  • 2374-3468
DOI
10.1609/aaai.v35i3.16367
OpenAlex
W3173317327
Cited by
155
References
7
Citations by year
2026: 12 · 2025: 40 · 2024: 40 · 2023: 38 · 2022: 23 · 2021: 2
FWCI
8.42
Type
conference paper
Language
en
Open access
Locations
  • Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
  • AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
Funders
  • National Natural Science Foundation of China (62072387, 62025603, U1705262, 62072389, 61802324, No. 62025603, 61772443, 62002305, No. 62002305, 62072386, 61702136)
  • Basic and Applied Basic Research Foundation of Guangdong Province (No. 2019B1515120049, 2019B1515120049)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Keywords
face forgery detection, deepfake detection, domain generalization, unseen domains, meta-optimization, synthetic face detection, domain bias

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review