CoChat
Sign Up
Cited by 609Digital Media Forensic DetectionGenerative Adversarial Networks and Image SynthesisBiometric Identification and Security

Generalizing Face Forgery Detection with High-frequency Features

Open access
Luo, Yuchen, Zhang, Yong, Yan, Junchi, Liu, Wei·2021·2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
DOI 10.1109/cvpr46437.2021.01605
Open this paper in CoChat — free accountOpen PDF

Abstract

Current face forgery detection methods achieve high accuracy under the within-database scenario where training and testing forgeries are synthesized by the same algorithm. However, few of them gain satisfying performance under the cross-database scenario where training and testing forgeries are synthesized by different algorithms. In this paper, we find that current CNN-based detectors tend to overfit to method-specific color textures and thus fail to generalize. Observing that image noises remove color textures and expose discrepancies between authentic and tampered regions, we propose to utilize the high-frequency noises for face forgery detection. We carefully devise three functional modules to take full advantage of the high-frequency features. The first is the multi-scale high-frequency feature extraction module that extracts high-frequency noises at multiple scales and composes a novel modality. The second is the residual-guided spatial attention module that guides the low-level RGB feature extractor to concentrate more on forgery traces from a new perspective. The last is the cross-modality attention module that leverages the correlation between the two complementary modalities to promote feature learning for each other. Comprehensive evaluations on several benchmark databases corroborate the superior generalization performance of our proposed method.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/cvpr46437.2021.01605

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A broadly received explanation for poor cross-dataset generalization is that deepfake detectors overfit to forgery-specific artifacts rather than learning features widely applicable across forgeries.

    “A broadly received explanation is the tendency of these detectors to be overfitted to forgery-specific artifacts, rather than learning features that are widely applicable across various forgeries.”

    From Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detection (2024) · cites this paper as [34]

  • Each forgery method typically possesses its specific characteristics, and overfitting to a particular forgery type may impede generalization to other types.

    “given that each forgery method typically possesses its specific characteristics, the overfitting to a particular type of forgery may impede the model's ability to generalize effectively to other types (also indicated in previous works [34, 42, 54]).”

    From Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detection (2024) · cites this paper as [34]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 35

  1. Belhassen Bayar and Matthew C Stamm. A deep learning approach to universal image manipulation detection using a new convolutional layer. In Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security, pages 5-10, 2016. 8
  2. Zehao Chen and Hua Yang. Manipulated face detector: Joint spatial and frequency domain attention network. arXiv preprint arXiv:2005.02958, 2020. 1, 2, 4 · arXiv
  3. Davide Cozzolino, Justus Thies, Andreas Rössler, Christian Riess, Matthias Nießner, and Luisa Verdoliva. Forensictrans- fer: Weakly-supervised domain adaptation for forgery detec- tion. arXiv preprint arXiv:1812.02510, 2018. 1, 8 · arXiv
  4. Mengnan Du, Shiva Pentyala, Yuening Li, and Xia Hu. To- wards generalizable forgery detection with locality-aware autoencoder. arXiv preprint arXiv:1909.05999, 2019. 1, 2, 8 · arXiv
  5. Ricard Durall, Margret Keuper, Franz-Josef Pfreundt, and Janis Keuper. Unmasking deepfakes with simple features. arXiv preprint arXiv:1911.00686, 2019. 2 · arXiv
  6. Hany Farid. Image forgery detection. IEEE Signal process- ing magazine, 2009. 2
  7. J. Fridrich. Digital image forensics. IEEE Signal Processing Magazine, 26(2):26-37, 2009. 4
  8. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NIPS, 2014. 1
  9. Ruibing Hou, Hong Chang, MA Bingpeng, Shiguang Shan, and Xilin Chen. Cross attention network for few-shot classi- fication. In NIPS, 2019. 5
  10. Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation net- works. In CVPR, 2018. 5, 6
  11. Thibaut Julliand, Vincent Nozick, and Hugues Talbot. Image noise and digital image forensics. volume 9569, pages 3-17, 03 2016. 4
  12. Diederik P Kingma and Max Welling. Auto-encoding varia- tional bayes. ICLR, 2014. 1
  13. Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang, Dong Chen, Fang Wen, and Baining Guo. Face x-ray for more general face forgery detection. CVPR, 2020. 1, 2, 4, 7
  14. Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. Celeb-df: A new dataset for deepfake forensics. arXiv preprint arXiv:1909.12962, 2019. 6, 7 · arXiv
  15. Babak Mahdian and Stanislav Saic. Using noise inconsisten- cies for blind image forensics. Image and Vision Computing, 27(10):1497 -1503, 2009. Special Section: Computer Vi- sion Methods for Ambient Intelligence. 4
  16. Huy H Nguyen, Fuming Fang, Junichi Yamagishi, and Isao Echizen. Multi-task learning for detecting and segment- ing manipulated facial images and videos. arXiv preprint arXiv:1906.06876, 2019. 1, 2, 8 · arXiv
  17. Huy H Nguyen, Junichi Yamagishi, and Isao Echizen. Capsule-forensics: Using capsule networks to detect forged images and videos. In ICASSP, 2019. 2
  18. Marco Pedersoli, Thomas Lucas, Cordelia Schmid, and Jakob Verbeek. Areas of attention for image captioning. In ICCV, 2017. 5
  19. Yuyang Qian, Guojun Yin, Lu Sheng, Zixuan Chen, and Jing Shao. Thinking in frequency: Face forgery detec- tion by mining frequency-aware clues. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm, edi- tors, Computer Vision -ECCV 2020, pages 86-103, Cham, 2020. Springer International Publishing. 1, 2, 8
  20. Anderson Rocha, Walter Scheirer, Terrance Boult, and Siome Goldenstein. Vision of the unseen: Current trends and challenges in digital image and video forensics. ACM Computing Surveys (CSUR), 2011. 2
  21. Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Chris- tian Riess, Justus Thies, and Matthias Nießner. Faceforen- sics++: Learning to detect manipulated facial images. In ICCV, 2019. 6, 7
  22. R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 618-626, 2017. 3
  23. L. Song, D. Gong, Z. Li, C. Liu, and W. Liu. Occlusion ro- bust face recognition based on mask learning with pairwise differential siamese network. In 2019 IEEE/CVF Interna- tional Conference on Computer Vision (ICCV), pages 773- 782, 2019. 6
  24. Kritaphat Songsri-in and Stefanos Zafeiriou. Complement face forensic detection and localization with facialland- marks. arXiv preprint arXiv:1910.05455, 2019. 2 · arXiv
  25. Joel Stehouwer, Hao Dang, Feng Liu, Xiaoming Liu, and Anil Jain. On the detection of digital face manipulation. CVPR, 2020. 1, 2, 8
  26. Justus Thies, Michael Zollhofer, Marc Stamminger, Chris- tian Theobalt, and Matthias Nießner. Face2face: Real-time face capture and reenactment of rgb videos. In CVPR, 2016. 3, 6, 7
  27. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, 2017. 5
  28. Feng Wang, Jian Cheng, Weiyang Liu, and Haijun Liu. Ad- ditive margin softmax for face verification. IEEE Signal Pro- cessing Letters, 2018. 6
  29. Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. Cbam: Convolutional block attention module. In ECCV, 2018. 4
  30. Xi Wu, Zhen Xie, YuTao Gao, and Yu Xiao. Sstnet: De- tecting manipulated faces through spatial, steganalysis and temporal features. In ICASSP, 2020. 1, 2, 8
  31. Yue Wu, Wael AbdAlmageed, and Premkumar Natarajan. Mantra-net: Manipulation tracing network for detection and localization of image forgeries with anomalous features. In CVPR, 2019. 4
  32. Chao Yang and Ser-Nam Lim. One-shot domain adaptation for face generation. CVPR, 2020. 1, 2
  33. Xin Yang, Yuezun Li, and Siwei Lyu. Exposing deep fakes using inconsistent head poses. In ICASSP, 2019. 2
  34. Linwei Ye, Mrigank Rochan, Zhi Liu, and Yang Wang. Cross-modal self-attention network for referring image seg- mentation. In CVPR, 2019. 5
  35. Peng Zhou, Xintong Han, Vlad I Morariu, and Larry S Davis. Learning rich features for image manipulation detection. In CVPR, 2018. 4, 8

Cited by

Sorted by citations

Authors
Luo, Y., Zhang, Y., Yan, J., & Liu, W.
Affiliations
  • Shanghai Jiao Tong University (CN)
  • Tencent (China) (CN)
ORCID
Venue
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2021-06
Pages
16312-16321
ISSN
1063-6919 (ISSN-L)
DOI
10.1109/cvpr46437.2021.01605
OpenAlex
W3174656926
Cited by
609
References
10
Citations by year
2026: 121 · 2025: 184 · 2024: 147 · 2023: 104 · 2022: 49 · 2021: 4
FWCI
24.86
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • Research and Development
  • Tencent
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Signal Processing › Biometric Identification and Security
Keywords
face forgery detection, image tampering, cross-dataset generalization, high-frequency characteristics, image noise, color texture, multi-scale feature extraction, spatial attention, cross-modal attention

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review