CoChat
Sign Up
Cited by 43Digital Media Forensic DetectionAdversarial Robustness in Machine LearningGenerative Adversarial Networks and Image Synthesis

Exploring Adversarial Fake Images on Face Manifold

Open access
Li, Dongze, Wang, Wei, Fan, Hongxing, Dong, Jing·2021·2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
DOI 10.1109/cvpr46437.2021.00573
Open this paper in CoChat — free accountOpen PDF

Abstract

Images synthesized by powerful generative adversarial network (GAN) based methods have drawn moral and privacy concerns. Although image forensic models have reached great performance in detecting fake images from real ones, these models can be easily fooled with a simple adversarial attack. But, the noise adding adversarial samples are also arousing suspicion. In this paper, instead of adding adversarial noise, we optimally search adversarial points on face manifold to generate anti-forensic fake face images. We iteratively do a gradient-descent with each small step in the latent space of a generative model, e.g. Style-GAN, to find an adversarial latent vector, which is similar to norm-based adversarial attack but in latent space. Then, the generated fake images driven by the adversarial latent vectors with the help of GANs can defeat main-stream forensic models. For examples, they make the accuracy of deepfake detection models based on Xception or EfficientNet drop from over 90% to nearly 0%, mean-while maintaining high visual quality. In addition, we find manipulating noise vectors n at different levels have different impacts on attack success rate, and the generated adversarial images mainly have changes on facial texture or face attributes.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/cvpr46437.2021.00573

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • DF detectors are increasingly vulnerable to adversarial attacks, which introduce carefully crafted perturbations into DF content to deceive detection systems while remaining imperceptible to humans.

    “DF detectors are increasingly vulnerable to adversarial attacks, which introduce carefully crafted perturbations into DF content to deceive detection systems while remaining imperceptible to humans.”

    From Deepfake detection across image, video, and audio: a comprehensive survey with empirical evaluation of generalization and robustness (2026) · cites this paper as [197]

    The Introduction describes adversarial fake face images that fool forensic detectors, and §4.2 reports that the attack sharply reduced two detectors’ accuracy. But the paper does not establish that detectors are increasingly vulnerable over time. It also describes its method as searching StyleGAN’s latent space, rather than adding imperceptible perturbations to existing deepfake content; the Introduction warns that conventional image-space attacks may leave visible perturbations. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Recent studies have demonstrated that even SoTA detection models can be significantly compromised by adversarial examples.

    “Recent studies have demonstrated that even SoTA detection models can be significantly compromised by adversarial examples (AEs) [21, 197-199].”

    From Deepfake detection across image, video, and audio: a comprehensive survey with empirical evaluation of generalization and robustness (2026) · cites this paper as [197]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 37

  1. Darius Afchar, Vincent Nozick, Junichi Yamagishi, and Isao Echizen. Mesonet: a compact facial video forgery detection network. In 2018 IEEE International Workshop on Informa- tion Forensics and Security (WIFS), pages 1-7. IEEE, 2018. 2
  2. Franc ¸ois Chollet. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pages 1251-1258, 2017. 2, 3, 5
  3. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255. Ieee, 2009. 3
  4. Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial at- tacks with momentum. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pages 9185-9193, 2018. 3
  5. Logan Engstrom, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. A rotation and a translation suffice: Fooling cnns with simple transformations. arXiv preprint arXiv:1712.02779, 1(2):3, 2017. 3 · arXiv
  6. Pablo Garrido, Levi Valgaerts, Ole Rehmsen, Thorsten Thormahlen, Patrick Perez, and Christian Theobalt. Auto- matic face reenactment. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pages 4217-4224, 2014. 2
  7. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672-2680, 2014. 2
  8. Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014. 1, 2, 3 · arXiv
  9. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adver- sarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1125-1134, 2017. 2
  10. Kazuya Kakizaki and Kosuke Yoshida. Adversarial image translation: Unrestricted adversarial examples in face recog- nition systems. arXiv preprint arXiv:1905.03421, 2019. 2, 3 · arXiv
  11. Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196, 2017. 2 · arXiv
  12. Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4401-4410, 2019. 1, 2
  13. Iryna Korshunova, Wenzhe Shi, Joni Dambre, and Lucas Theis. Fast face-swap using convolutional neural networks. In Proceedings of the IEEE International Conference on Computer Vision, pages 3677-3685, 2017. 2
  14. Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang, Dong Chen, Fang Wen, and Baining Guo. Face x-ray for more gen- eral face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5001-5010, 2020. 2
  15. Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. Celeb-df: A new dataset for deepfake forensics. arXiv preprint arXiv:1909.12962, 2019. 1 · arXiv
  16. Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1778-1787, 2018. 3
  17. Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial examples and black- box attacks. arXiv preprint arXiv:1611.02770, 2016. 3 · arXiv
  18. Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), De- cember 2015. 3
  19. Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014. 2 · arXiv
  20. Yuval Nirkin, Yosi Keller, and Tal Hassner. Fsgan: Subject agnostic face swapping and reenactment. In Proceedings of the IEEE international conference on computer vision, pages 7184-7193, 2019. 2
  21. Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277, 2016. 3 · arXiv
  22. Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense to adver- sarial perturbations against deep neural networks. In 2016 IEEE Symposium on Security and Privacy (SP), pages 582- 597. IEEE, 2016. 3
  23. Omid Poursaeed, Tianxing Jiang, Harry Yang, Serge Belongie, and Ser-Nam Lim. Fine-grained synthesis of unrestricted adversarial examples. arXiv preprint arXiv:1911.09058, 2019. 3, 4, 5 · arXiv
  24. Haonan Qiu, Chaowei Xiao, Lei Yang, Xinchen Yan, Honglak Lee, and Bo Li. Semanticadv: Generating adversar- ial examples via attribute-conditional image editing. arXiv preprint arXiv:1906.07927, 2019. 3 · arXiv
  25. Jérôme Rony, Luiz G Hafemann, Luiz S Oliveira, Ismail Ben Ayed, Robert Sabourin, and Eric Granger. Decoupling direc- tion and norm for efficient gradient-based l2 adversarial at- tacks and defenses. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4322- 4330, 2019. 3
  26. Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Chris- tian Riess, Justus Thies, and Matthias Nießner. Faceforen- sics++: Learning to detect manipulated facial images. In Proceedings of the IEEE International Conference on Com- puter Vision, pages 1-11, 2019. 1, 2
  27. Yang Song, Rui Shu, Nate Kushman, and Stefano Ermon. Constructing unrestricted adversarial examples with genera- tive models. In Advances in Neural Information Processing Systems, pages 8312-8323, 2018. 2, 3
  28. M. C. Stamm and K. J. R. Liu. Anti-forensics of digital im- age compression. IEEE Transactions on Information Foren- sics and Security, 6(3):1050-1065, 2011. 1
  29. Mingxing Tan and Quoc V Le. Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:1905.11946, 2019. 2, 3, 5 · arXiv
  30. Justus Thies, Michael Zollhofer, Marc Stamminger, Chris- tian Theobalt, and Matthias Nießner. Face2face: Real-time face capture and reenactment of rgb videos. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 2387-2395, 2016. 2
  31. Xiaosen Wang, Kun He, Chuanbiao Song, Liwei Wang, and John E Hopcroft. At-gan: An adversarial generator model for non-constrained adversarial examples. arXiv preprint arXiv:1904.07793, 2019. 2 · arXiv
  32. Weibin Wu, Yuxin Su, Xixian Chen, Shenglin Zhao, Irwin King, Michael R Lyu, and Yu-Wing Tai. Boosting the trans- ferability of adversarial samples via attention. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1161-1170, 2020. 3
  33. Chaowei Xiao, Jun-Yan Zhu, Bo Li, Warren He, Mingyan Liu, and Dawn Song. Spatially transformed adversarial ex- amples. arXiv preprint arXiv:1801.02612, 2018. 3 · arXiv
  34. Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015. 3 · arXiv
  35. Gang Zhang, Meina Kan, Shiguang Shan, and Xilin Chen. Generative adversarial network with spatial attention for face attribute editing. In Proceedings of the European conference on computer vision (ECCV), pages 417-432, 2018. 2
  36. Peng Zhou, Xintong Han, Vlad I Morariu, and Larry S Davis. Two-stream neural networks for tampered face detection. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1831-1839. IEEE, 2017. 2
  37. Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle- consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 2223- 2232, 2017.

Cited by

Sorted by citations

Authors
Li, D., Wang, W., Fan, H., & Dong, J.
Affiliations
Institute of Automation (CN)
ORCID
Venue
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2021-06
Pages
5785-5794
ISSN
1063-6919 (ISSN-L)
DOI
10.1109/cvpr46437.2021.00573
OpenAlex
W3193285461
Cited by
43
References
5
Citations by year
2026: 4 · 2025: 10 · 2024: 9 · 2023: 9 · 2022: 8 · 2021: 3
FWCI
2.37
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
National Natural Science Foundation of China
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Keywords
adversarial attacks, deepfake detection, image forensics, StyleGAN, generative adversarial networks, gradient-based attacks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review