Evading Deepfake-Image Detectors with White- and Black-Box Attacks
Open accessAbstract
It is now possible to synthesize highly realistic images of people who do not exist. Such content has, for example, been implicated in the creation of fraudulent socialmedia profiles responsible for dis-information campaigns. Significant efforts are, therefore, being deployed to detect synthetically-generated content. One popular forensic approach trains a neural network to distinguish real from synthetic content.We show that such forensic classifiers are vulnerable to a range of attacks that reduce the classifier to near- 0% accuracy. We develop five attack case studies on a state- of-the-art classifier that achieves an area under the ROC curve (AUC) of 0.95 on almost all existing image generators, when only trained on one generator. With full access to the classifier, we can flip the lowest bit of each pixel in an image to reduce the classifier's AUC to 0.0005; perturb 1% of the image area to reduce the classifier's AUC to 0.08; or add a single noise pattern in the synthesizer's latent space to reduce the classifier's AUC to 0.17. We also develop a black-box attack that, with no access to the target classifier, reduces the AUC to 0.22. These attacks reveal significant vulnerabilities of certain image-forensic classifiers.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
All studies failed in evaluating their effectiveness in tackling adversarial noise attacks.
“However, these DNN-based detection methods all suffer from the adversarial attacks with additive noises and all the studies failed in evaluating their effectiveness in tackling adversarial noise attacks (Carlini and Farid 2020).”
From Countering Malicious DeepFakes: Survey, Battleground, and Horizon (2022) · cites this paper as Carlini and Farid 2020
The Introduction and Conclusions report that the forensic classifiers tested were highly vulnerable to white- and black-box adversarial attacks. They do not establish that all DNN-based detection studies failed to evaluate effectiveness against adversarial noise attacks; the paper tested two previously published classifiers and one of its own. (LLM, conf=0.83)
Experimental — accuracy not yet measuredDNN-based detection methods all suffer from adversarial attacks with additive noises.
“However, these DNN-based detection methods all suffer from the adversarial attacks with additive noises and all the studies failed in evaluating their effectiveness in tackling adversarial noise attacks (Carlini and Farid 2020).”
From Countering Malicious DeepFakes: Survey, Battleground, and Horizon (2022) · cites this paper as Carlini and Farid 2020
The Introduction and Conclusions report that the neural-network deepfake-image detectors the authors examined were highly vulnerable to adversarial perturbations. The paper does not establish that all DNN-based detection methods are vulnerable, and the cited passages do not support the sentence’s further assertion that all prior studies failed to evaluate resistance to adversarial noise attacks. (LLM, conf=0.86)
Experimental — accuracy not yet measuredThere are two critical challenges via spatial based solution: poor generalization capability against unknown synthetic techniques and low robustness to adversarial attacks.
“Nevertheless, there are two critical challenges via the spatial based solution to fight against DeepFakes, i.e., poor generalization capability against unknown synthetic techniques and low robustness to adversarial attacks (Carlini and Farid 2020).”
From Countering Malicious DeepFakes: Survey, Battleground, and Horizon (2022) · cites this paper as Carlini and Farid 2020
The Introduction says low-level forensic techniques struggle to generalize to novel datasets, and it reports that the classifiers studied are vulnerable to adversarial attacks. But it also says recent detectors generalize across previously unseen generators, so the claim overstates poor generalization as a general limitation of these approaches. Page numbers are not available in our copy. (LLM, conf=0.86)
Experimental — accuracy not yet measuredCarlini and Farid (2020) add imperceptible adversarial perturbations to fake/real faces and show that even state-of-the-art DeepFake detectors are vulnerable to both white-box and black-box attacks with significant accuracy reduction on public datasets.
“For example, Carlini and Farid (2020) add imperceptible adversarial perturbations to the fake/real faces and show that even the state-of-the-art DeepFake detectors are vulnerable to both white-box and black-box attacks (Carlini and Wagner 2017; Brown et al. 2017) with significant accuracy reduction on the public datasets (Wang et al. 2020e; Frank et al. 2020).”
From Countering Malicious DeepFakes: Survey, Battleground, and Horizon (2022) · cites this paper as Carlini and Farid 2020
CNN-based deepfake detection systems have recently been exposed to adversarial strategies using gradient-based adversarial attacks, and these attacks cause the classifier's accuracy to degrade to a near-0% level.
“According to Neekhara etal. (2021), Carlini and Farid (2020), CNN-based deepfake detection systems have recently been exposed to adversarial strategies using gradient-based adversarial attacks. These attacks cause the classifier's accuracy to degrade to a near-0% level.”
From Deepfake video detection: challenges and opportunities (2024) · cites this paper as Carlini and Farid 2020
Existing works have investigated the evasion and transferability of adversarial deepfakes in the lab setting.
“existing works have investigated the evasion and transferability of adversarial deepfakes in the lab setting”
From DEEPFAKER: A Unified Evaluation Platform for Facial Deepfake and Detection Models (2024) · cites this paper as [6]
Adversarial deepfakes can evade detectors by adding a carefully crafted perturbation to the deepfake samples.
“there have been many related works demonstrating that adversarial deepfakes can evade detectors by adding a carefully crafted perturbation to the deepfake samples”
From DEEPFAKER: A Unified Evaluation Platform for Facial Deepfake and Detection Models (2024) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 48 · Cited by 170
Research works printed but not linked: 22
- Shruti Agarwal, Hany Farid, Yuming Gu, Mingming He, Koki Nagano, and Hao Li. Protecting world leaders against deep fakes. In IEEE Conference on Computer Vision and Pattern Recognition, Workshop on Media Forensics, pages 38-45, 2019. 2
- Ross Anderson. Security Engineering. John Wiley & Sons, 2008. 6
- Anish Athalye, Nicholas Carlini, and David Wagner. Obfus- cated gradients give a false sense of security: Circumventing defenses to adversarial examples. arXiv: 1802.00420, 2018. · arXiv
- Rainer Böhme and Matthias Kirchner. Counter-forensics: Attacking image forensics. In Digital image forensics, pages 327-366. Springer, 2013. 3
- Jacob Buckman, Aurko Roy, Colin Raffel, and Ian Goodfel- low. Thermometer encoding: One hot way to resist adver- sarial examples. In International Conference on Learning Representations, 2018. 8
- Nicholas Carlini and David Wagner. Audio adversarial ex- amples: Targeted attacks on speech-to-text. In IEEE Security and Privacy Workshops, pages 1-7, 2018. 3
- Umur Aybars Ciftci and Ilke Demir. Fakecatcher: Detection of synthetic portrait videos using biological signals. arXiv: 1901.02212, 2019. 2 · arXiv
- Jeremy M Cohen, Elan Rosenfeld, and J Zico Kolter. Certified adversarial robustness via randomized smoothing. arXiv:1 902.02918, 2019. 8
- Davide Cozzolino, Justus Thies, Andreas Rössler, Christian Riess, Matthias Nießner, and Luisa Verdoliva. Forensictrans- fer: Weakly-supervised domain adaptation for forgery detec- tion. arXiv: 1812.02510, 2018. 2 · arXiv
- Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pat- tern Recognition, pages 248-255, 2009. 2
- Apurva Gandhi and Shomik Jain. Adversarial perturbations fool deepfake detectors. arXiv: 2003.10596, 2020. 3 · arXiv
- Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversar- ial examples are not bugs, they are features. In Advances in Neural Information Processing Systems, pages 125-136, 2019. 8
- Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive Growing of GANs for Improved Quality, Stabil- ity, and Variation. arXiv: 1710.10196, 2017. 2, 3 · arXiv
- Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and Improving the Image Quality of StyleGAN. arXiv: 1912.04958, 2019. 1, 2 · arXiv
- Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified robustness to ad- versarial examples with differential privacy. In 2019 IEEE Symposium on Security and Privacy, pages 656-672, 2019.
- Yuezun Li, Ming-Ching Chang, and Siwei Lyu. In ictu oculi: Exposing AI created fake videos by detecting eye blinking. In IEEE International Workshop on Information Forensics and Security, pages 1-7, 2018. 2
- Dongyu Meng and Hao Chen. Magnet: a two-pronged de- fense against adversarial examples. In ACM SIGSAC Con- ference on Computer and Communications Security, pages 135-147, 2017. 8
- Taesung Park, Ming-Yu Liu, Ting-Chun Wang, and Jun-Yan Zhu. Semantic image synthesis with spatially-adaptive nor- malization. In IEEE Conference on Computer Vision and Pattern Recognition, pages 2337-2346, 2019. 2
- Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Chris- tian Riess, Justus Thies, and Matthias Nießner. Faceforen- sics++: Learning to detect manipulated facial images. In IEEE International Conference on Computer Vision, pages 1-11, 2019. 2
- Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. CNN-generated images are surprisingly easy to spot...for now. In IEEE Conference on Computer Vision and Pattern Recognition, 2020. 1, 2, 3, 4, 5, 7, 8
- Xin Yang, Yuezun Li, and Siwei Lyu. Exposing deep fakes using inconsistent head poses. In IEEE International Con- ference on Acoustics, Speech and Signal Processing, pages 8261-8265, 2019. 2
- Ning Yu, Larry Davis, and Mario Fritz. Attributing fake im- ages to GANs: Analyzing fingerprints in generated images. arXiv:1811.08180, 2018. 2 · arXiv
- Authors
- Carlini, N., & Farid, H.
- Affiliations
- Google (United States) (US)
- University of California, Berkeley (US)
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 2804-2813
- OpenAlex
- W3015087080
- Cited by
- 170
- References
- 48
- Citations by year
- 2026: 12 · 2025: 24 · 2024: 30 · 2023: 33 · 2022: 32 · 2021: 25 · 2020: 13 · 2019: 1
- FWCI
- 7.94
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- U.S. Department of Defense
- Defense Advanced Research Projects Agency (FA8750-16-C-0166, FA8750)
- Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
- Keywords
- deepfake detection, image forensics, adversarial attacks, white-box attacks, black-box attacks, synthetic image generation, classifier robustness, disinformation, latent space perturbation, image forgery detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
