Exposing Vulnerabilities of Deepfake Detection Systems with Robust Attacks
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Recent advances in video manipulation techniques have made the generation of fake videos more accessible than ever before. Manipulated videos can fuel disinformation and reduce trust in media. Therefore detection of fake videos has garnered immense interest in academia and industry. Recently developed Deepfake detection methods rely on Deep Neural Networks (DNNs) to distinguish AI-generated fake videos from real videos. In this work, we demonstrate that it is possible to bypass such detectors by adversarially modifying fake videos synthesized using existing Deepfake generation methods. We further demonstrate that our adversarial perturbations are robust to image and video compression codecs, making them a real-world threat. We present pipelines in both white-box and black-box attack scenarios that can fool DNN-based Deepfake detectors into classifying fake videos as real. Finally, we study the extent to which adversarial perturbations transfer across different Deepfake detectors and create more accessible attacks using universal adversarial perturbations that pose a very feasible attack scenario since they can be easily shared amongst attackers. 1
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Limitations of Deep Learning in Adversarial Settings
Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2016) · 2016 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2016.36 · Cited by 3,993
Practical Black-Box Attacks against Machine Learning
Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z. B., & Swami, A. (2017) · Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security · doi:10.1145/3052973.3053009 · Cited by 3,495
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 21×
“Accuracy of Deepfake detectors on the FaceForensics++ HQ Dataset as reported in [57].”
BackgroundMethodInfluentialUniversal Adversarial Perturbations
Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., & Frossard, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.17 · Cited by 2,680
Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples
Athalye, A., Carlini, N., & Wagner, D. (2018) · arXiv · doi:10.48550/arxiv.1802.00420 · Cited by 2,172 · cited 2×
“Gaussian Blur Kernel k (d,d,σ ), d ∼ U[3, 7], σ ∼ U[5, 10] Gaussian Noise σ ∼ U[0.”
BackgroundMesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Face2Face: Real-Time Face Capture and Reenactment of RGB Videos
Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., & Niessner, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.262 · Cited by 1,747
RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild
Deng, J., Guo, J., Ververas, E., Kotsia, I., & Zafeiriou, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00525 · Cited by 1,681
Delving into Transferable Adversarial Examples and Black-box Attacks
Liu, Y., Chen, X., Liu, C., & Song, D. (2016) · arXiv · doi:10.48550/arxiv.1611.02770 · Cited by 1,310
Face X-Ray for More General Face Forgery Detection
Li, L., Bao, J., Zhang, T., Yang, H., Chen, D., Wen, F., & Guo, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00505 · Cited by 1,211
Exposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196
In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking
Li, Y., Chang, M.-C., & Lyu, S. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630787 · Cited by 1,089
Evolution Strategies as a Scalable Alternative to Reinforcement Learning
Salimans, T., Ho, J., Chen, X., Sidor, S., & Sutskever, I. (2017) · arXiv · doi:10.48550/arxiv.1703.03864 · Cited by 1,027 · cited 1×
“We use antithetic sampling to generate δi similar to [35, 59].”
MethodMedia Forensics and DeepFakes: An Overview
Verdoliva, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.3002101 · Cited by 979 · cited 1×
“To address the threats imposed by Deepfakes, the machine learning community has proposed several countermeasures to identify forgeries in digital media [69].”
BackgroundVaccari, C., & Chadwick, A. (2020) · Social Media + Society · doi:10.1177/2056305120903408 · Cited by 978
Exposing DeepFake Videos By Detecting Face Warping Artifacts
Li, Y., & Lyu, S. (2018) · arXiv · doi:10.48550/arxiv.1811.00656 · Cited by 785 · cited 1×
“The state-of-the-art methods for detecting manipulated facial content in videos rely on Convolutional Neural Networks (CNNs) [1, 2, 21, 43, 56, 57].”
MethodFSGAN: Subject Agnostic Face Swapping and Reenactment
Nirkin, Y., Keller, Y., & Hassner, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00728 · Cited by 706
Two-Stream Neural Networks for Tampered Face Detection
Zhou, P., Han, X., Morariu, V. I., & Davis, L. S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.229 · Cited by 700
Few-Shot Adversarial Learning of Realistic Neural Talking Head Models
Zakharov, E., Shysheya, A., Burkov, E., & Lempitsky, V. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00955 · Cited by 687
Recurrent convolutional strategies for face manipulation detection in videos
Sabir, E., Cheng, J., Jaiswal, A., AbdAlmageed, W., Masi, I., & Natarajan, P. (2019) · arXiv · doi:10.48550/arxiv.1905.00582 · Cited by 443 · cited 1×
“Other works [1, 30, 41, 42, 57, 58] have focused on detecting face manipulation artifacts resulting from Deepfake generation methods.”
BackgroundThe deepfake detection challenge (dfdc) dataset
Dolhansky, B., Bitton, J., Pflaum, B., Lu, J., Howes, R., Wang, M., & Ferrer, C. C. (2020) · arXiv · doi:10.48550/arxiv.2006.07397 · Cited by 427
Self-citationDeepfake Video Detection through Optical Flow Based CNN
Amerini, I., Galteri, L., Caldelli, R., & Del Bimbo, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00152 · Cited by 424 · cited 2×
“The state-of-the-art methods for detecting manipulated facial content in videos rely on Convolutional Neural Networks (CNNs) [1, 2, 21, 43, 56, 57].”
MethodDeepFake Detection by Analyzing Convolutional Traces
Guarnera, L., Giudice, O., & Battiato, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00341 · Cited by 374 · cited 1×
“We demonstrate that the current works on Deepfake detectors [1, 12, 21, 29, 57, 65] have been majorly ignoring the issue of adversarial inputs and can be fairly easily bypassed.”
BackgroundAdversarial Deepfakes: Evaluating Vulnerability of Deepfake Detectors to Adversarial Examples
Hussain, S., Neekhara, P., Jere, M., Koushanfar, F., & McAuley, J. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00339 · Cited by 204
Self-citationCurls & Whey: Boosting Black-Box Adversarial Attacks
Shi, Y., Wang, S., & Han, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00668 · Cited by 103 · cited 1×
“Prior work has shown a series of irst-order gradient-based attacks to be fairly efective in fooling DNN based models in images [13, 28, 45, 51, 52, 61, 62], audio [14, 48, 54] and text [9, 26, 47] domains.”
BackgroundVideo Face Manipulation Detection Through Ensemble of CNNs
Bonettini, N., Cannas, E. D., Mandelli, S., Bondi, L., Bestagini, P., & Tubaro, S. (2021) · 2020 25th International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr48806.2021.9412711 · Cited by 27
Cited by
Deepfake Detection: A Comprehensive Survey from the Reliability Perspective
Wang, T., Liao, X., Chow, K. P., Lin, X., & Wang, Y. (2025) · ACM Computing Surveys · doi:10.1145/3699710 · Cited by 107
Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion
Agarwal, S., Hu, L., Ng, E., Darrell, T., Li, H., & Rohrbach, A. (2023) · 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv56688.2023.00469 · Cited by 28
Wani, T. M., Qadri, S. A. A., Wani, F. A., & Amerini, I. (2024) · Foundations and Trends® in Privacy and Security · doi:10.1561/3300000048 · Cited by 16
DeepFake detection in the AIGC era: A survey, benchmarks, and future perspectives
Xie, S., Qiao, T., Li, S., Zhang, X., Zhou, J., & Feng, G. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103740 · Cited by 14
D4: Detection of Adversarial Diffusion Deepfakes Using Disjoint Ensembles
Hooda, A., Mangaokar, N., Feng, R., Fawaz, K., Jha, S., & Prakash, A. (2024) · 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv57701.2024.00377 · Cited by 7
“Recent work has shown that these “ad-versarial deepfakes” can even be crafted in a black-box setting, where the attacker only has query access to the detector [23,30,31,66].”
Background
Metadata
- Authors
- Hussain, S., Neekhara, P., Dolhansky, B., Bitton, J., Ferrer, C. C., McAuley, J., & Koushanfar, F.
- Venue
- Digital Threats: Research and Practice
- Published
- 2022-02-07
- Volume
- 3
- Issue
- 3
- Pages
- 1-23
- ISSN
- 2576-5337 (ISSN-L)
- 2692-1626
- OpenAlex
- W3164267381
- Cited by
- 30
- References
- 26
- FWCI
- 1.59
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.