DEEPFAKER: A Unified Evaluation Platform for Facial Deepfake and Detection Models
Open accessThe DOI matches its Crossref record; 90 of the 96 references it cites were found in a registry or an index; 29 of the 34 sources its claims cite were checked for support.
Abstract
Deepfake data contains realistically manipulated faces—its abuses pose a huge threat to the security and privacy-critical applications. Intensive research from academia and industry has produced many deepfake/detection models, leading to a constant race of attack and defense. However, due to the lack of a unified evaluation platform, many critical questions on this subject remain largely unexplored. How is the anti-detection ability of the existing deepfake models? How generalizable are existing detection models against different deepfake samples? How effective are the detection APIs provided by the cloud-based vendors? How evasive and transferable are adversarial deepfakes in the lab and real-world environment? How do various factors impact the performance of deepfake and detection models? To bridge the gap, we design and implement DEEPFAKER 1 a unified and comprehensive deepfake detection evaluation platform. Specifically, DEEPFAKER has integrated 10 state-of-the-art deepfake methods and 9 representative detection methods, while providing a user-friendly interface and modular design that allows for easy integration of new methods. Leveraging DEEPFAKER , we conduct a large-scale empirical study of facial deepfake/detection models and draw a set of key findings: (i) the detection methods have poor generalization on samples generated by different deepfake methods; (ii) there is no significant correlation between anti-detection ability and visual quality of deepfake samples; (iii) the current detection APIs have poor detection performance and adversarial deepfakes can achieve about 70% attack success rate on all cloud-based vendors, calling for an urgent need to deploy effective and robust detection APIs; (iv) the detection methods in the lab are more robust against transfer attacks than the detection APIs in the real-world environment; and (v) deepfake videos may not always be more difficult to detect after video compression. We envision that DEEPFAKER will benefit future research on facial deepfake and detection.
Overview
CoChat Summary
TL;DR
DEEPFAKER is a unified, modular platform integrating 10 deepfake methods, 9 detection methods, and 3 cloud detection APIs to evaluate facial deepfake and detection models in lab and real-world settings. Using it, the authors find poor detection generalization, weak commercial API performance, and substantial vulnerability to adversarial deepfakes.
Key findings
- Detection methods generalize poorly to unseen deepfake samples: average performance across all detection methods is 50%–70% on all deepfake samples, and CViT scores 96.73% AUC on ICface but only 41.73% on FaceShifter.
- Anti-detection ability does not correlate significantly with visual quality: FaceShifter has a slightly stronger average anti-detection score than FOMM (51.12% vs. 53.64% AUC), despite FaceShifter videos showing more obvious visual artifacts.
- Cloud detection APIs perform poorly: FNRs exceed 60% for all APIs, Baidu misclassifies all fake data as real (FNR 100%), and adversarial deepfakes achieve about 70% ASR on all cloud-based vendors; on Tencent, original deepfake video FNR is 61.5%, while CViT-based adversarial video ASR is 93.2%.
- Lab detection methods are more robust against transfer attacks than real-world detection APIs, and compressed deepfake videos are not always harder to detect: FOMM videos become easier to detect under strong compression, while FSGAN CViT ACC drops from 89.43% at C0 to 65.97% at C40.
Methods
- DEEPFAKER integrates 10 SOTA deepfake methods (3 face swapping, 4 face reenactment, 3 attribute editing), 9 representative detection methods, and 3 commercial detection APIs (Baidu Cloud, Tencent Cloud, Deepware), using Docker containers and a modular Flask-based architecture.
- Evaluation data use CelebA-HQ source images and driving videos from FaceForensics++, Celeb-DF, and SiW-M; each swapping/reenactment method generates 3,000 fake videos (50 source images × 60 driving videos), and each attribute-editing method generates 1,000 fake images (200 identities × 5 attributes), yielding a dataset of 21,000 deepfake videos and 24,000 deepfake images.
- Main analyses use AUC for detection/deepfake performance, FNR and FPR for detection APIs, and ASR for adversarial robustness; adversarial deepfakes are generated with iterative gradient sign, FGSM, and PGD under white-box and black-box transfer settings, plus case studies on facial similarity, input quality, and video compression.
Limitations
- The paper states that DEEPFAKER integrates 10 deepfake methods and 9 detection models, which cannot cover all models due to evolving technology and incomplete open-source code, though its modular design supports new methods.
- The paper states that because many methods do not release training code, it uses open-source pre-trained models and cannot uniformly explore training-phase factors such as training dataset or model architecture.
- The paper states that it focuses on visual deepfake images and videos, not audio or text modalities, and plans future extension; an evident limitation is that the real-world API evaluation covers only three vendors and depends on their current returned results and recommended thresholds.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
17 Valid · 6 Not valid · 5 Not checked · 116 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“these videos generated by two subjects with the same identity are generally more challenging for detection [47].”
What the sources say
- Supportedcites [47] [47] Huy H Nguyen, Fuming Fang, Junichi Yamagishi, and Isao Echizen. 2019. Multi-task learning for detecting and segmenting manipulated facial images and videos. arXiv preprint arXiv:1906.06876 (2019). - source
Section 4.1 (p. 4) describes self-reenactment as using the same source and target video and calls it a more challenging benchmark than source-to-target reenactment. Section 4.2 (p. 5) says its images were harder to detect. The paper's comparison is specifically between these Face2Face datasets, rather than all videos involving subjects with the same identity. (LLM, conf=0.88)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“existing works have investigated the evasion and transferability of adversarial deepfakes in the lab setting”
What the sources say
- Supportedcites [6] [6] Nicholas Carlini and Hany Farid. 2020. Evading deepfake-image detectors with white-and black-box attacks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops . IEEE, 658ś659. - source
Section 4, “Black-Box Attack” (page not provided), describes testing whether adversarial images transfer between deepfake-image detectors in a controlled experiment. Section 6, “Conclusions” (page not provided), reports that both white-box and black-box attacks can make fake images evade detection. (LLM, conf=0.91)
- Supportedcites [46] [46] Paarth Neekhara, Brian Dolhansky, Joanna Bitton, and Cristian Canton Ferrer. 2021. Adversarial threats to deepfake detection: A practical perspective. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . IEEE, 923ś932. - source
The Introduction says the paper studies whether adversarial examples can fool DeepFake detectors and transfer across detection methods. Section 4.3 reports experimental transfer attacks against unseen detectors, while Section 2.3 discusses earlier work on evading them. These passages support the claim about investigating evasion and transferability in an experimental setting. (LLM, conf=0.89)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“They lack a comprehensive experimental evaluation to demonstrate the practical eicacy of state-of-the-art methods.”
What the sources say
- Partly supportedcites [68] [68] Ruben Tolosana, Ruben Vera-Rodriguez, Julian Fierrez, Aythami Morales, and Javier Ortega-Garcia. 2020. Deepfakes and beyond: A survey of face manipulation and fake detection. Information Fusion 64 (2020), 131ś148. - sourceExperimental — accuracy not yet measured
In the introductory discussion of other surveys (page not available), the paper says its review adds a more detailed account of detection benchmarks and their results. That supports a narrower claim about the detail of earlier surveys, but it does not establish that existing surveys generally lack a comprehensive experimental evaluation of state-of-the-art methods. (LLM, conf=0.76)
- Could not verifycites [70] [70] Luisa Verdoliva. 2020. Media forensics and deepfakes: an overview. IEEE Journal of Selected Topics in Signal Processing 14, 5 (2020), 910ś932. - sourceExperimental — accuracy not yet measured
The Datasets section (p. 12) calls for comparing methods across multiple datasets, and the Deep Learning Considerations section (p. 12) stresses the importance of experimental protocols. Neither says that existing surveys lack a comprehensive experimental evaluation. Searches for that assertion and paraphrases of it found no supporting passage in our copy. (LLM, conf=0.76)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“current benchmarks [37], such as FaceForensics++ [57] and Celeb-DF [36], primarily focus on their own datasets and evaluate various detection methods on these public datasets.”
What the sources say
- Supportedcites [37] [37] Chenhao Lin, Jingyi Deng, Pengbin Hu, Chao Shen, Qian Wang, and Qi Li. 2022. Towards Benchmarking and Evaluating Deepfake Detection. arXiv preprint arXiv:2203.02115 (2022). - source
Section 2.3 (Evaluation Metrics and Forensic Benchmarks) says FaceForensics++ evaluates six detection methods on its dataset and Celeb-DF evaluates ten methods using publicly released pretrained models. This supports the citing sentence’s description of dataset-focused benchmarks evaluating detection methods. (LLM, conf=0.90)
- Supportedcites [57] [57] Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. 2019. Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF International Conference on Computer Vision . IEEE, 1ś11. - source
Section 5 (Benchmark) describes a public benchmark built from the paper’s own videos and reports results for multiple detection methods. The Additional information section likewise says FaceForensics++ evaluates detection approaches on its generated dataset. This supports the claim as an example of such a benchmark, without establishing the broader trend across all benchmarks. (LLM, conf=0.92)
- Supportedcites [36] [36] Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. 2020. Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . IEEE, 3207ś3216. - source
The Introduction (p. 1) presents Celeb-DF as a dataset for evaluating DeepFake detection methods. Section 4 (p. 6) says the paper evaluates multiple detection methods using Celeb-DF and other existing datasets. Celeb-DF is a central focus, though the evaluation is not limited to its own dataset. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“the existing detection methods generally only evaluate their overall performance on publicly available datasets”
What the sources say
- Could not verifycites [57] [57] Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. 2019. Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF International Conference on Computer Vision . IEEE, 1ś11. - sourceExperimental — accuracy not yet measured
Section 4.2.2 reports the authors' evaluation of detection methods on their dataset, and Section 5 describes their public benchmark. Neither passage says that existing methods generally evaluate only overall performance on publicly available datasets. Searches for prior evaluation practices and that exclusivity claim found no supporting passage in our copy. (LLM, conf=0.70)
- Partly supportedcites [36] [36] Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. 2020. Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . IEEE, 3207ś3216. - sourceExperimental — accuracy not yet measured
Section 4 (p. 6) evaluates existing DeepFake detection methods across several datasets, and Section 4.2 (p. 6) measures their overall performance using frame-level AUC. These passages support evaluation on datasets, but do not establish that detection methods generally evaluate themselves only on publicly available datasets. (LLM, conf=0.78)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
81 more claims · 4 with their cited sources checked
Create a free account to see all 144 claims and to verify the 3 references still open.
- It may state two separate claims as one. 41
- Probably not a substantial claim. 25
- The quote only partly states this claim. 3
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 5.1 Case Study 1: Facial Similarity · p. 18
“we choose two representative deepfake methods (FOMM and SimSwap) from the face swapping and reenactment methods, respectively.”
§ 4.1 Overall Experimental Seting · p. 9
“Face Swapping : FSGAN [49], FaceShifter [31], SimSwap [8]; Face Reenactment : FSGAN [49], ICface [69], FOMM [62], MRAA [63]”
Citations
Cites
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267 · cited 1×
“To further demonstrate the functionality of DEEPFAKER as a comprehensive and practical evaluation platform, we present three case studies in this section.”
MethodA Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714 · cited 1×
“As shown in Figure 1(c), this task is to alter, add, or remove the attributes of the source images, such as changing hair, age, ethnicity, and gender [24, 73, 77].”
BackgroundExplaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381
Denoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712 · cited 2×
“Furthermore, Preechakul et al. [52] explored the potential of utilizing diffusion probability models (DPMs) [19] for representation learning.”
BackgroundIntriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
Deep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747
The use of the area under the ROC curve in the evaluation of machine learning algorithms
Bradley, A. P. (1997) · Pattern Recognition · doi:10.1016/s0031-3203(96)00142-2 · Cited by 7,217
Towards Deep Learning Models Resistant to Adversarial Attacks
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017) · arXiv · doi:10.48550/arxiv.1706.06083 · Cited by 6,602
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019
Adversarial examples in the physical world
Kurakin, A., Goodfellow, I. J., & Bengio, S. (2018) · Artificial Intelligence Safety and Security · doi:10.1201/9781351251389-8 · Cited by 5,375
Progressive Growing of GANs for Improved Quality, Stability, and Variation
Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017) · arXiv · doi:10.48550/arxiv.1710.10196 · Cited by 4,727
On Assessing ML Model Robustness: A Methodological Framework (Academic Track)
Awadid, A., & Robert, B. (2025) · OASIcs : OpenAccess Series in Informatics · doi:10.4230/oasics.saia.2024.1 · Cited by 4,702
StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., & Choo, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00916 · Cited by 4,163
StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation
Choi, Y., Choi, M.-J., Kim, M., Ha, J.-W., Kim, S., & Choo, J. (2017) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.48550/arxiv.1711.09020 · Cited by 4,057
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 9×
“Furthermore, we obtain facial images from the internet as source images and utilize each deepfake method to generate 3,000 fake videos for assessing the detection model’s generalization ability (the results are presented in Appendix A.3).”
BackgroundMethodInfluentialVGGFace2: A Dataset for Recognising Faces across Pose and Age
Cao, Q., Shen, L., Xie, W., Parkhi, O. M., & Zisserman, A. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00020 · Cited by 3,032
Denoising Diffusion Implicit Models
Song, J., Meng, C., & Ermon, S. (2020) · arXiv · doi:10.48550/arxiv.2010.02502 · Cited by 2,507
VoxCeleb: A Large-Scale Speaker Identification Dataset
Nagrani, A., Chung, J. S., & Zisserman, A. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-950 · Cited by 2,210
Pérez, P., Gangnet, M., & Blake, A. (2003) · ACM SIGGRAPH 2003 Papers · doi:10.1145/1201775.882269 · Cited by 2,020
MesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · International Workshop on Information Forensics and Security · doi:10.48550/arxiv.1809.00888 · Cited by 1,940
MesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics
Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00327 · Cited by 1,846 · cited 4×
“Second, current benchmarks [37], such as FaceForensics++ [57] and Celeb-DF [36], primarily focus on their own datasets and evaluate various detection methods on these public datasets.”
BackgroundMethodInfluentialPérez, P., Gangnet, M., & Blake, A. (2003) · ACM Transactions on Graphics · doi:10.1145/882262.882269 · Cited by 1,773
Face2Face: Real-Time Face Capture and Reenactment of RGB Videos
Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., & Niessner, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.262 · Cited by 1,747
Bulat, A., & Tzimiropoulos, G. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.116 · Cited by 1,464 · cited 1×
“For the face swapping and face reenactment methods, we randomly select 50 images from the source image dataset as the source images to provide identity information.”
Method(2008) · doi:10.1007/978-0-387-71041-9 · Cited by 1,406
Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
Papernot, N., McDaniel, P., & Goodfellow, I. (2016) · arXiv · doi:10.48550/arxiv.1605.07277 · Cited by 1,326
Deepfake Video Detection Using Recurrent Neural Networks
Guera, D., & Delp, E. J. (2018) · 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2018.8639163 · Cited by 1,285
Bulat, A., & Tzimiropoulos, G. (2017) · Nottingham ePrints (University of Nottingham) · Cited by 1,277
DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection
Tolosana, R., Vera-Rodríguez, R., Fierrez, J., Morales, A., & Ortega-García, J. (2020) · Information Fusion · doi:10.48550/arxiv.2001.00179 · Cited by 1,252
Deepfakes and beyond: A Survey of face manipulation and fake detection
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.06.014 · Cited by 1,237
Exposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y.-Z., & Lyu, S. (2018) · IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.48550/arxiv.1811.00661 · Cited by 1,224
Face X-Ray for More General Face Forgery Detection
Li, L., Bao, J., Zhang, T., Yang, H., Chen, D., Wen, F., & Guo, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00505 · Cited by 1,211
CNN-Generated Images Are Surprisingly Easy to Spot… for Now
Wang, S.-Y., Wang, O., Zhang, R., Owens, A., & Efros, A. A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00872 · Cited by 1,210
Exposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196
Thinking in Frequency: Face Forgery Detection by Mining Frequency-Aware Clues
Qian, Y., Yin, G., Sheng, L., Chen, Z., & Shao, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58610-2_6 · Cited by 1,058
Multi-attentional Deepfake Detection
Zhao, H., Wei, T., Zhou, W., Zhang, W., Chen, D., & Yu, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00222 · Cited by 1,019
Media Forensics and DeepFakes: An Overview
Verdoliva, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.3002101 · Cited by 979
Capsule-forensics: Using Capsule Networks to Detect Forged Images and Videos
Nguyen, H. H., Yamagishi, J., & Echizen, I. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682602 · Cited by 951
The Creation and Detection of Deepfakes: A Survey
Mirsky, Y., & Lee, W. (2022) · ACM Computing Surveys · doi:10.1145/3425780 · Cited by 863 · cited 2×
“For convenience, we summarize the abbreviations in Table 1.”
BackgroundThe Creation and Detection of Deepfakes: A Survey
Mirsky, Y., & Lee, W. (2020) · arXiv · doi:10.48550/arxiv.2004.11138 · Cited by 863
AttGAN: Facial attribute editing by only changing what you want
He, Z., Zuo, W., Kan, M., Shan, S., & Chen, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2916751 · Cited by 833 · cited 2×
“In the work of He et al. [18], instead of imposing the attribute-independent constraints on the latent representations [29], AttGAN imposes an attribute classification constraint to the generated images for correct attribute editing.”
BackgroundMethodExploiting Visual Artifacts to Expose Deepfakes and Face Manipulations
Matern, F., Riess, C., & Stamminger, M. (2019) · 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw.2019.00020 · Cited by 824
Exposing DeepFake Videos By Detecting Face Warping Artifacts
Li, Y., & Lyu, S. (2018) · arXiv · doi:10.48550/arxiv.1811.00656 · Cited by 785 · cited 1×
“Li and Lyu [35] found that the deepfake algorithm can only generate images with limited resolution, which would bring the face warping artifacts (FWA).”
BackgroundFSGAN: Subject Agnostic Face Swapping and Reenactment
Nirkin, Y., Keller, Y., & Hassner, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00728 · Cited by 706
Demo of Face2Face: real-time face capture and reenactment of RGB videos
Thies, J., Zollhöfer, M., Stamminger, M., Theobalt, C., & Nießner, M. (2016) · ACM SIGGRAPH 2016 Emerging Technologies · doi:10.1145/2929464.2929475 · Cited by 589
First Order Motion Model for Image Animation
Siarohin, A., Lathuilière, S., Tulyakov, S., Ricci, E., & Sebe, N. (2020) · arXiv · doi:10.48550/arxiv.2003.00196 · Cited by 585 · cited 2×
“Siarohin et al. [62] proposed a first-order motion model (FOMM), which supports complex motions using keypoint displacements and local affine transformations.”
MethodDiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation
Kim, G., Kwon, T., & Ye, J. C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00246 · Cited by 570 · cited 1×
“Based on the perfect inversion capability and high-quality image generation of the recent diffusion models [64], Kim et al. [26] proposed DiffusionCLIP.”
MethodMulti-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos
Nguyen, H. H., Fang, F., Yamagishi, J., & Echizen, I. (2019) · arXiv · doi:10.48550/arxiv.1906.06876 · Cited by 551
Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos
Nguyen, H. H., Fang, F., Yamagishi, J., & Echizen, I. (2019) · 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS) · doi:10.1109/btas46853.2019.9185974 · Cited by 549
DeepFakes: a New Threat to Face Recognition? Assessment and Detection
Korshunov, P., & Marcel, S. (2018) · arXiv · doi:10.48550/arxiv.1812.08685 · Cited by 497
Recurrent convolutional strategies for face manipulation detection in videos
Sabir, E., Cheng, J., Jaiswal, A., AbdAlmageed, W., Masi, I., & Natarajan, P. (2019) · arXiv · doi:10.48550/arxiv.1905.00582 · Cited by 443 · cited 1×
“Thethird category of detection methods takes the temporal features between frames into account [16, 58].”
BackgroundAttributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints
Yu, N., Davis, L., & Fritz, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00765 · Cited by 437
The deepfake detection challenge (dfdc) dataset
Dolhansky, B., Bitton, J., Pflaum, B., Lu, J., Howes, R., Wang, M., & Ferrer, C. C. (2020) · arXiv · doi:10.48550/arxiv.2006.07397 · Cited by 427
SimSwap: An Efficient Framework For High Fidelity Face Swapping
Chen, R., Chen, X., Ni, B., & Ge, Y. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413630 · Cited by 418 · cited 3×
“The anti-detection ability should be used as an important metric for evaluating the performance of the deepfake methods.”
MethodStyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation
Wu, Z., Lischinski, D., & Shechtman, E. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01267 · Cited by 389
FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces
Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Nießner, M. (2018) · arXiv · doi:10.48550/arxiv.1803.09179 · Cited by 339
Diffusion Autoencoders: Toward a Meaningful and Decodable Representation
Preechakul, K., Chatthee, N., Wizadwongsa, S., & Suwajanakorn, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01036 · Cited by 338 · cited 1×
“We conjecture that there is a gap between the detector and human vision, and it is uncertain which discriminative features can be captured by the detectors.”
BackgroundEmotions Don't Lie: An Audio-Visual Deepfake Detection Method using Affective Cues
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., & Manocha, D. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413570 · Cited by 319 · cited 1×
“We conjecture that there is a gap between the detector and human vision, and it is uncertain which discriminative features can be captured by the detectors.”
BackgroundDeep Tree Learning for Zero-Shot Face Anti-Spoofing
Liu, Y., Stehouwer, J., Jourabloo, A., & Liu, X. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00481 · Cited by 278
Fader Networks: Manipulating Images by Sliding Attributes
Lample, G., Zeghidour, N., Usunier, N., Bordes, A., Denoyer, L., & Ranzato, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.00409 · Cited by 271
Motion Representations for Articulated Animation
Siarohin, A., Woodford, O. J., Ren, J., Chai, M., & Tulyakov, S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01344 · Cited by 245
Can We Trust AI-Powered Real-Time Embedded Systems? (Invited Paper)
Buttazzo, G. (2022) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/oasics.ng-res.2022.1 · Cited by 240
FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping
Li, L., Bao, J., Yang, H., Chen, D., & Wen, F. (2019) · arXiv · doi:10.48550/arxiv.1912.13457 · Cited by 236
Adversarial Deepfakes: Evaluating Vulnerability of Deepfake Detectors to Adversarial Examples
Hussain, S., Neekhara, P., Jere, M., Koushanfar, F., & McAuley, J. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00339 · Cited by 204
Image-to-image Translation via Hierarchical Style Disentanglement
Li, X., Zhang, S., Hu, J., Cao, L., Hong, X., Mao, X., Huang, F., Wu, Y., & Ji, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00853 · Cited by 204
Countering Malicious DeepFakes: Survey, Battleground, and HorizonPreprint
Juefei-Xu, F., Wang, R., Huang, Y., Guo, Q., Ma, L., & Liu, Y. (2021) · International Journal of Computer Vision · doi:10.48550/arxiv.2103.00218 · Cited by 203
A general framework for adversarial examples with objectives
Sharif, M., Bhagavatula, S., Bauer, L., & Reiter, M. K. (2019) · ACM Transactions on Privacy and Security · doi:10.1145/3317611 · Cited by 200 · cited 1×
“However, the advanced detection methods mainly rely on DNN-based binary classification models, which are known to be vulnerable to adversarial examples [9, 38, 61, 66].”
MethodVGGFace2: A dataset for recognising faces across pose and agePreprint
Cao, Q., Shen, L., Xie, W., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.08092 · Cited by 192
Countering Malicious DeepFakes: Survey, Battleground, and Horizon
Juefei-Xu, F., Wang, R., Huang, Y., Guo, Q., Ma, L., & Liu, Y. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01606-8 · Cited by 191 · cited 2×
“Since the attribute editing methods can only generate fake images, we choose the detection meth-ods whose open source codes can directly process the images for evaluation.”
BackgroundMethodImproving the Efficiency and Robustness of Deepfakes Detection through Precise Geometric Features
Sun, Z., Han, Y., Hua, Z., Ruan, N., & Jia, W. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00361 · Cited by 180
Evading Deepfake-Image Detectors with White- and Black-Box Attacks
Carlini, N., & Farid, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00337 · Cited by 170
ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis
He, Y., Gan, B., Chen, S., Zhou, Y., Yin, G., Song, L., Sheng, L., Shao, J., & Liu, Z. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00434 · Cited by 161 · cited 1×
“We conjecture that there is a gap between the detector and human vision, and it is uncertain which discriminative features can be captured by the detectors.”
BackgroundDEEPSEC: A Uniform Platform for Security Analysis of Deep Learning Model
Ling, X., Ji, S., Zou, J., Wang, J., Wu, C., Li, B., & Wang, T. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00023 · Cited by 152
Self-citationAdversarial Perturbations Fool Deepfake Detectors
Gandhi, A., & Jain, S. (2020) · 2020 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207034 · Cited by 149
DeepFake video detection using convolutional vision transformer
Wodajo, D., & Atnafu, S. (2021) · arXiv · doi:10.48550/arxiv.2102.11126 · Cited by 147
Adversarial Threats to DeepFake Detection: A Practical Perspective
Neekhara, P., Dolhansky, B., Bitton, J., & Ferrer, C. C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00103 · Cited by 117
FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset
Khalid, H., Tariq, S., Kim, M., & Woo, S. S. (2021) · arXiv · doi:10.48550/arxiv.2108.05080 · Cited by 116
DefakeHop: A Light-Weight High-Performance Deepfake Detector
Chen, H.-S., Rouhsedaghat, M., Ghani, H., Hu, S., You, S., & Jay Kuo, C.-C. (2021) · 2021 IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme51207.2021.9428361 · Cited by 100
ICface: Interpretable and Controllable Face Reenactment Using GANs
Tripathy, S., Kannala, J., & Rahtu, E. (2020) · 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv45572.2020.9093474 · Cited by 88 · cited 3×
“The state-of-the-art deepfake methods can create fake videos in real time and only requires a single face image of the victim [69].”
BackgroundMethodDeep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.7766 · Cited by 76
FENCE: Feasible Evasion Attacks on Neural Networks in Constrained Environments
Chernikova, A., & Oprea, A. (2022) · ACM Transactions on Privacy and Security · doi:10.1145/3544746 · Cited by 52
CNN-generated images are surprisingly easy to spot... for nowPreprint
Wang, S.-Y., Wang, O., Zhang, R., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.11035 · Cited by 44
Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware CluesPreprint
Qian, Y.-Y., Yin, G., Sheng, L., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.09355 · Cited by 42
Qin, L., Peng, F., Long, M., Ramachandra, R., & Busch, C. (2022) · ACM Transactions on Privacy and Security · doi:10.1145/3491199 · Cited by 14
Seeing is Living? Rethinking the Security of Facial Liveness Verification in the Deepfake Era
Li, C., Wang, L., Ji, S., Zhang, X., Xi, Z., Guo, S., & Wang, T. (2022) · arXiv · doi:10.48550/arxiv.2202.10673 · Cited by 14
Self-citationTowards Benchmarking and Evaluating Deepfake Detection
Lin, C., Deng, J., Hu, P., Shen, C., Wang, Q., & Li, Q. (2022) · arXiv · doi:10.48550/arxiv.2203.02115 · Cited by 5
VoxCeleb: a large-scale speaker identification dataset
Nagrani, A., Chung, J. S., & Zisserman, A. (2017) · arXiv · doi:10.48550/arxiv.1706.08612
Research works printed but not linked: 3
- Ivana Chingovska, André Anjos, and Sébastien Marcel. 2012. On the effectiveness of local binary patterns in face anti-spoofing. In Proceedings of the International Conference of the Biometrics Special Interest Group (BIOSIG’12). IEEE, Los Alamitos, CA, 1–7.
- Paul Ekman and Wallace V. Friesen. 1978. FacialActionCodingSystem:A Technique for the Measurement of FacialMovement. Consulting Psychologists Press Palo Alto CA.
- 10.5555/1324787 · doi:10.5555/1324787
Cited by
A Comprehensive Review of DeepFake Detection Using Advanced Machine Learning and Fusion Methods
Gupta, G., Raja, K., Gupta, M., Jan, T., Whiteside, S. T., & Prasad, M. (2023) · Electronics · doi:10.3390/electronics13010095 · Cited by 99
Video and Audio Deepfake Datasets and Open Issues in Deepfake Technology: Being Ahead of the Curve
Akhtar, Z., Pendyala, T. L., & Athmakuri, V. S. (2024) · Forensic Sciences · doi:10.3390/forensicsci4030021 · Cited by 60
Khimi, W., Albarqi, K., Saif, K., & Elhag, S. (2024) · International Journal of Computers and Informatics · doi:10.59992/ijci.2024.v3n8p3 · Cited by 11
Dong, Y., Wang, L., Li, Z., Li, H., Tang, P., Hu, C., & Guo, S. (2024) · ACM Transactions on Privacy and Security · doi:10.1145/3705611 · Cited by 14
“And AEs have been demonstrated the potential to pose significant security risks to deep learning models operating in various domains [12, 13, 23, 24, 33, 43, 52, 63, 65, 70, 77, 79].”
Self-citationAdaptive Network Intrusion Detection Using Reinforcement Learning with Proximal Policy Optimization
Suresh, A., & Jose, A. C. (2025) · ACM Transactions on Privacy and Security · doi:10.1145/3764586 · Cited by 19
“Stacked ensembles, in particular, enhance decision-making by using meta-learners [24], often MLP [66], to aggregate predictions from base models, creating a more accurate and adaptable framework.”
Metadata
- Authors
- Wang, L., Meng, X., Li, D., Zhang, X., Ji, S., & Guo, S.
- Affiliations
- Shandong University (CN)
- China Electronic Product Reliability and Environmental Test Institute (CN)
- Zhejiang University (CN)
- Venue
- ACM Transactions on Privacy and Security
- Published
- 2024-02-05
- Volume
- 27
- Issue
- 1
- Pages
- 1-34
- ISSN
- 2471-2566 (ISSN-L)
- 2471-2574
- OpenAlex
- W4389140876
- Cited by
- 11
- References
- 96
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 3
- FWCI
- 1.00
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Transactions on Privacy and Security (publishedVersion)
- Funders
- National Natural Science Foundation of China (61902148)
- Shandong Provincial Natural Science Foundation (ZR2020MF055, ZR2021LZH007, ZR2020LZH002, ZR2020QF045)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Signal Processing › Biometric Identification and Security
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Keywords
- deepfake generation, evaluation platform, transfer attacks, video compression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-12, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-12 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.