Assessment framework for deepfake detection in real-world situations
Open accessThe DOI matches its Crossref record; 71 of the 74 references it cites were found in a registry or an index, and 1 are not checked yet; 6 of the 9 sources its claims cite were checked for support.
Abstract
Abstract Detecting digital face manipulation in images and video has attracted extensive attention due to the potential risk to public trust. To counteract the malicious usage of such techniques, deep learning-based deepfake detection methods have been employed and have exhibited remarkable performance. However, the performance of such detectors is often assessed on related benchmarks that hardly reflect real-world situations. For example, the impact of various image and video processing operations and typical workflow distortions on detection accuracy has not been systematically measured. In this paper, a more reliable assessment framework is proposed to evaluate the performance of learning-based deepfake detectors in more realistic settings. To the best of our acknowledgment, it is the first systematic assessment approach for deepfake detectors that not only reports the general performance under real-world conditions but also quantitatively measures their robustness toward different processing operations. To demonstrate the effectiveness and usage of the framework, extensive experiments and detailed analysis of four popular deepfake detection methods are further presented in this paper. In addition, a stochastic degradation-based data augmentation method driven by realistic processing operations is designed, which significantly improves the robustness of deepfake detectors.
Verified 2026-11-20 · 1 reference not checked yet
Overview
CoChat Summary
## TL;DR
The paper proposes a systematic assessment framework to measure deepfake detector robustness in realistic image and video conditions, rather than only on standard benchmark data. It also introduces stochastic degradation-based augmentation (SDAug), which the paper reports improves robustness against many real-world distortions.
## Key findings
- On realistic image deepfakes, even mild processing operations caused noticeable drops in detection accuracy; the paper says Capsule-Forensics and XceptionNet showed “severe performance deterioration” on modified data, and that UIA-VIT also degraded under processing operations.
- On realistic video deepfakes, Capsule-Forensics and XceptionNet performed poorly on pre-processed video deepfakes, while SBIs and UIA-VIT were more stable; however, none of the three methods could properly classify heavily compressed, low-resolution, or noisy video deepfakes.
- In cross-manipulation evaluation on Celeb-DFv1 and Celeb-DFv2 after training on FFpp, Capsule rose from AUC 43.36/54.39 with no augmentation to 74.84/71.86 with SDAug; XceptionNet rose from 39.35/50.00 to 80.67/73.88 with SDAug.
- The paper reports that SDAug improved robustness while maintaining high performance on original data, and that it outperformed standard Augmix and the non-stochastic degradation-based variant (DAug) in the reported experiments.
## Methods
- The paper proposes an assessment framework that trains deepfake detectors on their original datasets, then creates multiple distorted copies of the test set by applying one processing operation at a time at different severity levels; performance is measured with AUC from ROC curves.
- Data used: FaceForensics++ (FFpp; 1,000 pristine and 4,000 manipulated videos, with C23 and C40 compressed versions) and Celeb-DFv2 (590 pristine celebrity videos and 5,639 fake videos); evaluation focuses on unaltered and distorted variants.
- Models evaluated include Capsule-Forensics, XceptionNet, UIA-VIT, and SBIs; for augmentation experiments, the paper compares No Aug, Augmix, DAug, and SDAug.
## Limitations
- Stated by the paper: experiments are mainly limited to video deepfakes or standard-quality image deepfakes; HD single-image deepfakes, including those created by GANs and diffusion models, were not evaluated with the framework.
- Stated by the paper: the SDAug chain is hand-designed and its hyperparameters may not be optimal; the authors suggest AutoML-based parameter search.
- Stated by the paper: SDAug appears to help SBIs less than the other models because SBIs relies entirely on synthetic data and augmentation may corrupt its manually designed forgery traces.
- Evident from the text: several tables in the excerpt are partially garbled/truncated, so some detailed numeric results are not fully recoverable from the provided text.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 11 Not valid · 2 Not checked · 73 Own results and interpretations
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Although the bmshj -based compression method is capable of achieving lower bitrates than JPEG, it brings significant negative impact to both detectors, whose predictions are close to random guess regardless of the select compression factor.”
What the sources say
- Could not verifycites [60] 60. J. Ballé, D. Minnen, S. Singh, S.J. Hwang, N. Johnston, Variational image compression with a scale hyperprior. In: International Conference on Learning Representations (2018) sourceExperimental — accuracy not yet measured
Our copy of Ballé et al. (2018) evaluates only rate-distortion: §4.1 (Experimental Setup) describes 32 models trained on MSE or MS-SSIM losses, and §4.2 (Experimental Results) reports PSNR/MS-SSIM curves on Kodak and Tecnick versus JPEG, BPG, Theis, Toderici, Rippel and Johnston. No passage anywhere in the paper tests object detectors or any downstream vision model, and the closest statement — 'neither distortion metric is sophisticated enough to capture image semantics' (§4.2) — concerns perceptual metrics, not detector accuracy, so the near-random-predictions result cannot be checked against this source. The concessive half of the sentence (the method reaching lower bitrates than JPEG) is consistent with §4.2, but the detector impact attributed here is the citing authors' own experiment. Searched: object detection, downstream task, classifier/detector accuracy, semantic content preservation; the copy is complete (not truncated). (LLM, conf=0.70)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“On the other hand, the resizing operation, more specifically the down-sampling operation, can significantly reduce the performance of modern deep learningbased detectors [58, 59] due to a loss of discriminative information.”
What the sources say
- Partly supportedcites [58] 58. T. Marciniak, A. Chmielewska, R. Weychan, M. Parzych, A. Dabrowski, Influence of low resolution of images on reliability of face detection and recognition. Multimed. Tools Appl. 74 (2013). https:// doi. org/ 10. 1007/ s11042- 013- 1568-8 sourceExperimental — accuracy not yet measured
Section 5.2 (p. 9) reports that eightfold down-sampling reduced face-detection efficiency by about 10–20% for certain head positions. But the detector tested was Haar-like, not a modern deep-learning detector (section 5.1, p. 6), and the cited passages do not establish loss of discriminative information as the cause. (LLM, conf=0.88)
- Partly supportedcites [59] 59. P. Li, L. Prieto, D. Mery, P .J. Flynn, On low-resolution face recognition in the wild: comparisons and new techniques. IEEE Trans. Inf. Forensics Secur. 14 , 2000-2012 (2019) sourceExperimental — accuracy not yet measured
In “A. Super-resolution Techniques” (p. 4), the paper reports that face-matching performance decreases as downsampled resolution decreases. “A. Low-Resolution Face Recognition” (p. 2) links low-resolution images to a lack of information for robust feature representation. These passages support the general effect of information loss on recognition, but the paper does not establish the claimed performance reduction for deep learning-based detectors. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“although the SBIs method demonstrates a good generalization ability in cross-dataset experiments in their paper [21], our assessment framework shows that it is susceptible to some common perturbations in the real world, such as video compression, video noise, and low resolution.”
What the sources say
- Supportedcites [21] 21. K. Shiohara, T. Yamasaki, Detecting deepfakes with self-blended images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18720-18729 (2022) source
The citation is attached to the claim that SBIs generalized well in the source paper’s cross-dataset experiments. Table 1 (Implementation Details, p. 4) reports that SBIs outperformed prior methods on four of five test datasets and ranked second on the fifth. The statement about susceptibility to compression, noise, and low resolution is explicitly attributed to the citing authors’ own assessment, not to this source. (LLM, conf=0.92)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“On the contrary, both tested detectors are more robust to test data compressed using hific codec than using JPEG operation or bmshj codec, even with extremely low bitrates.”
What the sources say
- Could not verifycites [61] 61. F. Mentzer, G.D. Toderici, M. Tschannen, E. Agustsson, High-fidelity generative image compression. Adv. Neural. Inf. Process. Syst. 33 , 11913-11924 (2020) sourceExperimental — accuracy not yet measured
Our copy of the HiFiC paper evaluates only perceptual and distortion quality: the user study and rate-distortion-perception curves (Sec. 5.1, Fig. 3; Metrics section; Appendix A.3, A.6, B.2) compare HiFiC against BPG, M&S Hyperprior and a no-GAN baseline. No passage addresses 'detectors' or their robustness to compressed test data, and no codec named 'bmshj' appears anywhere (searched: detector robustness/downstream classifier accuracy, JPEG vs. codec robustness comparisons, BMShJ, extremely low bitrates). The nearest text is the comparison to Agustsson et al. at 'extremely low' bitrates (A.3), which concerns reconstruction quality, not detector performance, so the claim cannot be checked against this source. (LLM, conf=0.70)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, when the previous two methods are trained directly on compressed data, they maintain higher robustness toward multiple processing operations and even outperform the SBIs method, whose overall score even decreases by 0.66% instead.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
50 more claims
Create a free account to see all 88 claims and every source verdict.
- It may state two separate claims as one. 13
- The quote only partly states this claim. 2
- Probably not a substantial claim. 20
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 2×
“In [47], Hendrycks et al. presented a corrupted version of ImageNet [48] to benchmark the robustness of image recognition models against common image manipulations.”
BackgroundMethodXception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · cited 2×
“The two detectors present exceptional performance on unaltered FFpp and CelebDFv2 testing data as expected, but then show severe performance deterioration on all kinds of modified data from the assessment framework, which indicates a lack of robustness.”
BackgroundEfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714 · cited 1×
“For example, the recent advancement of generative adversarial networks (GANs) [1, 2, 3] has made it possible to generate realistic forged contents that are difficult for humans to distinguish from their authentic counterparts.”
BackgroundBeyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662206 · Cited by 8,983 · cited 1×
“The assessment framework takes this situation into consideration and measures the performance of a detector under different brightness conditions.”
BackgroundAnalyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00813 · Cited by 6,332 · cited 1×
“For example, the recent advancement of generative adversarial networks (GANs) [1, 2, 3] has made it possible to generate realistic forged contents that are difficult for humans to distinguish from their authentic counterparts.”
BackgroundAnalyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2019) · arXiv · doi:10.48550/arxiv.1912.04958 · Cited by 6,332
Progressive Growing of GANs for Improved Quality, Stability, and Variation
Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017) · arXiv · doi:10.48550/arxiv.1710.10196 · Cited by 4,727
Dynamic Routing Between Capsules
Sabour, S., Frosst, N., & Hinton, G. E. (2017) · arXiv · doi:10.48550/arxiv.1710.09829 · Cited by 4,167
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 8×
“For both methods, 100 frames are randomly sampled from each video for training purposes and 32 frames are extracted for validation and testing.”
BackgroundMethodInfluentialDlib-ml: A Machine Learning Toolkit
King, D. E. (2009) · doi:10.5555/1577069.1755843 · Cited by 2,777
ImageNet: A large-scale hierarchical image database
Deng, J. H., Dong, W. F., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206848 · Cited by 2,714
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics
Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00327 · Cited by 1,846 · cited 2×
“Only 8 frames per video are sampled for training while 32 frames are for validation and testing.”
BackgroundMethodDeferred neural rendering: image synthesis using neural textures
Thies, J., Zollhöfer, M., & Nießner, M. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323035 · Cited by 1,356
Variational image compression with a scale hyperprior
Ballé, J., Minnen, D. C., Singh, S., Hwang, S. J., & Johnston, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.01436 · Cited by 1,318
Deepfake Video Detection Using Recurrent Neural Networks
Guera, D., & Delp, E. J. (2018) · 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2018.8639163 · Cited by 1,285 · cited 1×
“In this paper, three different types of deepfake detectors [4, 11, 21] are adopted for experiments.”
Face X-Ray for More General Face Forgery Detection
Li, L., Bao, J., Zhang, T., Yang, H., Chen, D., Wen, F., & Guo, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00505 · Cited by 1,211 · cited 1×
“In this paper, three different types of deepfake detectors [4, 11, 21] are adopted for experiments.”
MethodExposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196 · cited 1×
“Overall log loss Selim Seferbekov [42] 0.4279 WM [43] 0.4284 NTechLab [44] 0.4345 Eighteen Years Old [45] 0.4347 The Medics [46] 0.4371 representations for deepfake detection.”
BackgroundInfluentialThinking in Frequency: Face Forgery Detection by Mining Frequency-Aware Clues
Qian, Y., Yin, G., Sheng, L., Chen, Z., & Shao, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58610-2_6 · Cited by 1,058 · cited 2×
“with the advanced deep learning techniques and large-scale datasets, numerous detection methods [4, 11, 12, 13, 14, 15, 16] have been published and have reported promising results on different datasets.”
BackgroundMulti-attentional Deepfake Detection
Zhao, H., Wei, T., Zhou, W., Zhang, W., Chen, D., & Yu, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00222 · Cited by 1,019 · cited 2×
“with the advanced deep learning techniques and large-scale datasets, numerous detection methods [4, 11, 12, 13, 14, 15, 16] have been published and have reported promising results on different datasets.”
BackgroundCapsule-forensics: Using Capsule Networks to Detect Forged Images and Videos
Nguyen, H. H., Yamagishi, J., & Echizen, I. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682602 · Cited by 951
Practical Poissonian-Gaussian Noise Modeling and Fitting for Single-Image Raw-Data
Foi, A., Trimeche, M., Katkovnik, V., & Egiazarian, K. (2008) · IEEE Transactions on Image Processing · doi:10.1109/tip.2008.2001399 · Cited by 928 · cited 1×
“However, the possible side effect of lossy compression artifacts on deep learning-based detectors has not been sufficiently studied.”
Attacks on Copyright Marking Systems
Petitcolas, F. A. P., Anderson, R. J., & Kuhn, M. G. (1998) · Lecture Notes in Computer Science · doi:10.1007/3-540-49380-8_16 · Cited by 835 · cited 1×
“In the media forensics community, StirMark [54] tested the robustness of image watermarking algorithms.”
Understanding how image quality affects deep neural networks
Dodge, S., & Karam, L. (2016) · 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2016.7498955 · Cited by 835 · cited 2×
“In fact, it has long been shown that DCNN-based methods are vulnerable to real-world perturbations and processing operations [22, 23, 24] in different vision tasks.”
On the Detection of Digital Face Manipulation
Dang, H., Liu, F., Stehouwer, J., Liu, X., & Jain, A. K. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00582 · Cited by 764 · cited 1×
“al [39] proposed a detection system based on an attention mechanism.”
BackgroundFSGAN: Subject Agnostic Face Swapping and Reenactment
Nirkin, Y., Keller, Y., & Hassner, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00728 · Cited by 706 · cited 1×
“In particular, current deep learning-based face manipulation techniques [4, 5, 6, 7] are capable of changing the expression, attributes, and even identity of a human face image, the outcome of which refers to the popular term ‘Deepfake’.”
BackgroundTwo-Stream Neural Networks for Tampered Face Detection
Zhou, P., Han, X., Morariu, V. I., & Davis, L. S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.229 · Cited by 700 · cited 1×
“For example, Xray [41] and SBIs [21] meth-ods manually generate blended faces during the training process as fake samples, which reproduce the blending artifacts existing in real-world GAN-synthesized deep-fakes.”
BackgroundFew-Shot Adversarial Learning of Realistic Neural Talking Head Models
Zakharov, E., Shysheya, A., Burkov, E., & Lempitsky, V. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00955 · Cited by 687 · cited 1×
“In particular, current deep learning-based face manipulation techniques [4, 5, 6, 7] are capable of changing the expression, attributes, and even identity of a human face image, the outcome of which refers to the popular term ‘Deepfake’.”
BackgroundGeneralizing Face Forgery Detection with High-frequency Features
Luo, Y., Zhang, Y., Yan, J., & Liu, W. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01605 · Cited by 609 · cited 2×
“with the advanced deep learning techniques and large-scale datasets, numerous detection methods [4, 11, 12, 13, 14, 15, 16] have been published and have reported promising results on different datasets.”
BackgroundSpatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain
Liu, H., Li, X., Zhou, W., Chen, Y., He, Y., Xue, H., Zhang, W., & Yu, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00083 · Cited by 600 · cited 2×
“with the advanced deep learning techniques and large-scale datasets, numerous detection methods [4, 11, 12, 13, 14, 15, 16] have been published and have reported promising results on different datasets.”
BackgroundDeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection
Jiang, L., Li, R., Wu, W., Qian, C., & Loy, C. C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00296 · Cited by 599 · cited 4×
“Some benchmarks, such as [8, 9], attempt to measure the performance of deepfake detectors under more realistic”
BackgroundMethodInfluentialACDC: The Adverse Conditions Dataset with Correspondences for Semantic Driving Scene Understanding
Sakaridis, C., Dai, D., & Van Gool, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01059 · Cited by 572 · cited 1×
“[49, 50, 51] focused on a safety-critical task, autonomous driving, and provided a robustness benchmark for various relevant vision tasks, such as object detection and semantic segmentation.”
BackgroundMulti-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos
Nguyen, H. H., Fang, F., Yamagishi, J., & Echizen, I. (2019) · arXiv · doi:10.48550/arxiv.1906.06876 · Cited by 551
Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos
Nguyen, H. H., Fang, F., Yamagishi, J., & Echizen, I. (2019) · 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS) · doi:10.1109/btas46853.2019.9185974 · Cited by 549 · cited 1×
“In this paper, three different types of deepfake detectors [4, 11, 21] are adopted for experiments.”
MethodXception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2016) · arXiv · doi:10.48550/arxiv.1610.02357 · Cited by 538
Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection
Haliassos, A., Vougioukas, K., Petridis, S., & Pantic, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00500 · Cited by 537 · cited 1×
“Therefore, cross-dataset evaluation has become an important step in recent studies to better show the advantages of deepfake detection methods, encouraging researchers [19, 20, 21] to propose detection methods with better generalization ability to different types of manipulations.”
BackgroundSharpness-Aware Minimization for Efficiently Improving Generalization
Foret, P., Kleiner, A., Mobahi, H., & Neyshabur, B. (2020) · arXiv · doi:10.48550/arxiv.2010.01412 · Cited by 521 · cited 1×
“However, when the previous two methods are trained directly on compressed data, they maintain higher robustness toward multiple processing operations and even outperform the SBIs method, whose overall score even decreases by 0.66% instead.”
MethodInfluentialDetecting Deepfakes with Self-Blended Images
Shiohara, K., & Yamasaki, T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01816 · Cited by 503 · cited 4×
“Moreover, among the considered influencing factors, noise and blurriness effects on images are the most prominent for deepfake detectors.”
BackgroundMethodInfluentialTwo-Branch Recurrent Network for Isolating Deepfakes in Videos
Masi, I., Killekar, A., Mascarenhas, R. M., Gurudatt, S. P., & AbdAlmageed, W. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58571-6_39 · Cited by 471 · cited 1×
“In this paper, three different types of deepfake detectors [4, 11, 21] are adopted for experiments.”
MethodRecurrent convolutional strategies for face manipulation detection in videos
Sabir, E., Cheng, J., Jaiswal, A., AbdAlmageed, W., Masi, I., & Natarajan, P. (2019) · arXiv · doi:10.48550/arxiv.1905.00582 · Cited by 443 · cited 1×
“Some video deepfake detectors [32, 33, 34] leveraged recurrent convolutional neural networks to track forgery clues from the temporal sequences.”
BackgroundLi, J., Xie, H., Li, J., Wang, Z., & Zhang, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00639 · Cited by 357 · cited 3×
“with the advanced deep learning techniques and large-scale datasets, numerous detection methods [4, 11, 12, 13, 14, 15, 16] have been published and have reported promising results on different datasets.”
BackgroundMethodDeepVision: Deepfakes Detection Using Human Eye Blinking Pattern
Jung, T., Kim, S., & Kim, K. (2020) · IEEE Access · doi:10.1109/access.2020.2988660 · Cited by 326 · cited 1×
“For example, Xray [41] and SBIs [21] meth-ods manually generate blended faces during the training process as fake samples, which reproduce the blending artifacts existing in real-world GAN-synthesized deep-fakes.”
BackgroundBenchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming
Michaelis, C., Mitzkus, B., Geirhos, R., Rusak, E., Bringmann, O., Ecker, A. S., Bethge, M., & Brendel, W. (2019) · arXiv · doi:10.48550/arxiv.1907.07484 · Cited by 324
Exploring Temporal Coherence for More General Video Face Forgery Detection
Zheng, Y., Bao, J., Chen, D., Zeng, M., & Wen, F. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01477 · Cited by 320 · cited 1×
“In this paper, three different types of deepfake detectors [4, 11, 21] are adopted for experiments.”
MethodZhuang, W., Chu, Q., Tan, Z., Liu, Q., Yuan, H., Miao, C., Luo, Z., & Yu, N. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20065-6_23 · Cited by 225
Strengths and weaknesses of deep learning models for face recognition against image degradations
Grm, K., Štruc, V., Artiges, A., Caron, M., & Ekenel, H. K. (2018) · IET Biometrics · doi:10.1049/iet-bmt.2017.0083 · Cited by 216 · cited 2×
“Similar work has been done for face recognition tasks, [52, 23, 24, 53] analyzed the robustness of CNN-based face recognition models towards face variations caused by illumination change, occlusion, and standard image processing operations.”
BackgroundBenchmarking the Robustness of Semantic Segmentation Models
Kamann, C., & Rother, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00885 · Cited by 212
High-Fidelity Generative Image Compression
Mentzer, F., Toderici, G. D., Tschannen, M., & Agustsson, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.09965 · Cited by 206
On the Generalization of GAN Image Forensics
Xuan, X., Peng, B., Wang, W., & Dong, J. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-31456-9_15 · Cited by 184 · cited 1×
“C V ] 12 A p r 2023 with the advanced deep learning techniques and large-scale datasets, numerous detection methods [4, 11, 12, 13, 14, 15, 16] have been published and have reported promising results on different datasets.”
BackgroundDeepfakes Detection with Automatic Face Weighting
Montserrat, D. M., Hao, H., Yarlagadda, S. K., Baireddy, S., Shao, R., Horvath, J., Bartusiak, E., Yang, J., Guera, D., Zhu, F., et al. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00342 · Cited by 181 · cited 1×
“In this paper, three different types of deepfake detectors [4, 11, 21] are adopted for experiments.”
MethodOn Low-Resolution Face Recognition in the Wild: Comparisons and New Techniques
Li, P., Prieto, L., Mery, D., & Flynn, P. J. (2019) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2018.2890812 · Cited by 171 · cited 1×
“Flip: Flipping a video horizontally describes the creation of a mirror video of the original footage.”
BackgroundFake Face Detection Methods: Can They Be Generalized?
Khodabakhsh, A., Ramachandra, R., Raja, K., Wasnik, P., & Busch, C. (2018) · 2018 International Conference of the Biometrics Special Interest Group (BIOSIG) · doi:10.23919/biosig.2018.8553251 · Cited by 151 · cited 1×
“C V ] 12 A p r 2023 with the advanced deep learning techniques and large-scale datasets, numerous detection methods [4, 11, 12, 13, 14, 15, 16] have been published and have reported promising results on different datasets.”
BackgroundZhuang, W., Chu, Q., Tan, Z., Liu, Q., Yuan, H.-J., Miao, C.-T., Luo, Z., & Yu, N.-H. (2022) · European Conference on Computer Vision · doi:10.48550/arxiv.2210.12752 · Cited by 147
ALASKA#2: Challenging Academic Research on Steganalysis with Realistic Images
Cogranne, R., Giboulot, E., & Bas, P. (2020) · IEEE International Workshop on Information Forensics and Security · doi:10.1109/wifs49906.2020.9360896 · Cited by 128 · cited 1×
“The ALASKA#2 dataset [55] was created following a careful evaluation of ISO parameters, JPEG compression, and noise level on FlickR images, etc., in order to help researchers in designing way more general and robust steganographic and steganalysis methods.”
MethodUse of a Capsule Network to Detect Fake Images and Videos
Nguyen, H. H., Yamagishi, J., & Echizen, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12467 · Cited by 121
FReTAL: Generalizing Deepfake Detection using Knowledge Distillation and Representation Learning
Kim, M., Tariq, S., & Woo, S. S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00111 · Cited by 118 · cited 1×
“Therefore, cross-dataset evaluation has become an important step in recent studies to better show the advantages of deepfake detection methods, encouraging researchers [19, 20, 21] to propose detection methods with better generalization ability to different types of manipulations.”
BackgroundTowards Generalizable Deepfake Detection with Locality-aware AutoEncoder
Du, M., Pentyala, S., Li, Y., & Hu, X. (2020) · Proceedings of the 29th ACM International Conference on Information & Knowledge Management · doi:10.1145/3340531.3411892 · Cited by 114 · cited 1×
“Other creative attempts in network architectures include but are not limited to, multi-task autoencoders [35, 36], efficient networks [37, 21] and vision transformers [38].”
BackgroundA Comprehensive Analysis of Deep Learning Based Representation for Face Recognition
Ghazi, M. M., & Ekenel, H. K. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.20 · Cited by 102
How Image Degradations Affect Deep CNN-Based Face Recognition?
Karahan, S., Kilinc Yildirum, M., Kirtac, K., Rende, F. S., Butun, G., & Ekenel, H. K. (2016) · 2016 International Conference of the Biometrics Special Interest Group (BIOSIG) · doi:10.1109/biosig.2016.7736924 · Cited by 99 · cited 1×
“Similar work has been done for face recognition tasks, [52, 23, 24, 53] analyzed the robustness of CNN-based face recognition models towards face variations caused by illumination change, occlusion, and standard image processing operations.”
BackgroundBenchmarking Neural Network Robustness to Common Corruptions and Surface Variations
Hendrycks, D., & Dietterich, T. G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.01697 · Cited by 99
Improving robustness against common corruptions with frequency biased models
Saikia, T., Schmid, C., & Brox, T. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01005 · Cited by 68 · cited 1×
“Besides focusing on the spatial domain, recent work [13, 14, 15, 16, 40] attempts to resolve the problem in the frequency domain.”
BackgroundInfluence of low resolution of images on reliability of face detection and recognition
Marciniak, T., Chmielewska, A., Weychan, R., Parzych, M., & Dabrowski, A. (2015) · Multimedia Tools and Applications · doi:10.1007/s11042-013-1568-8 · Cited by 58 · cited 1×
“On the other hand, the resizing operation, more specifically the down-sampling operation can significantly reduce the performance of modern deep learning-based detectors [57, 58] due to a loss of discriminative information.”
BackgroundDeferred Neural Rendering: Image Synthesis using Neural Textures
Thies, J., Zollhöfer, M., & Nießner, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.12356 · Cited by 46
Sakaridis, C., Wang, H., Li, K., Zurbrügg, R., Jadon, A., Abbeloos, W., Reino, D. O., Gool, L. V., & Dai, D. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3633063 · Cited by 46
Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware CluesPreprint
Qian, Y.-Y., Yin, G., Sheng, L., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.09355 · Cited by 42
On the Detection of Digital Face ManipulationPreprint
Dang, H., Liu, F., Stehouwer, J., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.01717 · Cited by 41
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency DomainPreprint
Liu, H., Li, X., Zhou, W., Chen, Y., He, Y., Xue, H., Zhang, W., & Yu, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.01856 · Cited by 22
On the Robustness and Generalizability of Face Synthesis Detection Methods
Sabel, J., & Johansson, F. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00107 · Cited by 12 · cited 1×
“In this work, a more systematic performance assessment approach is proposed for deepfake detectors in realistic situations.”
BackgroundA novel assessment framework for learning-based deepfake detectors in realistic conditions
Lu, Y., & Ebrahimi, T. (2022) · SPIE Applications of Digital Image Processing · doi:10.1117/12.2636683 · Cited by 11 · cited 1×
“This article is an extended version of our recent publication [25].”
BackgroundSelf-citationTrusted Media Challenge Dataset and User Study
Chen, W., Chua, S. L. B., Winkler, S., & Ng, S.-K. (2022) · Proceedings of the 31st ACM International Conference on Information & Knowledge Management · doi:10.1145/3511808.3557715 · Cited by 7
A Novel Framework for Assessment of Deep Face Recognition Systems in Realistic Conditions
Lu, Y., Barras, L., & Ebrahimi, T. (2022) · European Workshop on Visual Information Processing · doi:10.1109/euvip53989.2022.9922840 · Cited by 5 · cited 1×
“Similar work has been done for face recognition tasks, [52, 23, 24, 53] analyzed the robustness of CNN-based face recognition models towards face variations caused by illumination change, occlusion, and standard image processing operations.”
BackgroundSelf-citationAI Singapore Trusted Media Challenge Dataset
Chen, W., Chua, B., & Winkler, S. (2022) · arXiv.org · doi:10.48550/arxiv.2201.04788 · Cited by 4
Lu, Y., Luo, R., & Ebrahimi, T. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.11797 · Cited by 1
Self-citationAugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
Hendrycks, D., Mu, N., Cubuk, E. D., Zoph, B., Gilmer, J., & Lakshminarayanan, B. (2019) · arXiv · doi:10.48550/arxiv.1912.02781
Research works printed but not linked: 1
- 26. S. Agarwal, H. Farid, Y. Gu, M. He, K. Nagano, H. Li, Protecting world leaders against deep fakes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2019)
Cited by
Khan, A. A., Laghari, A. A., Inam, S. A., Ullah, S., Shahzad, M., & Syed, D. (2025) · Discover Computing · doi:10.1007/s10791-025-09550-0 · Cited by 99
Video and Audio Deepfake Datasets and Open Issues in Deepfake Technology: Being Ahead of the Curve
Akhtar, Z., Pendyala, T. L., & Athmakuri, V. S. (2024) · Forensic Sciences · doi:10.3390/forensicsci4030021 · Cited by 60
Parents and Children: Distinguishing Multimodal Deepfakes from Natural Images
Amoroso, R., Morelli, D., Cornia, M., Baraldi, L., Bimbo, A. D., & Cucchiara, R. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3665497 · Cited by 64
Deep Fake Image Detection using Xception Architecture
Joshi, P., & Nivethitha, V. (2024) · International Conference on Recent Trends in Computer Science and Technology (ICRTCST) · doi:10.1109/icrtcst61793.2024.10578398 · Cited by 37
Continuous fake media detection: Adapting deepfake detectors to new generative techniques
Tassone, F., Maiano, L., & Amerini, I. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.104143 · Cited by 27
Metadata
- Authors
- Lu, Y., & Ebrahimi, T.
- Affiliations
- École Polytechnique Fédérale de Lausanne (CH)
- Corresponding author
- Lu, Yuhang
- Venue
- EURASIP Journal on Image and Video Processing
- Published
- 2024-02-13
- Volume
- 2024
- Issue
- 1
- ISSN
- 1687-5176 (ISSN-L)
- 1687-5281
- OpenAlex
- W4391778999
- Cited by
- 60
- References
- 63
- Citations by year
- 2026: 18 · 2025: 25 · 2024: 15
- FWCI
- 8.08
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- EURASIP Journal on Image and Video Processing (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- Swiss National Foundation for Scientific Research (20CH21 195532)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Signal Processing › Biometric Identification and Security
- Keywords
- deepfake detection, real-world robustness, evaluation framework, data augmentation, stochastic deterioration, benchmark limitations
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.