Towards Generalizable Deepfake Detection with Locality-aware AutoEncoder
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
With advancements of deep learning techniques, it is now possible to generate super-realistic images and videos, i.e., deepfakes. These deepfakes could reach mass audience and result in adverse impacts on our society. Although lots of efforts have been devoted to detect deepfakes, their performance drops significantly on previously unseen but related manipulations and the detection generalization capability remains a problem. Motivated by the fine-grained nature and spatial locality characteristics of deepfakes, we propose Locality-Aware AutoEncoder (LAE) to bridge the generalization gap. In the training process, we use a pixel-wise mask to regularize local interpretation of LAE to enforce the model to learn intrinsic representation from the forgery region, instead of capturing artifacts in the training set and learning superficial correlations to perform detection. We further propose an active learning framework to select the challenging candidates for labeling, which requires human masks for less than 3% of the training data, dramatically reducing the annotation efforts to regularize interpretations. Experimental results on three deepfake detection tasks indicate that LAE could focus on the forgery regions to make decisions. The analysis further shows that LAE outperforms the state-of-the-arts by 6.52%, 12.03%, and 3.08% respectively on three deepfake detection tasks in terms of generalization accuracy on previously unseen manipulations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“GAN-generated digital content and not actual events captured by a camera implies that they still can be detected using advanced AI models [13].”
From DFDT: An End-to-End DeepFake Detection Framework Using Vision Transformer (2022) · cites this paper as [13]
The cited source says what the claim attributes to it.
In the citing paper
“This means that these detection systems work very well when dealing with deepfakes that come from the same data distribution as they were trained on. However, they struggle to perform well when exposed to deepfakes generated using different methods than the ones used for training.”
From Deepfake Detection: Analyzing Model Generalization Across Architectures, Datasets, and Pre-Training Paradigms (2024) · cites this paper as [12]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“P i x e l - w i s e m a s k The Adam optimizer [19] is utilized to optimize these models with betas of 0.9 and 0.999, epsilon set to 10 − 8 , and batch size set as 64.”
MethodInfluentialGrad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · cited 1×
“• XceptionNet: This is a CNN network based on separable convolutions with residual connections [6].”
BackgroundLearning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428 · cited 3×
“Secondly, in our preliminary experiments we have observed that these models fail to focus on the forgery regions to make detection, leveraging heatmaps by local interpretation methods [32, 39].”
MethodDeep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747
StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., & Choo, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00916 · Cited by 4,163
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 1×
“We use videos from Faceforensics++ [31].”
Generative Image Inpainting with Contextual Attention
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00577 · Cited by 2,527
Globally and locally consistent image completion
Iizuka, S., Simo-Serra, E., & Ishikawa, H. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073659 · Cited by 2,203 · cited 1×
“In this task we consider fake images by two inpainting methods, G&L [17] and ContextAtten [38], consisting seen and unseen dataset respectively.”
Method
Research works printed but not linked: 8
- [2] Shruti Agarwal, Hany Farid, Yuming Gu, Mingming He, Koki Nagano, and Hao Li. 2019. Protecting World Leaders Against Deep Fakes. In The Conference on Computer Vision and Pattern Recognition (CVPR) Workshop . -
- [10] Alexey Dosovitskiy and Thomas Brox. 2016. Generating images with perceptual similarity metrics based on deep networks. In Thirtieth Conference on Neural Information Processing Systems (NIPS) . -
- [12] Mengnan Du, Ninghao Liu, Qingquan Song, and Xia Hu. 2018. Towards Explanation of DNN-based Prediction with Guided Feature Inversion. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) (2018). -
- [20] Durk P Kingma and Prafulla Dhariwal. 2018. Glow: Generative flow with invertible 1x1 convolutions. In Thirty-second Conference on Neural Information Processing Systems (NeurIPS) . -
- [23] Yuezun Li and Siwei Lyu. 2019. Exposing deepfake videos by detecting face warping artifacts. Workshop on Media Forensics (in conjuction with CVPR) (2019). -
- [27] Thanh Thi Nguyen, Cuong M Nguyen, Dung Tien Nguyen, Duc Thanh Nguyen, and Saeid Nahavandi. 2019. Deep Learning for Deepfakes Creation and Detection. arXiv preprint arXiv:1909.11573 (2019). - · arXiv
- [33] Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. The International Conference on Learning Representations (ICLR) (2015). -
- [36] Yue Wu, Wael AbdAlmageed, and Premkumar Natarajan. 2019. ManTra-Net: Manipulation Tracing Network for Detection and Localization of Image Forgeries With Anomalous Features. In The Conference on Computer Vision and Pattern Recognition (CVPR) . -
Cited by
Media Forensics and DeepFakes: An Overview
Verdoliva, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.3002101 · Cited by 979
Deepfake Detection: A Systematic Literature Review
Rana, M. S., Nobi, M. N., Murali, B., & Sung, A. H. (2022) · IEEE Access · doi:10.1109/access.2022.3154404 · Cited by 573
Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection
Haliassos, A., Vougioukas, K., Petridis, S., & Pantic, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00500 · Cited by 537
Detecting Deepfakes with Self-Blended Images
Shiohara, K., & Yamasaki, T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01816 · Cited by 503
“Most previous detection methods [8,16,26,30,36,48,53, 64] perform well on the in-dataset scenario where they detect forgeries they learned in training; however, some studies [15,21,33,61] have found that the detection performance significantly drops in the cross-dataset scenario where fake Landmark matching”
BackgroundMethodEnd-to-End Reconstruction-Classification Learning for Face Forgery Detection
Cao, J., Ma, C., Yao, T., Chen, S., Ding, S., & Yang, X. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00408 · Cited by 428
Deepfake detection using deep learning methods: A systematic and comprehensive review
Heidari, A., Jafari Navimipour, N., Dag, H., & Unal, M. (2024) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1520 · Cited by 349
“Also, because of the robust deprivation of frame data following audio-visual compression, maximum image-based deepfake recognition techniques cannot be applied to videos (Du et al., 2020).”
BackgroundExploring Temporal Coherence for More General Video Face Forgery Detection
Zheng, Y., Bao, J., Chen, D., Zeng, M., & Wen, F. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01477 · Cited by 320
Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection
Chen, L., Zhang, Y., Song, Y., Liu, L., & Wang, J. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01815 · Cited by 307
“Recent studies have made various attempts for deepfake detection and achieve remarkable success [26, 41, 3, 2, 32, 39, 14, 24, 30, 29, 19, 47, 23, 36, 5].”
BackgroundMethodID-Reveal: Identity-aware DeepFake Video Detection
Cozzolino, D., Rossler, A., Thies, J., Niesner, M., & Verdoliva, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01483 · Cited by 221
Countering Malicious DeepFakes: Survey, Battleground, and Horizon
Juefei-Xu, F., Wang, R., Huang, Y., Guo, Q., Ma, L., & Liu, Y. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01606-8 · Cited by 191
Zhou, T., Wang, W., Liang, Z., & Shen, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00572 · Cited by 156
A comprehensive overview of Deepfake: generation, detection, datasets, and opportunities
Seow, J. W., Lim, M. K., Phan, R. C. W., & Liu, J. K. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.09.135 · Cited by 149
Raising the Bar of AI-generated Image Detection with CLIP
Cozzolino, D., Poggi, G., Corvi, R., Nießner, M., & Verdoliva, L. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00439 · Cited by 308
“Other few-shot solutions have been proposed lately in this field [13, 19, 32, 46].”
BackgroundDetect and locate: Exposing face manipulation by semantic-and noise-level telltales
Kong, C., Chen, B., Li, H., Wang, S., Rocha, A., & Kwong, S. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3169921 · Cited by 120
“[13] designed a locality-aware autoencoder to enforce the model to learn the feature representation of the forgery region, which boosts the generalization capability.”
BackgroundMethodA review of deep learning‐based approaches for deepfake content detection
(2024) · doi:10.1111/exsy.13570 · Cited by 116
“[25] motivated by the degrading generalizability of deepfake detection models, they proposed a Locality-Aware AutoEncoder in which the model is forced to focus on correct forgery regions to make detection predictions.”
BackgroundMethodDFDT: An End-to-End DeepFake Detection Framework Using Vision Transformer
Khormali, A., & Yuan, J.-S. (2022) · Applied Sciences · doi:10.3390/app12062953 · Cited by 88
“GAN-generated digital content and not actual events captured by a camera implies that they still can be detected using advanced AI models [13].”
BackgroundMethodAUNet: Learning Relations Between Action Units for Face Forgery Detection
Bai, W., Liu, Y., Zhang, Z., Li, B., & Hu, W. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.02367 · Cited by 70
A literature review and perspectives in deepfakes: generation, detection, and applications
Dagar, D., & Vishwakarma, D. K. (2022) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-022-00241-w · Cited by 113
Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU models
Du, M., Manjunatha, V., Jain, R., Deshpande, R., Dernoncourt, F., Gu, J., Sun, T., & Hu, X. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.71 · Cited by 132
Self-citationGuo, Z., Wang, L., Yang, W., Yang, G., & Li, K. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3289147 · Cited by 60
“Some works promote the existing backbone networks to capture forgery patterns by designing attention mechanisms [20], [21], [22].”
BackgroundAn Investigation of the Effectiveness of Deepfake Models and Tools
Mukta, M. S. H., Ahmad, J., Raiaan, M. A. K., Islam, S., Azam, S., Ali, M. E., & Jonkman, M. (2023) · Journal of Sensor and Actuator Networks · doi:10.3390/jsan12040061 · Cited by 85
“An autoencoder [21–23] is a form of neural network that is used in deepfakes.”
BackgroundA systematic literature review on deepfake detection techniques
Sharma, V. K., Garg, R. K., & Caudron, Q. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19906-1 · Cited by 87
ADD: Attention-Based DeepFake Detection Approach
Khormali, A., & Yuan, J.-S. (2021) · Big Data and Cognitive Computing · doi:10.3390/bdcc5040049 · Cited by 55
“However, our proposed method is different from [26] where the authors proposed a DeepFake detection method from FGVC angle that is built using an autoencoder structure different from our proposed method, which is based on a deep learning structure.”
BackgroundMethodInfluentialAssessment framework for deepfake detection in real-world situations
Lu, Y., & Ebrahimi, T. (2024) · EURASIP Journal on Image and Video Processing · doi:10.1186/s13640-024-00621-8 · Cited by 60
“Other creative attempts in network architectures include but are not limited to, multi-task autoencoders [35, 36], efficient networks [37, 21] and vision transformers [38].”
BackgroundA Survey on Learning to Reject
Zhang, X.-Y., Xie, G.-S., Li, X., Mei, T., & Liu, C. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3238024 · Cited by 86
“To improve generalization performance, Du et al. [59] proposed a locality-aware autoencoder to enforce the model to learn intrinsic representation from the forgery region, instead of capturing artifacts in the training set and learning superficial correlations to perform fake detection.”
Deepfake Forensics via an Adversarial Game
Wang, Z., Guo, Y., & Zuo, W. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3172845 · Cited by 49
Learning Forgery Region-Aware and ID-Independent Features for Face Manipulation Detection
Miao, C., Chu, Q., Li, W., Li, S., Tan, Z., Zhuang, W., & Yu, N. (2022) · IEEE Transactions on Biometrics, Behavior, and Identity Science · doi:10.1109/tbiom.2021.3119403 · Cited by 50
Multi-attention-based approach for deepfake face and expression swap detection and localization
Waseem, S., Abu–Bakar, S. A. R., Omar, Z., Ahmed, B. A., Baloch, S., & Hafeezallah, A. (2023) · EURASIP Journal on Image and Video Processing · doi:10.1186/s13640-023-00614-z · Cited by 66
Khan, S. A., & Dang-Nguyen, D.-T. (2024) · IEEE Access · doi:10.1109/access.2023.3348450 · Cited by 53
“However, a significant issue associated with current deepfake detection models is their lack of generalisation capability [1], [7], [12].”
BackgroundSeeABLE: Soft Discrepancies and Bounded Contrastive Learning for Exposing Deepfakes
Larue, N., Vu, N.-S., Struc, V., Peer, P., & Christophides, V. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01921 · Cited by 45
“However, as recent empirical studies [18, 39, 67] report, the performance of such (discriminatively-trained) detectors degrades significantly when used with unseen face-manipulation methods, which severely limits their use in real-life deployment scenarios [75].”
BackgroundLearning Pairwise Interaction for Generalizable DeepFake Detection
Xu, Y., Raja, K., Verdoliva, L., & Pedersen, M. (2023) · 2023 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw58289.2023.00074 · Cited by 43
Generalization of Forgery Detection With Meta Deepfake Detection Model
Tran, V.-N., Kwon, S.-G., Lee, S.-H., Le, H.-S., & Kwon, K.-R. (2023) · IEEE Access · doi:10.1109/access.2022.3232290 · Cited by 47
“This is also mentioned in some research related to image encryption [51], [52], [53], [54].”
BackgroundCross, C. (2022) · Crime Prevention and Community Safety · doi:10.1057/s41300-021-00134-w · Cited by 72
“To combat this, there is a long history of established techniques used to detect traditional forgery detection and manipulation of images (Du et al. 2020).”
BackgroundMethodKumar, S., Gupta, S. K., Kaur, M., & Gupta, U. C. (2022) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2022.103644 · Cited by 48
Detection and Localization of Facial Expression Manipulations
Mazaheri, G., & Roy-Chowdhury, A. K. (2022) · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv51458.2022.00283 · Cited by 44
Improving Generalization of Deepfake Detectors by Imposing Gradient Regularization
Guan, W., Wang, W., Dong, J., & Peng, B. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3396064 · Cited by 52
“For the fake samples manipulated by unseen forgery methods, the performance of trained deep-fake detectors always drops significantly, as some previous works [16], [17], [18], [19], [20] mentioned.”
MethodDeepfakes: evolution and trends
Gil, R., Virgili‐Gomà, J., López, J. M., & García, R. (2023) · Soft Computing · doi:10.1007/s00500-023-08605-y · Cited by 50
“…et al. (2020), Nasar et al. (2020), Mittal et al. (2020a), Du et al. (2020a), El Rai et al. (2020), Ramadhani and Munir (2020), Gupta et al. (2020), Du et al. (2020b), Lewis et al. (2020), Chugh et al. (2020), Shah et al. (2020), Tarasiou and Zafeiriou (2020), Ross et al. (2020), Nguyen and…”
Deepfake Video Detection Using 3D-Attentional Inception Convolutional Neural Network
Lu, C., Liu, B., Zhou, W., Chu, Q., & Yu, N. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506381 · Cited by 37
“Attention mechanism has been demonstrated to be effective in capturing subtle texture information in frame-level deepfake detection methods[7, 16].”
A Review of Deep Learning-based Approaches for Deepfake Content Detection
Passos, L. A., Jodas, D., Costa, K. A. P., Júnior, L. A. S., Rodrigues, D., Ser, J. D., Camacho*, D., & Papa, J. (2023) · doi:10.22541/au.169735672.27713914/v1 · Cited by 116
Zero-Shot Detection of AI-Generated Images
Cozzolino, D., Poggi, G., Nießner, M., & Verdoliva, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72649-1_4 · Cited by 80
“A significant step towards improved generalization is the use of few-shot or incremental learning strategies [12,17,33,47].”
BackgroundAutomated Anomaly Detection via Curiosity-Guided Search and Self-Imitation Learning
Li, Y., Chen, Z., Zha, D., Zhou, K., Jin, H., Chen, H., & Hu, X. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3105636 · Cited by 52
Self-citationLightweight Deepfake Detection Based on Multi-Feature Fusion
Yasir, S. M., & Kim, H. (2025) · Applied Sciences · doi:10.3390/app15041954 · Cited by 48
“The proposed model, focusing on auto-encoder-based generated videos and keyframe identification, achieves high accuracy with minimal computational demands, making it suitable for memory-constrained devices and enhancing deepfake detection capabilities [22,23].”
Graph Learning for Fake Review Detection
Yu, S., Ren, J. F., Li, S., Naseriparsa, M., & Xia, F. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.922589 · Cited by 30
“Among these methods, PC-GNN, AO-GNN, and DeepFD are with better generalization ability (Du et al., 2020; Betlei et al., 2021; Hibshman et al., 2021).”
MethodDatasets, clues and state-of-the-arts for multimedia forensics: An extensive review
Yadav, A. K., & Vishwakarma, D. K. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123756 · Cited by 27
Towards Understanding the Generalization of Deepfake Detectors from a Game-Theoretical View
Yao, K., Wang, J., Diao, B., & Li, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00194 · Cited by 37
DeepFake video detection: Insights into model generalisation — A Systematic review
Ramanaharan, R., Guruge, D. B., & Agbinya, J. I. (2025) · Data and Information Management · doi:10.1016/j.dim.2025.100099 · Cited by 48
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency DomainPreprint
Liu, H., Li, X., Zhou, W., Chen, Y., He, Y., Xue, H., Zhang, W., & Yu, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.01856 · Cited by 22
Videoforensicshq: Detecting High-Quality Manipulated Face Videos
Fox, G., Liu, W., Kim, H., Seidel, H.-P., Elgharib, M., & Theobalt, C. (2021) · 2021 IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme51207.2021.9428101 · Cited by 22
Deepfake Detection Fighting Against Noisy Label Attack
Qiao, T., Xie, S., Chen, Y., Retraint, F., Shi, R., & Luo, X. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3385286 · Cited by 21
“Some prior works such as [49], [50], [51] have demonstrated the potential of supervised learning frameworks for robust representation.”
BackgroundFuture Trends in Digital Face Manipulation and Detection
Tolosana, R., Rathgeb, C., Vera-Rodríguez, R., Busch, C., Verdoliva, L., Lyu, S., Nguyen, H. H., Yamagishi, J., Echizen, I., Rot, P., et al. (2022) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-030-87664-7_21 · Cited by 19
“However, their performances usually drop significantly when evaluated under cross-manipulation setting (where train and test sets are not from the same manipulation type or database) [12, 19, 37, 38].”
BackgroundCyber Vaccine for Deepfake Immunity
Chang, C., Nguyen, H. H., Yamagishi, J., & Echizen, I. (2023) · IEEE Access · doi:10.1109/access.2023.3311461 · Cited by 26
“To yield further forensic clues, certain methods involve training neural networks to localise the tampered-with areas [49], [50], [51].”
BackgroundFacial Forgery-Based Deepfake Detection Using Fine-Grained Features
Nadimpalli, A. V., & Rattani, A. (2023) · International Conference on Machine Learning and Applications · doi:10.1109/icmla58977.2023.00328 · Cited by 23
“A handful of facial-forgery-based deepfake detection meth-ods have been proposed based on fine-grained classification [17]–[19].”
BackgroundMethodEnhancing Deepfake Detection With Diversified Self-Blending Images and Residuals
Liu, Q., Xue, Z., Liu, H., & Liu, J. (2024) · IEEE Access · doi:10.1109/access.2024.3382196 · Cited by 22
“However, due to the complex scenes and diverse expressions of characters, many methods [1], [2], [3], [4] perform well in test datasets but lack effectiveness in cross-dataset experiments.”
BackgroundDepthFake: A Depth-Based Strategy for Detecting Deepfake Videos
Maiano, L., Papa, L., Vocaj, K., & Amerini, I. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-37745-7_2 · Cited by 18
ID-insensitive deepfake detection model based on multi-attention mechanism
Sheng, Y., Zou, Z., Yu, Z., Pang, M., Ou, W., & Han, W. (2025) · Scientific Reports · doi:10.1038/s41598-025-96254-6 · Cited by 18
Exploring Bi-Level Inconsistency via Blended Images for Generalizable Face Forgery Detection
Jiang, P., Xie, H., Yu, L., Jin, G., & Zhang, Y. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3417266 · Cited by 16
“However, several studies [38], [39] have pointed out that current models struggle to work well with unseen forgery techniques due to their overfitting of method-specific artifacts.”
BackgroundManipulating faces for identity theft via morphing and deepfake: Digital privacy
Agarwal, A., & Ratha, N. (2023) · Handbook of statistics · doi:10.1016/bs.host.2022.12.003 · Cited by 18
Deep Learning for Forgery Face Detection Using Fuzzy Fisher Capsule Dual Graph
Arunkumar, P. M., Sangeetha, Y., Raja, P. V., & Sangeetha, S. N. (2022) · Information Technology And Control · doi:10.5755/j01.itc.51.3.31510 · Cited by 14
“of the image or video to the real face of the image or video [10].”
BackgroundDeep forgery discriminator via image degradation analysis
Yu, M., Zhang, J., Li, S., Lei, J. W., Wang, F., & Zhou, H. (2021) · IET Image Processing · doi:10.1049/ipr2.12234 · Cited by 12
Deepfake Catcher: Can a Simple Fusion be Effective and Outperform Complex DNNs?
Agarwal, A., & Ratha, N. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00383 · Cited by 18
“Aware Autoencoder (LAE) [18], Face X-Ray [28] and Spatial Phase Shallow Learning (SPSL) [33] are the few effective detection architectures.”
BackgroundTowards Interpreting and Mitigating Shortcut Learning Behavior of NLU ModelsPreprint
Du, M., Manjunatha, V., Jain, R., Deshpande, R., Dernoncourt, F., Gu, J., Sun, T., & Hu, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.06922 · Cited by 11
“…the shortcut learning phenomenon for different kinds of language and vision tasks, such as NLI (Niven and Kao, 2019), question answering (Mudrakarta et al., 2018), reading comprehension (Si et al., 2019), VQA (Agrawal et al., 2018; Manjunatha et al., 2019), and deepfake detection (Du et al., 2020).”
BackgroundSelf-citationDeepfakes in Cyber Warfare: Threats, Detection, Techniques and Countermeasures
Cinar, B. (2023) · Asian Journal of Research in Computer Science · doi:10.9734/ajrcos/2023/v16i4381 · Cited by 14
The MeVer DeepFake Detection Service: Lessons Learnt from Developing and Deploying in the Wild
Baxevanakis, S., Kordopatis-Zilos, G., Galopoulos, P., Apostolidis, L., Levacher, K., Schlicht, I. B., Teyssou, D., Kompatsiaris, I. Y., & Papadopoulos, S. (2022) · doi:10.1145/3512732.3533587 · Cited by 12
“Research since then has evolved by combining CNNs with other architectures such as Recurrent Neural Networks (RNNs) [19], Long Short-Term Memories (LSTMs) [36, 47] or Attention heads [14, 30, 73, 80, 81, 84].”
MethodDeepfake Technology: Negative Impacts, Mitigation Methods, and Preventive Algorithms
Gonzales, N. H., Lobian, A. L., Hengky, F. M., Andhika, M. R., Achmad, S., & Sutoyo, R. (2023) · IEEE International Conference on Recent Advances and Innovations in Engineering (ICRAIE) · doi:10.1109/icraie59459.2023.10468370 · Cited by 13
DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake Detection
Peng, C., Guo, H., Liu, D., Wang, N., Hu, R., & Gao, X. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3621207 · Cited by 14
“[9] proposed a locality-aware auto-encoder to bridge the generalization gap, using extra pixel-wise forgery masks for regularization to learn meaningful and intrinsic forgery representations.”
MethodA comparison study to detect seam carving forgery in JPEG images with deep learning models
Celebi, N., Hsu, T.-L., & Liu, Q. (2022) · Journal of Surveillance Security and Safety · doi:10.20517/jsss.2022.02 · Cited by 8
“[44] proposed a locality-aware autoencoder to detect deepfake images.”
BackgroundTowards Generalizable and Robust Face Manipulation Detection via Bag-of-local-featurePreprint
Miao, C., Chu, Q., Li, W., Gong, T., Zhuang, W., & Yu, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.07915 · Cited by 8
Vulnerability of Automatic Identity Recognition to Audio-Visual Deepfakes
Korshunov, P., Chen, H., Garner, P. N., & Marcel, S. (2023) · IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb57857.2023.10449189 · Cited by 13
“Arguably, this is why deepfake detection methods do not generalize well when they are tested on a database that used the same GAN architecture as the training videos but different blending methods [10, 33, 29].”
BackgroundThinking in Patch: Towards Generalizable Forgery Detection with Patch Transformation
Zhang, X., Wang, S., Liu, C., Zhang, M., Liu, X., & Xie, H. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-89370-5_25 · Cited by 10
Leveraging High-Frequency Components for Deepfake Detection
Mejri, N., Papadopoulos, K., & Aouada, D. (2021) · IEEE International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp53017.2021.9733606 · Cited by 11
Kalyanasundaram, P., Priya, K., Kavinkumar, S., Gopinath, G., & Bommi, S. T. K. (2024) · doi:10.1109/icstem61137.2024.10561029 · Cited by 6
“Through pixel-wise masks, LAE learns intrinsic representations, outperforming state-of-the-art methods by 6.52%, 12.03%, and 3.08% across three detection tasks, showcasing superior generalization [3][4].”
MethodInfluentialChang, C., & Chien, W.-C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60405-8_20 · Cited by 9
Unknown presentation attack detection against rational attackers
Khodabakhsh, A., & Akhtar, Z. (2021) · IET Biometrics · doi:10.1049/bme2.12053 · Cited by 7
Deepfake Detection Using Adversarial Neural Network
Selvaraj, P., Jagatheesaperumal, S. K., Marimuthu, K., Saravanan, O., Alkhamees, B. F., & Hassan, M. M. (2025) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2025.064138 · Cited by 11
“The existing methods, for instance, Du et al. [15] presented the Locality-Aware AutoEncoder (LAE) for detection, emphasizing enhanced generalization accuracy.”
Assessing Image Inpainting via Re-Inpainting Self-Consistency Evaluation
Chen, T., Zhang, J., Hong, Y., Zhang, Y., & Zhang, L. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679693 · Cited by 6
“Evaluating generated images holds significant importance in various fields such as detection [6] and aes-thetics assessment [5].”
Abate, A. F., Cimmino, L., & Polsinelli, M. (2025) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-031-87784-1_39 · Cited by 5
Faces Blind Your Eyes: Unveiling the Content-Irrelevant Synthetic Artifacts for Deepfake Detection
Fu, X., Fu, B., Chen, S., Yao, T., Wang, Y., Ding, S., Liang, X., & Li, X. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3592576 · Cited by 8
“A significant challenge in deepfake detection is the generalization capacity [19], [20], [21], [22], [23], [24], [25], [26], [27] of detection models.”
Patil, M. B., Sangolgi, V. A., Bag, V. V., Patwegar, A. B., Koli, R., Naikwadi, A., & Shaikh, A. G. (2024) · Journal of Integrated Science and Technology · doi:10.62110/sciencein.jist.2024.v12.820 · Cited by 8
Manikandan, A. (2022) · Journal of Artificial Intelligence Machine Learning and Neural Network · doi:10.55529/jaimlnn.24.33.42 · Cited by 4
Towards Generalized Deepfake Detection With Continual Learning On Limited New Data
Huang, H., Sun, N., Lin, X., & Moustafa, N. (2022) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta56598.2022.10034569 · Cited by 8
Facial Manipulation Detection Based on the Color Distribution Analysis in Edge Region.Preprint
Kim, D.-K., Kim, D.-H., & Kim, K. (2021) · arXiv (Cornell University) · Cited by 6
Detect and Locate: A Face Anti-Manipulation Approach with Semantic and Noise-level Supervision.Preprint
Kong, C., Chen, B., Li, H., Wang, S., Rocha, A. D. R., & Kwong, S. (2021) · arXiv (Cornell University) · Cited by 3
Face forgery detection with image patch comparison and residual map estimation
Feng, C., Liu, C., Wang, Y., & Zhou, Q. (2024) · Journal of Image and Graphics · doi:10.11834/jig.230149 · Cited by 5
Domain-Adaptive Representation Learning for Multimodal Deepfake Detection
Haq, I. U., & Malik, K. M. (2025) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2025.3637242 · Cited by 3
“One possible solution to address the generalization issue requires continuous training of deep models using datasets with new forgeries [25], which is expensive in terms of resources and time.”
Explore and Enhance the Generalization of Anomaly DeepFake Detection
Wang, Y., Chen, S., Yao, T., Ma, L., Zhang, Z., & Tan, X. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-2092-7_2 · Cited by 4
“For instance, compact network Mesonet [1], capsule network [24], autoencoder [8], recurrent convolutional networks [12, 27], and attentional networks [34].”
BackgroundDetecting Deepfake Videos Through CNN-MLP Model in Media Forensics
Kandari, M., Tripathi, V., Pant, B., Sar, A., Choudhury, T., & Choudhury, T. (2024) · OPJU International Technology Conference on Smart Computing for Innovation and Advancement in Industry (OTCON) · doi:10.1109/otcon60325.2024.10687433 · Cited by 2
“Mengnan Du et al. [24] proposed a Locality-Aware AutoEncoder (LAE) to overcome the generalisation gap.”
BackgroundMulti-level Distributional Discrepancy Enhancement for Cross Domain Face Forgery Detection
Qiu, L., Jiang, K., Liu, S., & Tan, X. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-8499-8_35 · Cited by 3
Deepfake Detection Using Deep Learning
Prakash Raj, S., Pravin, D. V., Sabareeswaran, G., Sanjith, R. K., & Gomathi, B. (2024) · International Conference on Advanced Computing and Communication Systems (ICACCS) · doi:10.1109/icaccs60874.2024.10717155 · Cited by 3
“These techniques hold the promise of leveraging knowledge from broader datasets to enhance the robustness of deep fake detection models [20-24].”
Quick, Print This Page! The Value of Analogue Media in a Digital World
Haliburton, L., Hoppe, M., Schmidt, A., & Kosch, T. (2021) · CHI Extended Abstracts · doi:10.1145/3411763.3450375 · Cited by 2
“To combat rising concerns about Deepfakes, researchers have begun developing methods of Deepfake detection [8, 10, 22, 39].”
MethodDetecting Deepfake Videos in Data Scarcity Conditions by Means of Video Coding Features
Wang, J., Alamayreh, O., Tondi, B., Costanzo, A., & Barni, M. (2022) · APSIPA Transactions on Signal and Information Processing · doi:10.1561/116.00000032 · Cited by 2
Fair-FLIP: Fair Deepfake Detection with Fairness-Oriented Final Layer Input Prioritising
Szandała, T., Ezzeddine, F., Rusin, N., Giordano, S., & Ayoub, O. (2025) · IEEE Swiss Conference on Data Science (SDS) · doi:10.1109/sds66131.2025.00021 · Cited by 3
“Several studies have focused on detecting deepfake across images, audio, and video modalities [2], [3], [6], [10]–[13], leveraging statistical techniques [30]–[33], ML-based [34]– [36] or deep learning-based methods [37]–[42].”
Patch-Discontinuity Mining for Generalized Deepfake Detection
Yuan, H., Ping, Y., Xu, Z., Cao, J., Jia, S., & Ma, C. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3651014 · Cited by 3
“Therefore,recentdeepfakedetectionmethods[7], [10],[12],[13],[14],[23],[34]focusonimprovinggeneraliza-tionperformanceinunknowndomains.”
Raghavajosyula, V., & Pawar, D. (2026) · Multimedia Tools and Applications · doi:10.1007/s11042-026-21320-8 · Cited by 4
“Works such as Cozzolino et al. [ 30], Khodabkhsh et al.[31], Du et al.[32], Li et al. [33 ] focus on enhancing generalization capabilities.”
Recent Advances Digital Image Forgery Detection: A Systematic Mapping Study
Batista, G. D. C., Welligton, J., de Souza, M. A., Junior, W., & Kuribayashi, H. P. (2026) · IEEE Access · doi:10.1109/access.2026.3657232 · Cited by 2
A Generalizable Face Security Detection Method via Unified Texture and Semantic Feature Framework
Zhang, K.-Y., Chen, R., Sun, J., Wang, J., Yang, H., Yao, T., & Ding, S. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00337 · Cited by 3
UGLF: Uncertainty-Gated Global–Local Fusion for generalizable face forgery detection
Peng, S., 胡嗣驹, Lu, T., He, C., Ren, Y., & Li, Y. (2026) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2026.104847 · Cited by 1
Self-citationManipulation-Invariant Fingerprints for Cross-Dataset Deepfake Detection
Zuo-yan, L., Yang, W., Liu, R., & Zhu, Y. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-92273-3_15 · Cited by 1
Hybrid Spatio-Temporal Network for Face Forgery Detection
Liu, X., Gao, S., Zhou, P., Liu, J., Luo, X., Zhang, L., & Zhang, B. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47665-5_21 · Cited by 2
Spatial-Frequency Dual-Stream Reconstruction for Deepfake Detection
Peng, C., Chen, T., Liu, D., Zheng, Y., & Wang, N. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-8795-1_32 · Cited by 2
Revealing the compactness of real samples via image reconstruction for deepfake detection
Dong-Yup, H., Yang, G., Guo, T., Wang, X., & Zhang, J. (2025) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2025.104201 · Cited by 3
Metadata
- Authors
- Du, M., Pentyala, S., Li, Y., & Hu, X.
- Affiliations
- Texas A&M University (US)
- Venue
- Proceedings of the 29th ACM International Conference on Information & Knowledge Management
- Publisher
- ACM
- Published
- 2020-10-19
- Pages
- 325-334
- ISSN
- 2155-0751 (ISSN-L)
- OpenAlex
- W3094470648
- Cited by
- 114
- References
- 26
- Citations by year
- 2026: 10 · 2025: 17 · 2024: 23 · 2023: 20 · 2022: 22 · 2021: 20 · 2020: 2
- FWCI
- 4.94
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM International Conference on Information and Knowledge Management (CIKM) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- NSF (IIS-1900990, CNS-1816497)
- DARPA (N66001-17-2-4031)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- deepfake detection, spatial locality, active learning (machine learning), video forgery detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.