DeepFake-Adapter: Dual-Level Adapter for DeepFake Detection
Open accessThe DOI matches its Crossref record; 84 of the 84 references it cites were found in a registry or an index.
Abstract
Existing deepfake detection methods fail to generalize well to unseen or degraded samples, which can be attributed to the over-fitting of low-level forgery patterns. Here we argue that high-level semantics are also indispensable recipes for generalizable forgery detection. Recently, large pre-trained Vision Transformers (ViTs) have shown promising generalization capability. In this paper, we propose the first parameter-efficient tuning approach for deepfake detection, namely DeepFake-Adapter, to effectively and efficiently adapt the generalizable high-level semantics from large pre-trained ViTs to aid deepfake detection. Given large pre-trained models but limited deepfake data, DeepFake-Adapter introduces lightweight yet dedicated dual-level adapter modules to a ViT while keeping the model backbone frozen. Specifically, to guide the adaptation process to be aware of both global and local forgery cues of deepfake data, 1) we not only insert Globally-aware Bottleneck Adapters in parallel to MLP layers of ViT, 2) but also actively cross-attend Locally-aware Spatial Adapters with features from ViT. Unlike existing deepfake detection methods merely focusing on low-level forgery patterns, the forgery detection process of our model can be regularized by generalizable high-level semantics from a pre-trained ViT and adapted by global and local low-level forgeries of deepfake data. Extensive experiments on several standard deepfake detection benchmarks validate the effectiveness of our approach. Notably, DeepFake-Adapter demonstrates a convincing advantage under cross-dataset and cross-manipulation settings.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Shao et al. [22 ] proposed DeepFake-Adapter, a parameter-efficient framework that augments pre-trained ViTs with dual-level adapters to enhance transferability and domain generalization.”
From Cross-dataset video deepfake detection using Transformer and CNN architectures (2026) · cites this paper as [22]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 3×
“FaceForensics++ (c23) FaceForensics++ (c40) Methods DF FS F2F NT DF FS F2F NT ResNet-50 [23] 98.”
InfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 2×
“…we compare the linear separability regarding real/fake faces of FaceForensics++ dataset (Rossler et al., 2019) based on features extracted by Xception (Chollet, 2017) pre-trained on ImageNet-1K (Deng et al., 2009) and ViT-Base (Dosovit-skiy et al., 2020) pre-trained on ImageNet-21K, respectively.”
InfluentialAn Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472 · cited 10×
“, spatial locality, than Convolutional Neural Networks (CNNs) [14].”
InfluentialGrad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · cited 4×
“…we compare the linear separability regarding real/fake faces of FaceForensics++ dataset (Rossler et al., 2019) based on features extracted by Xception (Chollet, 2017) pre-trained on ImageNet-1K (Deng et al., 2009) and ViT-Base (Dosovit-skiy et al., 2020) pre-trained on ImageNet-21K, respectively.”
InfluentialHigh-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 1×
“Following the setting of linear-probe evaluation in Radford et al. (2021), we compare the linear separability regarding real/fake faces of FaceForensics++ dataset (Rossler et al., 2019) based on features extracted by Xception (Chollet, 2017) pre-trained on ImageNet-1K (Deng et al., 2009) and…”
Influential(2017) · Cited by 11,625
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Denoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712 · cited 1×
“We further test our model on deepfake samples generated by 5 representative diffusion modes, such as DDPM (Ho et al., 2020a), IDDPM (Quinn & Dhariwal, 2021), ADM (Dhariwal & Nichol, 2021a), PNDM (Liu et al., 2022), and LDM (Rombach et al., 2022).”
Masked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollar, P., & Girshick, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01553 · Cited by 8,489
A Comprehensive Survey on Transfer Learning
Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., & He, Q. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2020.3004555 · Cited by 6,678 · cited 2×
“…compared to other existing adaptation approaches, we compare it with the most widely-used tuning approaches, e.g., full-tuning and linear-probe (Zhuang et al., 2020), and two latest parameter-efficient adaptation methods named Visual Prompt Tuning (VPT) (Jia et al., 2022) and ViT-Adapter (Chen…”
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361 · cited 2×
“Prompt learning (Liu et al., 2023; Lester et al., 2021) is wide-used in NLP which prepends learnable tokens into the input text.”
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352 · cited 2×
“Prompt learning (Liu et al., 2023; Lester et al., 2021) is wide-used in NLP which prepends learnable tokens into the input text.”
Diffusion Models Beat GANs on Image Synthesis
Dhariwal, P., & Nichol, A. (2021) · arXiv · doi:10.48550/arxiv.2105.05233 · Cited by 3,362
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 6×
“Following the setting of linear-probe evaluation in [44], we compare the linear separability regarding real/fake faces of FaceForensics++ dataset [45] based on features extracted by Xception [10] pre-trained on ImageNet-1K [11] and ViT-Base [14] pre-trained on ImageNet-21K, respectively.”
InfluentialThe Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096 · cited 2×
“Prompt learning (Liu et al., 2023; Lester et al., 2021) is wide-used in NLP which prepends learnable tokens into the input text.”
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · arXiv · doi:10.48550/arxiv.2104.08691 · Cited by 3,092 · cited 2×
“Prompt learning (Liu et al., 2023; Lester et al., 2021) is wide-used in NLP which prepends learnable tokens into the input text.”
CvT: Introducing Convolutions to Vision Transformers
Wu, H., Xiao, B., Codella, N., Liu, M., Dai, X., Yuan, L., & Zhang, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00009 · Cited by 2,444 · cited 1×
“Inspired by recent works (Yuan et al., 2021; Wu et al., 2021) that integrate convolutional operations of CNNs into a ViT, we introduce the convolution-based head part of LSA.”
Parameter-efficient transfer learning for NLP
Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, M., & Gelly, S. (2019) · arXiv · doi:10.48550/arxiv.1902.00751 · Cited by 2,270 · cited 1×
“Adapter (Houlsby et al., 2019) and LoRA (Hu et al., 2022) add tiny learnable modules into NLP transformers.”
Jia, M., Tang, L., Chen, B.-C., Cardie, C., Belongie, S., Hariharan, B., & Lim, S.-N. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19827-4_41 · Cited by 1,965
Face2Face: Real-Time Face Capture and Reenactment of RGB Videos
Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., & Niessner, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.262 · Cited by 1,747
Improved Denoising Diffusion Probabilistic Models
Nichol, A., & Dhariwal, P. (2021) · arXiv · doi:10.48550/arxiv.2102.09672 · Cited by 1,550
Deferred neural rendering: image synthesis using neural textures
Thies, J., Zollhöfer, M., & Nießner, M. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323035 · Cited by 1,356 · cited 1×
“…in deepfake detection, FF++ collects 1,000 original videos, and 4000 fake videos generated by corresponding four face manipulation techniques: Deepfakes (DF) (Li et al., 2020), Face2Face (F2F) (Thies et al., 2016), FaceSwap (FS) (Thies et al., 2016), and Neural-Textures (NT) (Thies et al., 2019).”
Face X-Ray for More General Face Forgery Detection
Li, L., Bao, J., Zhang, T., Yang, H., Chen, D., Wen, F., & Guo, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00505 · Cited by 1,211
CNN-Generated Images Are Surprisingly Easy to Spot… for Now
Wang, S.-Y., Wang, O., Zhang, R., Owens, A., & Efros, A. A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00872 · Cited by 1,210
Thinking in Frequency: Face Forgery Detection by Mining Frequency-Aware Clues
Qian, Y., Yin, G., Sheng, L., Chen, Z., & Shao, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58610-2_6 · Cited by 1,058
Multi-attentional Deepfake Detection
Zhao, H., Wei, T., Zhou, W., Zhang, W., Chen, D., & Yu, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00222 · Cited by 1,019 · cited 6×
“…existing deepfake detection methods merely focus on exploiting low-level forgery features from local textures (Chen et al., 2021; Gu et al., 2022; Zhao et al., 2021; Liu et al., 2020; Shao et al., 2019, 2018), blending boundary (Li et al., 2020), or frequency information (Li et al., 2021; Qian…”
InfluentialHigh-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2021) · arXiv · doi:10.48550/arxiv.2112.10752 · Cited by 737
Two-Stream Neural Networks for Tampered Face Detection
Zhou, P., Han, X., Morariu, V. I., & Davis, L. S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.229 · Cited by 700 · cited 1×
“In addition, noise characteristics are exploited as the forgery clues in works (Gu et al., 2022; Zhou et al., 2017).”
Incorporating Convolution Designs into Visual Transformers
Yuan, K., Guo, S., Liu, Z., Zhou, A., Yu, F., & Wu, W. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00062 · Cited by 611 · cited 1×
“Inspired by recent works (Yuan et al., 2021; Wu et al., 2021) that integrate convolutional operations of CNNs into a ViT, we introduce the convolution-based head part of LSA.”
Self-citationGeneralizing Face Forgery Detection with High-frequency Features
Luo, Y., Zhang, Y., Yan, J., & Liu, W. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01605 · Cited by 609 · cited 2×
“Neverthe-less, their performance degrades once countering unseen or low-quality forgeries (Luo et al., 2021; Chai et al., 2020; Shao et al., 2022a, 2020).”
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain
Liu, H., Li, X., Zhou, W., Chen, Y., He, Y., Xue, H., Zhang, W., & Yu, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00083 · Cited by 600
DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection
Jiang, L., Li, R., Wu, W., Qian, C., & Loy, C. C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00296 · Cited by 599
Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection
Haliassos, A., Vougioukas, K., Petridis, S., & Pantic, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00500 · Cited by 537 · cited 3×
“Moreover, motion artifacts are dig out from mouth movements as the face forgery patterns by fine-tuning a temporal network pre-trained on lipreading (Haliassos et al., 2021).”
InfluentialAdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition
Chen, S., Ge, C., Tong, Z., Wang, J., Tong, Z., Wang, J., & Luo, P. (2022) · neural information processing systems · doi:10.52202/068431-1212 · Cited by 451
WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection
Zi, B., Chang, M., Chen, J., Ma, X., & Jiang, Y.-G. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413769 · Cited by 448
Recurrent convolutional strategies for face manipulation detection in videos
Sabir, E., Cheng, J., Jaiswal, A., AbdAlmageed, W., Masi, I., & Natarajan, P. (2019) · arXiv · doi:10.48550/arxiv.1905.00582 · Cited by 443
End-to-End Reconstruction-Classification Learning for Face Forgery Detection
Cao, J., Ma, C., Yao, T., Chen, S., Ding, S., & Yang, X. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00408 · Cited by 428 · cited 5×
“It shows that DeepFake-Adapter based on various ViT architectures can obtain the best or second-best performance compared to the SOTA method RECCE in both intra-and cross-manipulationsettings.”
InfluentialMulti-Adversarial Discriminative Deep Domain Generalization for Face Presentation Attack Detection
Shao, R., Lan, X., Li, J., & Yuen, P. C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01026 · Cited by 402
Self-citationGlobal Texture Enhancement for Fake Face Detection in the Wild
Liu, Z., Qi, X., & Torr, P. H. S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00808 · Cited by 401
Self-citationLearning Self-Consistency for Deepfake Detection
Zhao, T., Xu, X., Xu, M., Ding, H., Xiong, Y., & Xia, W. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01475 · Cited by 383 · cited 7×
“…existing deepfake detection methods merely focus on exploiting low-level forgery features from local textures (Chen et al., 2021; Gu et al., 2022; Zhao et al., 2021; Liu et al., 2020; Shao et al., 2019, 2018), blending boundary (Li et al., 2020), or frequency information (Li et al., 2021; Qian…”
InfluentialWhat Makes Fake Images Detectable? Understanding Properties that Generalize
Chai, L., Bau, D., Lim, S.-N., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58574-7_7 · Cited by 357
Li, J., Xie, H., Li, J., Wang, Z., & Zhang, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00639 · Cited by 357 · cited 4×
“What’smore,afrequency-awarediscriminativefeaturelearn-ing framework is proposed to perform metric learning in frequency features (Li et al., 2021).”
InfluentialLocal Relation Learning for Face Forgery Detection
Chen, S., Yao, T., Chen, Y., Ding, S., Li, J., & Ji, R. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16193 · Cited by 334
Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection
Chen, L., Zhang, Y., Song, Y., Liu, L., & Wang, J. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01815 · Cited by 307 · cited 5×
“0 datasets, following the setting proposed in SLADD [5].”
InfluentialRepresentative Forgery Mining for Fake Face Detection
Wang, C., & Deng, W. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01468 · Cited by 288
DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms
Qi, H., Guo, Q., Juefei-Xu, F., Xie, X., Ma, L., Feng, W., Liu, Y., & Zhao, J. (2020) · arXiv · doi:10.48550/arxiv.2006.07634 · Cited by 280
Towards a Unified View of Parameter-Efficient Transfer Learning
He, J., Zhou, C., Ma, X., Berg-Kirkpatrick, T., & Neubig, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.04366 · Cited by 268 · cited 4×
“Specifically, inspired by [6,21], as shown in Fig.”
InfluentialAdaptFormer: Adapting Vision Transformers for Scalable Visual RecognitionPreprint
Chen, S., Ge, C., Zhan, T., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.13535 · Cited by 261 · cited 5×
“Specifically, inspired by (Chen et al., 2022b; He et al., 2022), as shown in Fig.”
InfluentialTEINet: Towards an Efficient Architecture for Video Recognition
Liu, Z., Luo, D., Wang, Y., Wang, L., Tai, Y., Wang, C., Li, J., Huang, F., & Lu, T. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6836 · Cited by 254
Self-citationDeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms
Qi, H., Guo, Q., Juefei-Xu, F., Xie, X., Ma, L., Feng, W., Liu, Y., & Zhao, J. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413707 · Cited by 220
Regularized Fine-Grained Meta Face Anti-Spoofing
Shao, R., Lan, X., & Yuen, P. C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6873 · Cited by 196
Self-citationVision Transformer Adapter for Dense Predictions
Chen, Z., Duan, Y., Wang, W., He, J., Lu, T., Dai, J., & Qiao, Y. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.08534 · Cited by 195 · cited 5×
“…adaptation approaches, we compare it with the most widely-used tuning approaches, e.g., full-tuning and linear-probe (Zhuang et al., 2020), and two latest parameter-efficient adaptation methods named Visual Prompt Tuning (VPT) (Jia et al., 2022) and ViT-Adapter (Chen et al., 2023b) 123 in Table 8.”
InfluentialSpatiotemporal Inconsistency Learning for DeepFake Video Detection
Gu, Z., Chen, Y., Yao, T., Ding, S., Li, J., Huang, F., & Ma, L. (2021) · Proceedings of the 29th ACM International Conference on Multimedia · doi:10.1145/3474085.3475508 · Cited by 189
Unmasking DeepFakes with simple Features
Durall, R., Keuper, M., Pfreundt, F.-J., & Keuper, J. (2019) · arXiv · doi:10.48550/arxiv.1911.00686 · Cited by 184 · cited 1×
“High-frequency part of Discrete Fourier Transform (DFT) (Durall et al., 2019; Dzanic et al., 2020) are extracted to detect distinct spectrum distributions and characteristics between real and fake images.”
Protecting Celebrities from DeepFake with Identity Consistency Transformer
Dong, X., Bao, J., Chen, D., Zhang, T., Zhang, W., Yu, N., Chen, D., Wen, F., & Guo, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00925 · Cited by 181 · cited 1×
“We evaluate the proposed method and other baselines using the most commonly used metrics in related works (Cao et al., 2022; Dong et al., 2022; Chen et al., 2022a; Li et al., 2020; Chen et al., 2021; Qian et al., 2020; Zhao et al., 2021), including Accuracy (ACC), Area Under the Receiver Operating…”
Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection
Haliassos, A., Mira, R., Petridis, S., & Pantic, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01453 · Cited by 180 · cited 1×
“RealForensics (Haliassos et al., 2022) is also another work to exploit generalizable high-level temporal features by studying the natural correspondence between the visual and auditory modalities based on a self-supervised cross-modal manner.”
Exploiting Fine-Grained Face Forgery Clues via Progressive Enhancement Learning
Gu, Q., Chen, S., Yao, T., Chen, Y., Ding, S., & Yi, R. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19954 · Cited by 158
Sharp Multiple Instance Learning for DeepFake Video Detection
Li, X., Lang, Y., Chen, Y., Mao, X., He, Y., Wang, S., Xue, H., & Lu, Q. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3414034 · Cited by 157 · cited 11×
“…proposed method and other baselines using the most commonly used metrics in related works (Cao et al., 2022; Dong et al., 2022; Chen et al., 2022a; Li et al., 2020; Chen et al., 2021; Qian et al., 2020; Zhao et al., 2021), including Accuracy (ACC), Area Under the Receiver Operating Characteristic…”
InfluentialFine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution
Kumar, A., Raghunathan, A., Jones, R., Ma, T., & Liang, P. (2022) · arXiv · doi:10.48550/arxiv.2202.10054 · Cited by 154
LION : Empowering Multimodal Large Language Model with Dual-Level Visual Knowledge
Chen, G., Shen, L., Shao, R., Deng, X., & Nie, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02506 · Cited by 153 · cited 1×
“…and linear-probe (Zhuang et al., 2020), parameter-efficient tuning methods only need to train a small portion of model parameters in consideration of the rapid increase in model size of large pre-trained language models (Li et al., 2024; Chen et al., 2024; Shen et al., 2024; Ye et al., 2024).”
Self-citationCeleb-DF: A New Dataset for DeepFake ForensicsPreprint
Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2019) · arXiv (Cornell University) · Cited by 137
Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks
Li, Z., Xie, Y., Shao, R., Chen, G., Jiang, D., & Nie, L. (2024) · neural information processing systems · doi:10.52202/079017-1580 · Cited by 129
Self-citationPseudo Numerical Methods for Diffusion Models on Manifolds
Liu, L., Ren, Y., Lin, Z., & Zhao, Z. L. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.09778 · Cited by 123 · cited 1×
“We further test our model on deepfake samples generated by 5 representative diffusion modes, such as DDPM (Ho et al., 2020a), IDDPM (Quinn & Dhariwal, 2021), ADM (Dhariwal & Nichol, 2021a), PNDM (Liu et al., 2022), and LDM (Rombach et al., 2022).”
Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon TasksPreprint
Li, Z.-J., Xie, Y., Shao, R., et al. (2024) · Neural Information Processing Systems · doi:10.48550/arxiv.2408.03615 · Cited by 111 · cited 1×
“…and linear-probe (Zhuang et al., 2020), parameter-efficient tuning methods only need to train a small portion of model parameters in consideration of the rapid increase in model size of large pre-trained language models (Li et al., 2024; Chen et al., 2024; Shen et al., 2024; Ye et al., 2024).”
Self-citationFace Forgery Detection by 3D Decomposition
Zhu, X., Wang, H., Fei, H., Lei, Z., & Li, S. Z. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00295 · Cited by 106 · cited 2×
“Various local textures (Chen et al., 2021; Gu et al., 2022; Zhao et al., 2021; Zhu et al., 2021; Liu et al., 2020) are intensively analyzed and explored to highlight the appearance differences between real and forged faces.”
Joint Discriminative Learning of Deep Dynamic Textures for 3D Mask Face Anti-Spoofing
Shao, R., Lan, X., & Yuen, P. C. (2019) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2018.2868230 · Cited by 103
Self-citationMoME: Mixture of Multimodal Experts for Generalist Multimodal Large Language Models
Shen, L., Chen, G., Shao, R., Guan, W., & Nie, L. (2024) · neural information processing systems · doi:10.52202/079017-1330 · Cited by 96
Self-citationDelving into the Local: Dynamic Inconsistency Learning for DeepFake Video Detection
Gu, Z., Chen, Y., Yao, T., Ding, S., Li, J., & Ma, L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19955 · Cited by 93 · cited 11×
“It aims to adapt more local low-level forgery features, such as local textures [7,19,58,38] for our task.”
InfluentialDetecting and Recovering Sequential DeepFake Manipulation
Shao, R., Wu, T., & Liu, Z. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19778-9_41 · Cited by 85
Self-citationCAT: Enhancing Multimodal Large Language Model to Answer Questions in Dynamic Audio-Visual Scenarios
Ye, Q., Yu, Z., Shao, R., Xie, X., Torr, P. H. S., & Cao, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72684-2_9 · Cited by 85
Self-citationDetecting and Grounding Multi-Modal Media Manipulation
Shao, R., Wu, T., & Liu, Z. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00667 · Cited by 76 · cited 1×
“This leads to serious misinformation and fabrication problems in politics (Shao et al., 2023, 2024, 2022,b, 2023), entertainment and society Communicated by Maja Pantic.”
Self-citationTowards the Detection of Diffusion Model Deepfakes
Ricker, J., Damm, S., Holz, T., & Fischer, A. (2024) · International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) · doi:10.5220/0012422000003660 · Cited by 62
Detecting and Grounding Multi-Modal Media Manipulation and Beyond
Shao, R., Wu, T., Wu, J., Nie, L., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367749 · Cited by 53 · cited 1×
“This leads to serious misinformation and fabrication problems in politics (Shao et al., 2023, 2024, 2022,b, 2023), entertainment and society Communicated by Maja Pantic.”
Self-citationV4D:4D Convolutional Neural Networks for Video-level Representation Learning
Zhang, S., Guo, S., Huang, W., Scott, M. R., & Wang, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.07442 · Cited by 46
Federated Generalized Face Presentation Attack Detection
Shao, R., Perera, P., Yuen, P. C., & Patel, V. M. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3172316 · Cited by 44 · cited 2×
“This leads to serious misinformation and fabrication problems in politics (Shao et al., 2023, 2024, 2022,b, 2023), entertainment and society Communicated by Maja Pantic.”
Self-citationPoster: Towards Robust Open-World Detection of Deepfakes
Sohrawardi, S. J., Chintha, A., Thai, B., Seng, S., Hickerson, A., Ptucha, R., & Wright, M. (2019) · Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3319535.3363269 · Cited by 41
Open-Set Adversarial Defense with Clean-Adversarial Mutual Learning
Shao, R., Perera, P., Yuen, P. C., & Patel, V. M. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01581-0 · Cited by 41 · cited 2×
“This leads to serious misinformation and fabrication problems in politics (Shao et al., 2023, 2024, 2022,b, 2023), entertainment and society Communicated by Maja Pantic.”
Self-citationDSANet: Dynamic Segment Aggregation Network for Video-Level Representation Learning
Wu, W., Zhao, Y., Xu, Y., Tan, X., He, D., Zou, Z., Ye, J., Li, Y., Yao, M., Dong, Z., et al. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475344 · Cited by 40 · cited 1×
“Inspired by recent works (Yuan et al., 2021; Wu et al., 2021) that integrate convolutional operations of CNNs into a ViT, we introduce the convolution-based head part of LSA.”
Patch Diffusion: A General Module for Face Manipulation Detection
Zhang, B., Li, S., Feng, G., Qian, Z., & Zhang, X. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20233 · Cited by 37 · cited 2×
“Patch diffusion [56] and patch inconsistency [59] are also studied to explore the distinct correlation consistency among local patch features between real and forgery faces.”
MoME: Mixture of Multimodal Experts for Generalist Multimodal Large Language ModelsPreprint
Shen, L., Chen, G., Shao, R., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.12709 · Cited by 4
Self-citationFourier spectrum discrepancies in deep network generated images
Dzanic, T., Shah, K., & Witherden, F. (2019) · arXiv · doi:10.48550/arxiv.1911.06465 · cited 1×
“High-frequency part of Discrete Fourier Transform (DFT) (Durall et al., 2019; Dzanic et al., 2020) are extracted to detect distinct spectrum distributions and characteristics between real and fake images.”
Research works printed but not linked: 13
- Agarap, A.F. (2018). Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375. · arXiv
- Ba, J.L., Kiros, J.R., & Hinton, G.E. (2016). Layer normalization. arXiv preprint arXiv:1607.06450. · arXiv
- Chen, Z., Duan, Y., Wang, W., He, J., Lu, T., Dai, J., & Qiao, Y. (2023). Vision transformer adapter for dense predictions. In: ICLR.
- Dhariwal, P., & Nichol, A. (2021). Diffusion models beat gans on image synthesis. NeurIPS, 34, 8780–8794.
- Dolhansky, B., Bitton, J., Pflaum, B., Lu, J., Howes, R., Wang, M., & Ferrer, C.C. (2020). The deepfake detection challenge (dfdc) dataset. arXiv preprint arXiv:2006.07397. · arXiv
- Ho, J., Jain, A., & Abbeel, P. (2020). Denoising diffusion probabilistic models. In: NeurIPS, 33, 6840–6851.
- Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2022). Lora: Low-rank adaptation of large language models. In: ICLR.
- Liu, L., Ren, Y., Lin, Z., & Zhao, Z. (2022). Pseudo numerical methods for diffusion models on manifolds. In: ICLR.
- Van der Maaten, L., & Hinton, G. (2008). Visualizing data using t-sne. Journal of machine learning research 9(11).
- Quinn, N.A., & Dhariwal, P. (2021). Improved denoising diffusion probabilistic models. In: ICML, pp. 8162–8171.
- Shao, R., Wu, T., & Liu, Z. (2023). Robust sequential deepfake detection. arXiv preprint arXiv:2309.14991. · arXiv
- Tran, D., Bourdev, L.D., Fergus, R., Torresani, L., & Paluri, M. (2014). C3d: generic features for video analysis. CoRR, abs/1412.0767. · arXiv
- Wang, G., Zhou, J., & Wu, Y. (2020). Exposing deep-faked videos by anomalous co-motion pattern detection. arXiv preprint arXiv:2008.04848. · arXiv
Cited by
A Review of Deepfake Techniques: Architecture, Detection and Datasets
Edwards, P., Nebel, J.-C., Greenhill, D., & Liang, X. (2024) · IEEE Access · doi:10.1109/access.2024.3477257 · Cited by 50
“In the study [56], low and high-level semantic information is used to evaluate how discriminative features provide separability amongst real and fake distributions, trained separately using a pre-trained Xception and ViT network.”
BackgroundMethodInfluentialForgery-Aware Adaptive Learning With Vision Transformer for Generalized Face Forgery Detection
Luo, A., Cai, R., Kong, C., Ju, Y., Kang, X., Huang, J., & Kot, A. C. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3522091 · Cited by 41
“[56] propose a dual-level adapter based detector to capture both global and local low-level manipulation traces for face forgery detection.”
Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey and Benchmark
Xin, Y., Yang, J., Luo, S., Du, Y., Qin, Q., Chen, H., Cen, K., He, Y., Fu, B., Cao, Y., et al. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02864-6 · Cited by 202
Robust Sequential DeepFake Detection
Shao, R., Wu, T., & Liu, Z. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-024-02339-6 · Cited by 36
Self-citationDeepFake detection in the AIGC era: A survey, benchmarks, and future perspectives
Xie, S., Qiao, T., Li, S., Zhang, X., Zhou, J., & Feng, G. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103740 · Cited by 14
Metadata
- Authors
- Shao, R., Wu, T., Nie, L., & Liu, Z.
- Affiliations
- Harbin Institute of Technology (CN)
- Nanyang Technological University (SG)
- Corresponding author
- Liu, Ziwei
- Venue
- International Journal of Computer Vision
- Published
- 2025-01-24
- Volume
- 133
- Issue
- 6
- Pages
- 3613-3628
- ISSN
- 0920-5691 (ISSN-L)
- 1573-1405
- OpenAlex
- W4406804129
- Cited by
- 48
- References
- 79
- Citations by year
- 2026: 21 · 2025: 22
- FWCI
- 15.10
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- International Journal of Computer Vision (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- DR-NTU (Nanyang Technological University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- deepfake detection, parameter-efficient fine-tuning, vision transformer, high-level semantics, generalization to unseen data, cross-dataset generalization, frozen backbone
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.