DFDT: An End-to-End DeepFake Detection Framework Using Vision Transformer
Open accessThe DOI matches its Crossref record; 94 of the 94 references it cites were found in a registry or an index; 19 of the 25 sources its claims cite were checked for support.
Abstract
The ever-growing threat of deepfakes and large-scale societal implications has propelled the development of deepfake forensics to ascertain the trustworthiness of digital media. A common theme of existing detection methods is using Convolutional Neural Networks (CNNs) as a backbone. While CNNs have demonstrated decent performance on learning local discriminative information, they fail to learn relative spatial features and lose important information due to constrained receptive fields. Motivated by the aforementioned challenges, this work presents DFDT, an end-to-end deepfake detection framework that leverages the unique characteristics of transformer models, for learning hidden traces of perturbations from both local image features and global relationship of pixels at different forgery scales. DFDT is specifically designed for deepfake detection tasks consisting of four main components: patch extraction & embedding, multi-stream transformer block, attention-based patch selection followed by a multi-scale classifier. DFDT’s transformer layer benefits from a re-attention mechanism instead of a traditional multi-head self-attention layer. To evaluate the performance of DFDT, a comprehensive set of experiments are conducted on several deepfake forensics benchmarks. Obtained results demonstrated the surpassing detection rate of DFDT, achieving 99.41%, 99.31%, and 81.35% on FaceForensics++, Celeb-DF (V2), and WildDeepfake, respectively. Moreover, DFDT’s excellent cross-dataset & cross-manipulation generalization provides additional strong evidence on its effectiveness.
Overview
CoChat Summary
TL;DR
DFDT is an end-to-end vision-transformer deepfake detector combining patch/position embeddings, multi-stream transformer blocks, attention-based patch selection, and a multi-scale classifier. It reports state-of-the-art intra-dataset accuracy and strong cross-dataset/cross-manipulation generalization on several benchmarks.
Key findings
- Frame-level intra-dataset results: 99.41% ACC / 99.94% AUC on FaceForensics++ (raw), 99.31% ACC / 99.26% AUC on Celeb-DF (V2), and 81.35% ACC / 80.74% AUC on WildDeepfake.
- Video-level cross-dataset generalization when trained on FaceForensics++: AUC 88.3% on Celeb-DF, 76.1% on DFDC, 97.8% on FaceShifter, 96.9% on DeeperForensics, average 89.7%; LipForensics is reported at 82.4%, 73.5%, 97.1%, 97.6%, average 87.8%.
- Cross-manipulation generalization on FaceForensics++ subsets: DFDT average AUC 97.9% (Deepfakes 99.8%, FaceSwap 93.1%, Face2Face 99.6%, NeuralTextures 99.2%), compared with LipForensics average 97.1%.
- Ablation: removing the re-attention mechanism decreases AUC by 1.7% on FaceForensics++, 0.9% on Celeb-DF (V2), and 1.4% on WildDeepfake.
Methods
- DFDT consists of patch extraction & embedding, a multi-stream transformer block, attention-based patch selection, and a multi-scale classifier; its transformer layer uses a re-attention mechanism instead of traditional multi-head self-attention, with low-level and high-level branches using different patch sizes.
- Preprocessing extracts 20% of frames per video, uses RetinaFace for facial landmarks, crops/resizes/aligns the face area, and applies overlapping patch extraction with positional embeddings.
- Evaluation uses FaceForensics++, Celeb-DF (V2), and WildDeepfake for intra-dataset tests; DFDC, FaceShifter, and DeeperForensics for cross-dataset tests; 80% of videos are used for training, with the rest equally divided into test and validation. Metrics are ACC and AUC; implementation uses PyTorch, Adam with learning rate 10^-4 and tenfold decay every 40 steps, trained for 100 epochs.
Limitations
- The paper does not include a dedicated limitations section; it states that performance on more challenging real-world datasets like WildDeepfake is “still far from perfect” and reports lower WildDeepfake scores (81.35% ACC, 80.74% AUC) than on FaceForensics++/Celeb-DF.
- Evident from the reported results: cross-dataset DeeperForensics AUC is 96.9%, slightly below LipForensics’ 97.6%, so DFDT is competitive rather than superior there.
- Evident from the text: no explicit discussion of computational cost, inference speed, or robustness beyond the tested benchmarks; the paper reports hardware setup but not efficiency comparisons.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
13 Valid · 3 Not valid · 0 Not checked · 19 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“With the same rationale as in [17,68,70,76], the experiments are performed using the high-quality subset of the FaceForensics++ dataset. It is more likely to be closer to real-world deepfake videos, i.e., where videos are processed with nearly lossless compression.”
What the sources say
- Supportedcites [17] Haliassos, A., Vougioukas, K., Petridis, S., and Pantic, M. (2021, January 19–25). Lips Don’t Lie: A Generalisable and Robust Approach To Face Forgery Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA. source
Section 4.2 (p. 4) says the authors use the high-quality FaceForensics++ subset, whose videos have undergone visually nearly lossless compression. It states that this is more in line with videos found on social media, supporting the citing sentence’s comparison to real-world deepfake videos. (LLM, conf=0.92)
- Not checked — no full text of the sourcecites [68] Nguyen, H.H., Fang, F., Yamagishi, J., and Echizen, I. (2019, January 23–26). Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos. Proceedings of the 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS), Tampa, FL, USA. source
- Disputedcites [70] Masi, I., Killekar, A., Mascarenhas, R.M., Gurudatt, S.P., and AbdAlmageed, W. (2020, January 23–28). Two-branch recurrent network for isolating deepfakes in videos. Proceedings of the European Conference on Computer Vision, Glasgow, UK. sourceExperimental — accuracy not yet measured
In Prior Work (p. 3), the paper says the perceived quality of videos in FaceForensics++ and similar datasets remains lower than that of videos circulating on the web. Its FaceForensics++ experiments (p. 11–12) use medium- and high-compression subsets, not a nearly lossless subset presented as closer to real-world deepfakes. (LLM, conf=0.78)
Checked by a reasoning model — may be wrong.
- Not checked — no full text of the sourcecites [76] Chai, L., Bau, D., Lim, S.N., and Isola, P. (2020, January 23–28). What makes fake images detectable understanding properties that generalize. Proceedings of the European Conference on Computer Vision, Glasgow, UK. source
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“In addition, attention-based mechanisms have demonstrated their strong capability in improving the performance of traditional CNN-based deepfake detection models [18].”
What the sources say
- Supportedcites [18] Khormali, A., and Yuan, J.S. (2021). ADD: Attention-Based DeepFake Detection Approach. Big Data Cogn. Comput., 5. source
The Abstract describes ADD as an attention-based method applicable to CNN classifiers and reports significantly improved performance for four baseline classifiers on both datasets. Section 5.2 likewise reports that ADD boosted all baseline models above 77% accuracy on WildDeepFake. (LLM, conf=0.93)
states a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“GAN-generated digital content and not actual events captured by a camera implies that they still can be detected using advanced AI models [13].”
What the sources say
- Supportedcites [13] Du, M., Pentyala, S., Li, Y., and Hu, X. (2020, January 19–23). Towards Generalizable Deepfake Detection with Locality-aware AutoEncoder. Proceedings of the 29th ACM International Conference on Information &Knowledge Management, Virtual. source
The Introduction (p. 1) includes GAN-based image and video manipulations among deepfakes. It explains that, because these forgeries are generated by algorithms rather than captured by a real camera, deep learning models can extract features to detect them. (LLM, conf=0.91)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Furthermore, it has been proven that deep neural networks tend to achieve better performance than traditional image forensic tools [9].”
What the sources say
- Partly supportedcites [9] Mirsky, 2021, The creation and detection of deepfakes: A survey, ACM Comput. Surv. (CSUR) sourceExperimental — accuracy not yet measured
The Detection section (p. 28) reports that deep neural networks tend to outperform traditional image forensic tools, but specifically on compressed imagery. The citing sentence omits that condition and presents the finding more broadly. (LLM, conf=0.88)
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The re-attention mechanism re-generates the attention maps through establishing cross-head communications in an attempt to increase the diversity of attention maps at different layers.”
What the sources say
- Supportedcites [54] Zhou, D., Kang, B., Jin, X., Yang, L., Lian, X., Jiang, Z., Hou, Q., and Feng, J. (2021). Deepvit: Towards deeper vision transformer. arXiv. source
Section 4.2 (p. 5) says Re-attention establishes cross-head communication to regenerate attention maps. Section 5.3 (p. 8) says this process reduces similarity between attention maps and achieves the lowest cross-layer attention-map similarity, supporting the stated aim of increasing diversity across layers. (LLM, conf=0.94)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
13 more claims · 2 with their cited sources checked
Create a free account to see all 35 claims and every source verdict.
- Probably not a substantial claim. 7
- It may state two separate claims as one. 9
- The quote only partly states this claim. 1
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 5.1. Intra-Dataset Evaluation
“| FaceForensics++ | (Raw) | 99.41 | 99.94 |”
§ 5.1. Intra-Dataset Evaluation
“| DFDT (Ours) | 99.7 | DFDT (Ours) | 99.2 | DFDT (Ours) | 81.3 |”
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · arXiv · doi:10.48550/arxiv.2005.12872 · Cited by 18,253
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218
Deformable DETR: Deformable Transformers for End-to-End Object Detection
Zhu, X., Su, W., Lu, L., Li, B., Wang, X., & Dai, J. (2020) · arXiv · doi:10.48550/arxiv.2010.04159 · Cited by 6,027 · cited 1×
“Transformer models have proven their strong capabilities on learning long-term dependency on natural language processing tasks [19–21], and more recently on computer vision tasks [22–24].”
BackgroundGenerative Adversarial Networks
Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.2661 · Cited by 4,477
TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A. L., & Zhou, Y. (2021) · arXiv · doi:10.48550/arxiv.2102.04306 · Cited by 4,206
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314
One millisecond face alignment with an ensemble of regression trees
Kazemi, V., & Sullivan, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.241 · Cited by 2,831
GPT-3: Its Nature, Scope, Limits, and Consequences
Floridi, L., & Chiriatti, M. (2020) · Minds and Machines · doi:10.1007/s11023-020-09548-1 · Cited by 2,519
MesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · International Workshop on Information Forensics and Security · doi:10.48550/arxiv.1809.00888 · Cited by 1,940
MesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics
Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00327 · Cited by 1,846
Face2Face: Real-Time Face Capture and Reenactment of RGB Videos
Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., & Niessner, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.262 · Cited by 1,747
Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
Kim, T., Cha, M., Kim, H., Lee, J. K., & Kim, J. (2017) · arXiv · doi:10.48550/arxiv.1703.05192 · Cited by 1,716
RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild
Deng, J., Guo, J., Ververas, E., Kotsia, I., & Zafeiriou, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00525 · Cited by 1,681
Deferred neural rendering: image synthesis using neural textures
Thies, J., Zollhöfer, M., & Nießner, M. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323035 · Cited by 1,356
Deepfake Video Detection Using Recurrent Neural Networks
Guera, D., & Delp, E. J. (2018) · 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2018.8639163 · Cited by 1,285
Face X-Ray for More General Face Forgery Detection
Li, L., Bao, J., Zhang, T., Yang, H., Chen, D., Wen, F., & Guo, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00505 · Cited by 1,211
CNN-Generated Images Are Surprisingly Easy to Spot… for Now
Wang, S.-Y., Wang, O., Zhang, R., Owens, A., & Efros, A. A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00872 · Cited by 1,210
Exposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196
In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking
Li, Y., Chang, M.-C., & Lyu, S. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630787 · Cited by 1,089
Thinking in Frequency: Face Forgery Detection by Mining Frequency-Aware Clues
Qian, Y., Yin, G., Sheng, L., Chen, Z., & Shao, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58610-2_6 · Cited by 1,058
Multi-attentional Deepfake Detection
Zhao, H., Wei, T., Zhou, W., Zhang, W., Chen, D., & Yu, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00222 · Cited by 1,019 · cited 1×
“These two evaluation metrics commonly have been used in existing deepfake detection tasks [17,61,62].”
MethodQuantifying Attention Flow in Transformers
Abnar, S., & Zuidema, W. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.385 · Cited by 995
Vaccari, C., & Chadwick, A. (2020) · Social Media + Society · doi:10.1177/2056305120903408 · Cited by 978 · cited 1×
“Deepfakes are among the most sinister types of misinformation, posing large-scale and severe security and privacy risks targeting critical governmental institutions and ordinary people across the world [5,6].”
BackgroundCapsule-forensics: Using Capsule Networks to Detect Forged Images and Videos
Nguyen, H. H., Yamagishi, J., & Echizen, I. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682602 · Cited by 951
The Creation and Detection of Deepfakes: A Survey
Mirsky, Y., & Lee, W. (2022) · ACM Computing Surveys · doi:10.1145/3425780 · Cited by 863 · cited 3×
“Furthermore, it has been proven that deep neural networks tend to achieve better performance than traditional image forensic tools [9].”
BackgroundThe Creation and Detection of Deepfakes: A Survey
Mirsky, Y., & Lee, W. (2020) · arXiv · doi:10.48550/arxiv.2004.11138 · Cited by 863
Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations
Matern, F., Riess, C., & Stamminger, M. (2019) · 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw.2019.00020 · Cited by 824
Exposing DeepFake Videos By Detecting Face Warping Artifacts
Li, Y., & Lyu, S. (2018) · arXiv · doi:10.48550/arxiv.1811.00656 · Cited by 785 · cited 4×
“Two-stream [83] Multi-task [89] HeadPose [86] Meso4 [34] VA-MLP [46] FWA [12] Capsule [90] DSP-FWA [12] F3-Net [94] Xception-c40 [69] TBRN [91] SPSL [92] Face X-ray [93] PPA [95] FakeCatcher [45] ADD-ResNet [55] DFDT (Ours) AUC (%)”
BackgroundInfluentialRethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2015) · arXiv · doi:10.48550/arxiv.1512.00567 · Cited by 741
Two-Stream Neural Networks for Tampered Face Detection
Zhou, P., Han, X., Morariu, V. I., & Davis, L. S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.229 · Cited by 700
Vision Transformers for Remote Sensing Image Classification
Bazi, Y., Bashmal, L., Rahhal, M. M. A., Dayil, R. A., & Ajlan, N. A. (2021) · Remote Sensing · doi:10.3390/rs13030516 · Cited by 637
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain
Liu, H., Li, X., Zhou, W., Chen, Y., He, Y., Xue, H., Zhang, W., & Yu, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00083 · Cited by 600
DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection
Jiang, L., Li, R., Wu, W., Qian, C., & Loy, C. C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00296 · Cited by 599
TransFG: A Transformer Architecture for Fine-Grained Recognition
He, J., Chen, J.-N., Liu, S., Kortylewski, A., Yang, C., Bai, Y., & Wang, C. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19967 · Cited by 551
Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos
Nguyen, H. H., Fang, F., Yamagishi, J., & Echizen, I. (2019) · 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS) · doi:10.1109/btas46853.2019.9185974 · Cited by 549
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2016) · arXiv · doi:10.48550/arxiv.1610.02357 · Cited by 538
Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection
Haliassos, A., Vougioukas, K., Petridis, S., & Pantic, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00500 · Cited by 537 · cited 5×
“These two evaluation metrics commonly have been used in existing deepfake detection tasks [17,61,62].”
BackgroundMethodInfluentialFace aging with conditional generative adversarial networks
Antipov, G., Baccouche, M., & Dugelay, J.-L. (2017) · 2017 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2017.8296650 · Cited by 515
FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals
Ciftci, U. A., Demir, I., & Yin, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3009287 · Cited by 499
Kim, H., Garrido, P., Tewari, A., Xu, W., Thies, J., Niessner, M., Pérez, P., Richardt, C., Zollhöfer, M., & Theobalt, C. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201283 · Cited by 475
Two-Branch Recurrent Network for Isolating Deepfakes in Videos
Masi, I., Killekar, A., Mascarenhas, R. M., Gurudatt, S. P., & AbdAlmageed, W. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58571-6_39 · Cited by 471
WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection
Zi, B., Chang, M., Chen, J., Ma, X., & Jiang, Y.-G. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413769 · Cited by 448
Recurrent convolutional strategies for face manipulation detection in videos
Sabir, E., Cheng, J., Jaiswal, A., AbdAlmageed, W., Masi, I., & Natarajan, P. (2019) · arXiv · doi:10.48550/arxiv.1905.00582 · Cited by 443
The deepfake detection challenge (dfdc) dataset
Dolhansky, B., Bitton, J., Pflaum, B., Lu, J., Howes, R., Wang, M., & Ferrer, C. C. (2020) · arXiv · doi:10.48550/arxiv.2006.07397 · Cited by 427
The Deepfake Detection Challenge (DFDC) Preview Dataset
Dolhansky, B., Howes, R., Pflaum, B., Baram, N., & Ferrer, C. C. (2019) · arXiv · doi:10.48550/arxiv.1910.08854 · Cited by 413
DeepViT: Towards Deeper Vision Transformer
Zhou, D., Kang, B., Jin, X., Yang, L., Lian, X., Jiang, Z., Hou, Q., & Feng, J. (2021) · arXiv · doi:10.48550/arxiv.2103.11886 · Cited by 398
M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection
Wang, J., Wu, Z., Ouyang, W., Han, X., Chen, J., Jiang, Y.-G., & Li, S.-N. (2022) · Proceedings of the 2022 International Conference on Multimedia Retrieval · doi:10.1145/3512527.3531415 · Cited by 368
What Makes Fake Images Detectable? Understanding Properties that Generalize
Chai, L., Bau, D., Lim, S.-N., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58574-7_7 · Cited by 357
Maras, M.-H., & Alexandrou, A. (2019) · The International Journal of Evidence & Proof · doi:10.1177/1365712718807226 · Cited by 343 · cited 1×
“Deepfakes are synthesized yet super-realistic images and videos generated through combining, merging, superimposing, or replacing the facial area of images/videos leveraging advanced techniques from computer vision and deep learning domains [4].”
BackgroundUncertainty-Guided Transformer Reasoning for Camouflaged Object Detection
Yang, F., Zhai, Q., Li, X., Huang, R., Luo, A., Cheng, H., & Fan, D.-P. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00411 · Cited by 332
Emotions Don't Lie: An Audio-Visual Deepfake Detection Method using Affective Cues
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., & Manocha, D. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413570 · Cited by 319 · cited 1×
“[40] proposed a multi-modal approach composed of audio and video modalities to tackle deepfake detection tasks.”
BackgroundFaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping
Li, L., Bao, J., Yang, H., Chen, D., & Wen, F. (2019) · arXiv · doi:10.48550/arxiv.1912.13457 · Cited by 236
Attribute-Guided Face Generation Using Conditional CycleGAN
Lu, Y., Tai, Y.-W., & Tang, C.-K. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01258-8_18 · Cited by 233
ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection
Cozzolino, D., Thies, J., Rössler, A., Riess, C., Nießner, M., & Verdoliva, L. (2018) · arXiv · doi:10.48550/arxiv.1812.02510 · Cited by 221
In Ictu Oculi: Exposing AI Generated Fake Face Videos by Detecting Eye Blinking
Li, Y., Chang, M., & Lyu, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.02877 · Cited by 169
DeepFake video detection using convolutional vision transformer
Wodajo, D., & Atnafu, S. (2021) · arXiv · doi:10.48550/arxiv.2102.11126 · Cited by 147
DeepfakeStack: A Deep Ensemble-based Learning Technique for Deepfake Detection
Rana, M. S., & Sung, A. H. (2020) · 2020 7th IEEE International Conference on Cyber Security and Cloud Computing (CSCloud)/2020 6th IEEE International Conference on Edge Computing and Scalable Cloud (EdgeCom) · doi:10.1109/cscloud-edgecom49738.2020.00021 · Cited by 133 · cited 1×
“Furthermore, an ensemble learning approach was employed to improve deepfake detection composite model by Rana et al. [37].”
MethodUse of a Capsule Network to Detect Fake Images and Videos
Nguyen, H. H., Yamagishi, J., & Echizen, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12467 · Cited by 121
Towards Generalizable Deepfake Detection with Locality-aware AutoEncoder
Du, M., Pentyala, S., Li, Y., & Hu, X. (2020) · Proceedings of the 29th ACM International Conference on Information & Knowledge Management · doi:10.1145/3340531.3411892 · Cited by 114 · cited 2×
“GAN-generated digital content and not actual events captured by a camera implies that they still can be detected using advanced AI models [13].”
BackgroundMethodDefakeHop: A Light-Weight High-Performance Deepfake Detector
Chen, H.-S., Rouhsedaghat, M., Ghani, H., Hu, S., You, S., & Jay Kuo, C.-C. (2021) · 2021 IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme51207.2021.9428361 · Cited by 100
A Comprehensive Survey on Passive Techniques for Digital Video Forgery Detection
Shelke, N. A., & Kasana, S. S. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09974-4 · Cited by 87 · cited 2×
“Different deepfake detection approaches have been proposed to address this challenge [8,9].”
BackgroundVideo Transformer for Deepfake Detection with Incremental Learning
Khan, S. A., & Dai, H. (2021) · Proceedings of the 29th ACM International Conference on Multimedia · doi:10.1145/3474085.3475332 · Cited by 79
Progressive Transfer Learning for Face Anti-Spoofing
Quan, R., Wu, Y., Yu, X., & Yang, Y. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3066912 · Cited by 78
ADD: Attention-Based DeepFake Detection Approach
Khormali, A., & Yuan, J.-S. (2021) · Big Data and Cognitive Computing · doi:10.3390/bdcc5040049 · Cited by 55
Self-citationCapsule-Forensics: Using Capsule Networks to Detect Forged Images and VideosPreprint
Nguyen, H. H., Yamagishi, J., & Echizen, I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.11215 · Cited by 49
Tran, V.-N., Lee, S.-H., Le, H.-S., & Kwon, K.-R. (2021) · Applied Sciences · doi:10.3390/app11167678 · Cited by 47
Deferred Neural Rendering: Image Synthesis using Neural Textures
Thies, J., Zollhöfer, M., & Nießner, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.12356 · Cited by 46
Face Aging With Conditional Generative Adversarial NetworksPreprint
Antipov, G., Baccouche, M., & Dugelay, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.01983 · Cited by 44
Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware CluesPreprint
Qian, Y.-Y., Yin, G., Sheng, L., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.09355 · Cited by 42
Deepfake Detection Scheme Based on Vision Transformer and Distillation
Heo, Y. J., Choi, Y. J., Lee, Y., & Kim, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.01353 · Cited by 38
WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection
Zi, B., Chang, M., Chen, J., Ma, X., & Jiang, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.01456 · Cited by 37
Kaur, S., Kumar, P., & Kumaraguru, P. (2020) · Journal of Electronic Imaging · doi:10.1117/1.jei.29.3.033013 · Cited by 35
Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective CuesPreprint
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., & Manocha, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.06711 · Cited by 33
Detecting GANs and Retouching based Digital Alterations via DAD-HCNN
Jain, A., Majumdar, P., Singh, R., & Vatsa, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00344 · Cited by 29
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency DomainPreprint
Liu, H., Li, X., Zhou, W., Chen, Y., He, Y., Xue, H., Zhang, W., & Yu, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.01856 · Cited by 22
M2TR: Multi-modal Multi-scale Transformers for Deepfake DetectionPreprint
Wang, J., Wu, Z., Ouyang, W., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.09770 · Cited by 16
What makes fake images detectable? Understanding properties that generalizePreprint
Chai, L., Bau, D. A., Lim, S.-N., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.10588 · Cited by 13
Investigating the Impact of Pre-processing and Prediction Aggregation on the DeepFake Detection Task
Charitidis, P., Kordopatis-Zilos, G., Papadopoulos, S., & Kompatsiaris, I. Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07084 · Cited by 10
TransFG: A Transformer Architecture for Fine-grained RecognitionPreprint
Ju, H., Chen, J.-N., Liu, S., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.07976 · Cited by 4
Quantifying Attention Flow in Transformers
Abnar, S., & Zuidema, W. (2020) · arXiv · doi:10.48550/arxiv.2005.00928
Cited by
Abbas, F., & Taeihagh, A. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124260 · Cited by 184
TALL: Thumbnail Layout for Deepfake Video Detection
Xu, Y., Liang, J., Jia, G., Yang, Z., Zhang, Y., & He, R. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02071 · Cited by 109
A review of deep learning‐based approaches for deepfake content detection
(2024) · doi:10.1111/exsy.13570 · Cited by 116
A Contemporary Survey on Deepfake Detection: Datasets, Algorithms, and Challenges
Gong, L. Y., & Li, X. J. (2024) · Electronics · doi:10.3390/electronics13030585 · Cited by 75
“DFDT [44] proposes an end-to-end deepfake detection framework (see Figure 11) using vision transformers that can basically solve the problem that traditional CNN cannot obtain the correlation between spatial patches and the information loss caused by the receptive field.”
MethodPeng, C., Miao, Z., Liu, D., Wang, N., Hu, R., & Gao, X. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3381823 · Cited by 93
Metadata
- Authors
- Khormali, A., & Yuan, J.-S.
- Affiliations
- University of Central Florida (US)
- Corresponding author
- Yuan, Jiann-Shiun
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2022-03-14
- Volume
- 12
- Issue
- 6
- Pages
- 2953
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W4221045030
- Semantic Scholar
- ff1d47527279d353e29ef1071c7c58cc85c09146
- Cited by
- 88
- References
- 75
- Citations by year
- 2026: 20 · 2025: 31 · 2024: 22 · 2023: 13 · 2022: 2
- FWCI
- 5.56
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Applied Sciences (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
- Keywords
- deepfake detection, deepfake forensics, vision transformer, multi-scale classification, patch embedding, cross-dataset generalization, FaceForensics++, Celeb-DF
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.