Evaluation of an Audio-Video Multimodal Deepfake Dataset using Unimodal and Multimodal Detectors
Open accessThe DOI matches its Crossref record; 84 of the 84 references it cites were found in a registry or an index.
Abstract
Significant advancements made in the generation of deepfakes have caused security and privacy issues. Attackers can easily impersonate a person's identity in an image by replacing his face with the target person's face. Moreover, a new domain of cloning human voices using deep-learning technologies is also emerging. Now, an attacker can generate realistic cloned voices of humans using only a few seconds of audio of the target person. With the emerging threat of potential harm deepfakes can cause, researchers have proposed deepfake detection methods. However, they only focus on detecting a single modality, i.e., either video or audio. On the other hand, to develop a good deepfake detector that can cope with the recent advancements in deepfake generation, we need to have a detector that can detect deepfakes of multiple modalities, i.e., videos and audios. To build such a detector, we need a dataset that contains video and respective audio deepfakes. We were able to find a most recent deepfake dataset, Audio-Video Multimodal Deepfake Detection Dataset (FakeAVCeleb), that contains not only deepfake videos but synthesized fake audios as well. We used this multimodal deepfake dataset and performed detailed baseline experiments using state-of-the-art unimodal, ensemble-based, and multimodal detection methods to evaluate it. We conclude through detailed experimentation that unimodals, addressing only a single modality, video or audio, do not perform well compared to ensemble-based methods. Whereas purely multimodal-based baselines provide the worst performance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This may confuse the feature learning of a single modality, especially in ensemble-based fusion [9, 11, 15].”
From Integrating Audio-Visual Features For Multimodal Deepfake Detection (2023) · cites this paper as [11]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, existing feature fusion methods treat different types of deepfakes as the same fake type.”
From Integrating Audio-Visual Features For Multimodal Deepfake Detection (2023) · cites this paper as [11]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Moreover, the results of [38] show that an ensemble of audio and visual baselines outperforms monomodal counterparts.”
From A Robust Approach to Multimodal Deepfake Detection (2023) · cites this paper as [38]
The cited source says what the claim attributes to it.
In the citing paper
“Existing methods for audio-visual detection do not always surpass that of the analysis based on single modalities.”
From Integrating Audio-Visual Features For Multimodal Deepfake Detection (2023) · cites this paper as [11]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Estimating the Support of a High-Dimensional Distribution
Schölkopf, B., Platt, J. C., Shawe-Taylor, J., Smola, A. J., & Williamson, R. C. (2001) · Neural Computation · doi:10.1162/089976601750264965 · Cited by 6,168 · cited 1×
“Some of the most recent methods include detection based on splice detection in which they try to exploit the inconsistencies that arise from splicing near the boundaries of manipulated areas in an image [6, 94, 95] abnormal eye-blinking [38], one-class classification [27, 54, 65], irregular head poses [91] andmany oth…”
MethodGenerative Adversarial Networks
Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.2661 · Cited by 4,477
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 7×
“Google also a proposed Deepfake Detection dataset (DFD) [59] which contains 363 real videos and 3,000 deepfake videos, respectively.”
BackgroundMethodInfluentialVoxCeleb2: Deep Speaker Recognition
Chung, J. S., Nagrani, A., & Zisserman, A. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-1929 · Cited by 2,256
MesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics
Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00327 · Cited by 1,846 · cited 4×
“There is a significant amount of research done in generating deepfake videos and a number of deepfake video datasets are proposed by many researchers [15, 24, 32, 34, 40, 91].”
BackgroundMethodInfluential
Research works printed but not linked: 8
- [2] Shruti Agarwal, Hany Farid, Yuming Gu, Mingming He, Koki Nagano, and Hao Li. 2019. Protecting World Leaders Against Deep Fakes. In CVPR Workshops . CVPRW, Berlin, 38-45. -
- [16] Nick Dufour and Andrew Gully. 2019. Contributing data to deepfake detection research. Google AI Blog (2019). -
- [17] Baruch Epstein, Ron Meir, and Tomer Michaeli. 2018. Joint autoencoders: a flexible meta-learning framework. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases . Springer, 494-509. -
- [20] Md Rashidul Hasan, Mustafa Jamil, MGRMS Rahman, et al. 2004. Speaker identification using mel frequency cepstral coefficients. variations 1, 4 (2004), 565-568. -
- [31] Nikita Kireev and Eugene Ilyushin. 2020. Review of existing text-to-speech algorithms. International Journal of Open Information Technologies 8, 7 (2020), 84-90. -
- [41] Iain McCowan, Jean Carletta, Wessel Kraaij, Simone Ashby, S Bourban, M Flynn, M Guillemot, Thomas Hain, J Kadlec, Vasilis Karaiskos, et al. 2005. The AMI meeting corpus. In Proceedings of the 5th international conference on methods and techniques in behavioral research , Vol. 88. Citeseer, 100. -
- [52] Henri J Nussbaumer. 1981. The fast Fourier transform. In Fast Fourier Transform and Convolution Algorithms . Springer, 80-111. -
- [69] M Sridevi, C Mala, and Siddhant Sanyam. 2012. Comparative study of image forgery and copy-move techniques. Advances in Computer Science, Engineering & Applications (2012), 715-723. -
Cited by
A Review of Modern Audio Deepfake Detection Methods: Challenges and Future Directions
Almutairi, Z., & Elgibreen, H. (2022) · Algorithms · doi:10.3390/a15050155 · Cited by 187
“They used an image-based approach, which required a special transformation of the data, and the authors found that the model proposed failed to correctly classify new audio signals indicating that the model is not general enough.”
Ilyas, H., Javed, A., & Malik, K. M. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110124 · Cited by 186
FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset
Khalid, H., Tariq, S., Kim, M., & Woo, S. S. (2021) · arXiv · doi:10.48550/arxiv.2108.05080 · Cited by 116
Self-citationDeepfake Media Forensics: Status and Future Challenges
Amerini, I., Barni, M., Battiato, S., Bestagini, P., Boato, G., Bruni, V., Caldelli, R., De Natale, F., De Nicola, R., Guarnera, L., et al. (2025) · Journal of Imaging · doi:10.3390/jimaging11030073 · Cited by 113
Detecting Multimedia Generated by Large AI Models: A SurveyPreprint
Li, L., Gupta, N., Zhang, Y., Ren, H., Liu, C.-H., Ding, F., Wang, X., Li, X., Verdoliva, L., & Hu, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.00045 · Cited by 114
A systematic review of multimodal fake news detection on social media using deep learning models
Nasser, M., Arshad, N. I., Ali, A., Alhussian, H., Saeed, F., Da’u, A., & Nafea, I. T. (2025) · Results in Engineering · doi:10.1016/j.rineng.2025.104752 · Cited by 109
Shaaban, O. A., Yıldırım, R., & Alguttar, A. A. (2023) · IEEE Access · doi:10.1109/access.2023.3333866 · Cited by 107
Voice-Face Homogeneity Tells Deepfake
Cheng, H., Guo, Y., Wang, T., Li, Q., Chang, X., & Nie, L. (2024) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/3625231 · Cited by 102
Cai, Z., Stefanov, K., Dhall, A., & Hayat, M. (2022) · 2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta56598.2022.10034605 · Cited by 99
“Similar to [65] the proposed dataset includes three variations of deepfake data,”
ResultA Robust Approach to Multimodal Deepfake Detection
Salvi, D., Liu, H., Mandelli, S., Bestagini, P., Zhou, W., Zhang, W., & Tubaro, S. (2023) · Journal of Imaging · doi:10.3390/jimaging9060122 · Cited by 96
A Scientometric Analysis of Deep Learning Approaches for Detecting Fake News
Dhiman, P., Kaur, A., Iwendi, C., & Mohan, S. (2023) · Electronics · doi:10.3390/electronics12040948 · Cited by 86
Enhancing multimodal deepfake detection with local–global feature integration and diffusion models
Javed, M., Zhang, Z., Dahri, F. H., & Kumar, T. (2025) · Signal Image and Video Processing · doi:10.1007/s11760-025-03970-7 · Cited by 79
Balasubramanian, P., Liyana, S., Sankaran, H., Sivaramakrishnan, S., Pusuluri, S., Pirttikangas, S., & Peltonen, E. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11338-z · Cited by 84
Deepfake in the Metaverse: Security Implications for Virtual Gaming, Meetings, and Offices
Tariq, S., Abuadbba, A., & Moore, K. (2023) · doi:10.1145/3595353.3595880 · Cited by 77
Self-citationTariq, S., Jeon, S., & Woo, S. S. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512212 · Cited by 74
Self-citationCombating Online Misinformation Videos: Characterization, Detection, and Future Directions
Bu, Y., Sheng, Q., Cao, J., Qi, P., Wang, D., & Li, J. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612426 · Cited by 73
Cross-Modality and Within-Modality Regularization for Audio-Visual Deepfake Detection
Zou, H., Shen, M., Hu, Y., Chen, C., Chng, E. S., & Rajan, D. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447248 · Cited by 70
TIMIT-TTS: A Text-to-Speech Dataset for Multimodal Synthetic Media Detection
Salvi, D., Hosler, B., Bestagini, P., Stamm, M. C., & Tubaro, S. (2023) · IEEE Access · doi:10.1109/access.2023.3276480 · Cited by 65
Multimodaltrace: Deepfake Detection using Audiovisual Representation Learning
Anas Raza, M., & Mahmood Malik, K. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00106 · Cited by 63
Cai, Z., Ghosh, S., Dhall, A., Gedeon, T., Stefanov, K., & Hayat, M. (2023) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2023.103818 · Cited by 63
Audio–Visual Synchronization and Lip Movement Analysis for Real-Time Deepfake Detection
Javed, M., Zhang, Z., Dahri, F. H., Laghari, A. A., Krajčík, M., & Almadhor, A. S. (2025) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-025-00911-7 · Cited by 65
Kumar, A., Singh, D., Jain, R., Jain, D. K., Gan, C., & Zhao, X. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102993 · Cited by 59
Advancements in detecting Deepfakes: AI algorithms and future prospects − a review
Singh, L. H., Charanarur, P., & Chaudhary, N. K. (2025) · Discover Internet of Things · doi:10.1007/s43926-025-00154-0 · Cited by 58
Review of audio deepfake detection techniques: Issues and prospects
Dixit, A., Kaur, N., & Kingra, S. (2023) · Expert Systems · doi:10.1111/exsy.13322 · Cited by 56
Video Deepfake classification using particle swarm optimization-based evolving ensemble models
Zhang, L., Zhao, D., Lim, C. P., Asadi, H., Huang, H., Yu, Y., & Gao, R. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111461 · Cited by 48
MIS-AVoiDD: Modality Invariant and Specific Representation for Audio-Visual Deepfake Detection
Katamneni, V. S., & Rattani, A. (2023) · International Conference on Machine Learning and Applications · doi:10.1109/icmla58977.2023.00207 · Cited by 47
Hashmi, A., Shahzad, S. A., Lin, C., Tsao, Y., & Wang, H. (2025) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2025.3554477 · Cited by 48
Integrating Audio-Visual Features For Multimodal Deepfake Detection
Muppalla, S., Jia, S., & Lyu, S. (2023) · 2023 IEEE MIT Undergraduate Research Technology Conference (URTC) · doi:10.1109/urtc60662.2023.10534969 · Cited by 46
“Then we conduct a comparison with existing deepfake detection methods [11, 21] that were evaluated on the FakeAVCeleb dataset, including ensemble frame-based methods of MesoInception-4 [28], EfficientNet [29], and FTCN [30], and audio-visual multimodal methods including AVoiD-DF [14] and…”
BackgroundMethodInfluentialDeepfake Media Forensics: State of the Art and Challenges Ahead
Amerini, I., Barni, M., Battiato, S., Bestagini, P., Boato, G., Bonaventura, T. S., Bruni, V., Caldelli, R., Natale, F. G. B. D., Nicola, R. D., et al. (2025) · Lecture notes in social networks · doi:10.1007/978-3-031-85386-9_3 · Cited by 46
SoK: Systematization and Benchmarking of Deepfake Detectors in a Unified Framework
Le, B. M., Kim, J., Woo, S. S., Moore, K., Abuadbba, A., & Tariq, S. (2025) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp63326.2025.00055 · Cited by 43
Self-citationPolyGlotFake: A Novel Multilingual and Multimodal DeepFake Dataset
Hou, Y., Fu, H., Chen, C., Li, Z., Zhang, H., & Zhao, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78341-8_12 · Cited by 42
A Review of Deep Learning Techniques for Multimodal Fake News and Harmful Languages Detection
Ayetiran, E. F., & Özgöbek, Ö. (2024) · IEEE Access · doi:10.1109/access.2024.3406258 · Cited by 40
A Novel Smart Deepfake Video Detection System
Elpeltagy, M., Ismail, A., Zaki, M. S., & Eldahshan, K. (2023) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2023.0140144 · Cited by 39
Real-Time Deepfake Detection via Gaze and Blink Patterns: A Transformer Framework
Javed, M., Zhang, Z., Dahri, F. H., Laghari, A. A., Krajčík, M., & Almadhor, A. S. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.062954 · Cited by 36
Zhang, Y., Miao, C., Luo, M., Li, J., Deng, W., Yao, W., Li, Z., Hu, B., Feng, W., Gong, T., et al. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3688984 · Cited by 35
Lip Sync Matters: A Novel Multimodal Forgery Detector
Shahzad, S. A., Hashmi, A., Khan, S., Peng, Y.-T., Tsao, Y., & Wang, H.-M. (2022) · 2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.23919/apsipaasc55919.2022.9980296 · Cited by 33
“2) Comparing Audio-Visual Lip-Sync Model with other models: Next, we aimed to compare our model with various baseline unimodal, ensemble, and multimodal methods following [34], where the authors evaluated different models on the multimodal FakeAVCeleb dataset.”
MethodResultMultimodal Forgery Detection Using Ensemble Learning
Hashmi, A., Shahzad, S. A., Ahmad, W., Lin, C. W., Tsao, Y., & Wang, H.-M. (2022) · 2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.23919/apsipaasc55919.2022.9980255 · Cited by 32
“Ensemble (Soft-Voting) [16] VGG16 AV Real 0.”
BackgroundMethodResultInfluentialUsmani, S., Kumar, S., & Sadhya, D. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129256 · Cited by 32
Shahzad, S. A., Hashmi, A., Peng, Y., Tsao, Y., & Wang, H. (2025) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2025.3618409 · Cited by 32
Hearing and Seeing Abnormality: Self-Supervised Audio-Visual Mutual Learning for Deepfake Detection
Sung, C.-S., Chen, J.-C., & Chen, C. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095247 · Cited by 29
Audio–visual deepfake detection using articulatory representation learning
Wang, Y., & Huang, H. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.104133 · Cited by 28
Triple-modality interaction for deepfake detection on zero-shot identity
Yoon, J., Panizo-LLedot, Á., Camacho, D., & Choi, C. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102424 · Cited by 28
Audio Deep Fake Detection with Sonic Sleuth Model
Alshehri, A. S., Almalki, D., Alharbi, E., & Albaradei, S. (2024) · Computers · doi:10.3390/computers13100256 · Cited by 27
Identity-Driven Multimedia Forgery Detection via Reference Assistance
Xu, J., Chen, J., Song, X., Han, F., Shan, H., & Jiang, Y. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680622 · Cited by 25
Deepfake detection: critical review of state-of-the-art approaches and future perspectives
Soundarya, B. C., & Gururaj, H. L. (2026) · Discover Applied Sciences · doi:10.1007/s42452-025-08174-9 · Cited by 27
Audio-Visual Deepfake Detection With Local Temporal Inconsistencies
Astrid, M., Ghorbel, E., & Aouada, D. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889087 · Cited by 24
Deepfake CAPTCHA: A Method for Preventing Fake Calls
Yasur, L., Frankovits, G., Grabovski, F. M., & Mirsky, Y. (2023) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3579856.3595801 · Cited by 25
Hashmi, A., Shahzad, S. A., Lin, C., Tsao, Y., & Wang, H. (2026) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2026.3656145 · Cited by 23
A review of deep learning based multimodal forgery detection for video and audio
Tan, D., Yang, Y., Niu, C., Li, S., Yang, D., & Tan, B. (2025) · Discover Applied Sciences · doi:10.1007/s42452-025-07629-3 · Cited by 24
Dimitri, G. M. (2022) · Computers · doi:10.3390/computers11110163 · Cited by 22
Gura, D. A., Dong, B., Mehiar, D. A. F., & Said, N. A. (2024) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2024.048238 · Cited by 22
Continuous Learning of Transformer-based Audio Deepfake DetectionPreprint
Le, T. D. N., Teh, K. K., & Tran, H. D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.05924 · Cited by 22
Deepfake Detection in Video and Audio Clips: A Comprehensive Survey and Analysis
Jbara, W. A., Hussein, N. A.-H. K., & Soud, J. H. (2024) · Mesopotamian Journal of CyberSecurity · doi:10.58496/mjcs/2024/025 · Cited by 21
Towards Understanding of Deepfake Videos in the WildPreprint
Cho, B., Le, B. M., Kim, J. W., Woo, S. S., Tariq, S., Abuadbba, A., & Moore, K. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.01919 · Cited by 21
Self-citationDu, Y., Wang, Z., Luo, Y., Piao, C., Yan, Z., Li, H., & Yuan, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.15233 · Cited by 21
Contextual Cross-Modal Attention for Audio-Visual Deepfake Detection and Localization
Katamneni, V. S., & Rattani, A. (2024) · 2024 IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb62174.2024.10744480 · Cited by 20
Multi-modal Deepfake Detection via Multi-task Audio-Visual Prompt Learning
Miao, H., Guo, Y., Liu, Z., & Wang, Y. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i1.32042 · Cited by 21
Shahzad, S. A., Hashmi, A., Peng, Y., Tsao, Y., & Wang, H. (2025) · APSIPA Transactions on Signal and Information Processing · doi:10.1561/116.20250004 · Cited by 18
Deepfake Catcher: Can a Simple Fusion be Effective and Outperform Complex DNNs?
Agarwal, A., & Ratha, N. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00383 · Cited by 18
Pitch Imperfect: Detecting Audio Deepfakes Through Acoustic Prosodic AnalysisPreprint
Warren, K., Olszewski, D., Layton, S., Butler, K. R. B., Gates, C. E., & Traynor, P. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.14726 · Cited by 18
A defensive attention mechanism to detect deepfake content across multiple modalities
Asha, S., Vinod, P., & Menon, V. G. (2024) · Multimedia Systems · doi:10.1007/s00530-023-01248-x · Cited by 16
Audio-Visual Multimodal Deepfake Detection Leveraging Emotional Recognition
Alsaeedi, A., AlMansour, A. A., & Jamal, A. T. (2025) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2025.0160622 · Cited by 17
Tariq, S., Woo, S. S., Singh, P., Irmalasari, I., Gupta, S., & Gupta, D. (2025) · Proceedings of the 33rd ACM International Conference on Multimedia · doi:10.1145/3746027.3755786 · Cited by 15
“State-of-the-art detection systems based primarily on Convolu-tional Neural Networks (CNNs), vision transformers, and multimodal embeddings [27, 29–31, 34–36, 39, 41, 54, 73–76]—have achieved impressive classification accuracy on large-scale datasets such as FaceForensics++ [63], FakeAVCeleb [28]…”
Self-citation"Better Be Computer or I'm Dumb": A Large-Scale Evaluation of Humans as Audio Deepfake Detectors
Warren, K., Tucker, T., Crowder, A., Olszewski, D., Lu, A., Fedele, C., Pasternak, M., Layton, S., Butler, K., Gates, C., et al. (2024) · Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3658644.3670325 · Cited by 15
A Multi-Stream Fusion Approach with One-Class Learning for Audio-Visual Deepfake Detection
Lee, K., Zhang, Y., & Duan, Z. (2024) · IEEE International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp61759.2024.10743671 · Cited by 15
DeepFake detection in the AIGC era: A survey, benchmarks, and future perspectives
Xie, S., Qiao, T., Li, S., Zhang, X., Zhou, J., & Feng, G. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103740 · Cited by 14
Beyond the Screen: Evaluating Deepfake Detectors under Moiré Pattern Effects
Tariq, R., Heo, M., Woo, S. S., & Tariq, S. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00446 · Cited by 14
Self-citationFakeOut: Leveraging Out-of-domain Self-supervision for Multi-modal Video Deepfake DetectionPreprint
Knafo, G., & Fried, O. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.00773 · Cited by 14
Detecting Audio-Visual Deepfakes with Fine-Grained InconsistenciesPreprint
Astrid, M., Ghorbel, E., & Aouada, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.06753 · Cited by 14
X3A: Efficient Multimodal Deepfake Detection with Score-Level Fusion
Park, C. S., Moon, B., Jeon, M., Jung, J.-W., & Woo, S. S. (2025) · ACM Symposium on Applied Computing · doi:10.1145/3672608.3707934 · Cited by 13
Self-citationDeepfake Detection for Fake Images with Facemasks
Lee, S., Ko, D., Park, J., Shin, S., Hong, D., & Woo, S. S. (2022) · doi:10.1145/3494109.3527189 · Cited by 13
Self-citationSpecViT: A Custom Vision-Transformer based Approach for Audio Deepfake Detection
Modak, S., Das, A. K., & Naskar, R. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889022 · Cited by 12
NPVForensics: Learning VA correlations in non-critical phoneme–viseme regions for deepfake detection
Chen, Y., Yu, Y., Ni, R., Li, H., Wang, W., & Zhao, Y. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105461 · Cited by 12
Analysing the landscape of Deep Fake Detection: A Survey
Vyas, K., Pareek, P., Jayaswal, R., & Patil, S. (2024) · International Journal of Intelligent Systems and Applications in Engineering · Cited by 12
LLMs Are Not Yet Ready for Deepfake Image DetectionPreprint
Tariq, S., Nguyen, D. D., Chamikara, M. A. P., Wu, T., Abuadbba, A., & Moore, K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.10474 · Cited by 12
Self-citationD-Fence layer: an ensemble framework for comprehensive deepfake detection
Asha, S., Vinod, P., Amerini, I., & Menon, V. G. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-18130-1 · Cited by 11
Deepfake Detection for Facial Images with Facemasks
Ko, D., Lee, S., Park, J., Shin, S., Hong, D., & Woo, S. S. (2022) · arXiv · doi:10.48550/arxiv.2202.11359 · Cited by 10
Self-citationEnhancing Generalization Ability in Deepfake Detection via Continual Learning
Usmani, S., Kumar, S., & Sadhya, D. (2024) · Indian Conference on Computer Vision, Graphics and Image Processing · doi:10.1145/3702250.3702258 · Cited by 10
Statistics-Aware Audio-Visual Deepfake Detector
Astrid, M., Ghorbel, E., & Aouada, D. (2024) · Proceedings - International Conference on Image Processing · doi:10.1109/icip51287.2024.10648224 · Cited by 10
PTD: privacy-preserving human face processing framework using tensor decomposition
Kim, J., Tariq, S., & Woo, S. S. (2022) · ACM Symposium on Applied Computing · doi:10.1145/3477314.3507036 · Cited by 10
Self-citationKatamneni, V. S., Nadimpalli, A. V., & Rattani, A. (2024) · Springer eBooks · doi:10.1007/978-981-97-2059-0_8 · Cited by 9
Revisiting face forgery detection towards generalization
Peng, C., Chen, T., Liu, D., Guo, H., Wang, N., & Gao, X. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107310 · Cited by 9
Security Strengthen and Detection of Deepfake Videos and Images Using Deep Learning Techniques
Talreja, S., Bindle, A., Kumar, V., Budhiraja, I., & Bhattacharya, P. (2024) · IEEE International Conference on Communications workshops · doi:10.1109/iccworkshops59551.2024.10615811 · Cited by 9
A Face Pre-Processing Approach to Evade Deepfake Detector
Kim, T., Kim, J., Kim, J., & Woo, S. S. (2022) · doi:10.1145/3494109.3527190 · Cited by 8
Self-citationSeeing Through the Blur: Unlocking Defocus Maps for Deepfake Detection
Jeon, M., & Woo, S. S. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761260 · Cited by 7
Self-citationExtracting Efficient Spectrograms From MP3 Compressed Speech Signals for Synthetic Speech Detection
Xiang, Z., Yadav, A. K. S., Tubaro, S., Bestagini, P., & Delp, E. J. (2023) · Proceedings of the 2023 ACM Workshop on Information Hiding and Multimedia Security · doi:10.1145/3577163.3595104 · Cited by 6
“Typical methods use deep neural networks with 2D time-frequency audio features such as the spectrogram [3], MFCC [16], and CQT [44].”
MethodAttention-based Multimodal learning framework for Generalized Audio-Visual Deepfake Detection
Masood, M., Javed, A., & Irtaza, A. (2023) · Preprint (Research Square) · doi:10.21203/rs.3.rs-3415144/v1 · Cited by 6
Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges
Liu, P., Tao, Q., & Zhou, J. T. (2024) · arXiv · doi:10.48550/arxiv.2406.06965 · Cited by 6
Exploring the Impact of Moire Pattern on Deepfake Detectors
Tariq, R., Tariq, S., & Woo, S. S. (2024) · Proceedings - International Conference on Image Processing · doi:10.1109/icip51287.2024.10647902 · Cited by 6
Self-citationEnsemble-Based Biometric Verification: Defending Against Multi-Strategy Deepfake Image Generation
Zen, H., Wagh, R., Wanderley, M., Bicalho, G. C., Park, R. Y., Sun, M., Palacios, R., Carvalho, L. A. D. R., Rinaldo, G., & Gupta, A. D. (2025) · Computers · doi:10.3390/computers14060225 · Cited by 6
Ahmed, A. S., & Khaleel, A. M. (2024) · Journal of Robotics and Control (JRC) · doi:10.18196/jrc.v5i6.23502 · Cited by 7
Moorthy, T. V. K., Reddy, Y. N., Venkataiah, C., Kumar, N. N., Prasad, D. R., Rao, Y. M., & Jayamma, M. (2025) · Cognitive Computation · doi:10.1007/s12559-025-10465-7 · Cited by 7
Voice Cloning and Forgery Detection Using WaveGAN and SpecGAN
Ghadekar, P. P., Rajput, K., Dhabekar, H., Helge, P., Mundhra, H., & Rathi, C. (2023) · International Conference on Computing Communication Control and automation · doi:10.1109/iccubea58933.2023.10392082 · Cited by 5
Park, C. S., Muneer, M. S., & Woo, S. S. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3760853 · Cited by 5
Self-citationORVAE: One-Class Residual Variational Autoencoder for Voice Activity Detection in Noisy Environment
Khalid, H., Tariq, S., Kim, T., Ko, J. H., & Woo, S. S. (2022) · Neural Processing Letters · doi:10.1007/s11063-021-10695-4 · Cited by 5
Self-citationDeepFake Technology for Breast Cancer Dataset Generation Using Autoencoders and Deep Neural Networks
Anwar, S., Anwer, M. A., & Al-Nadawi, D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88220-3_1 · Cited by 4
Kiaei, S., Honarparvar, S., Saeedi, S., & Liang, S. H. L. (2021) · IEEE Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON) · doi:10.1109/iemcon53756.2021.9623141 · Cited by 4
Spectrogram-Based CNN Framework for Live Deepfake Voice Detection in Google Meet
Jyani, M., & Rathore, P. S. (2025) · International Conference on Computing Technologies & Data Communication (ICCTDC) · doi:10.1109/icctdc64446.2025.11158792 · Cited by 3
Deepfakes: A comprehensive survey on techniques, challenges and future directions
Usmani, S., Kumar, S., & Sadhya, D. (2026) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2026.111210 · Cited by 3
Toward comprehensive short utterances manipulations detection in videos
Moufidi, A., Rousseau, D., & Rasti, P. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-20284-x · Cited by 3
Metadata
- Authors
- Khalid, H., Kim, M., Tariq, S., & Woo, S. S.
- Affiliations
- Sungkyunkwan University (KR)
- Venue
- Proceedings of the 1st Workshop on Synthetic Multimedia - Audiovisual Deepfake Generation and Detection
- Publisher
- ACM
- Published
- 2021-10-20
- Pages
- 7-15
- OpenAlex
- W3197123936
- Cited by
- 110
- References
- 56
- Citations by year
- 2026: 16 · 2025: 34 · 2024: 27 · 2023: 19 · 2022: 13 · 2021: 1
- FWCI
- 4.90
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Research Foundation of Korea (NRF) grant funded by Korea government MSIT Basic Science Research Program (2020R1C1C1006004)
- IITP grant funded by the Korea government (MSIT) AI Graduate School Support Program (2019-0-00421)
- MSIT (Ministry of Science ICT) Korea under the High-Potential Individuals Global Training Program (2020-0-01550)
- IITP grant funded by the Korea government MSITOriginal Technology Development of Artificial Intelligence Industry (2021-0-00017)
- IITP grant funded by the Korea government (MSIT) Regional strategic industry convergence security core talent training business (2019-0-01343)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Signal Processing › Biometric Identification and Security
- Keywords
- deepfake detection, synthetic audio, voice cloning, face swapping
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.