Not made for each other- Audio-Visual Dissonance-based Deepfake Detection and Localization
Open accessAbstract
We propose detection of deepfake videos based on the dissimilarity between the audio and visual modalities, termed as the Modality Dissonance Score (MDS). We hypothesize that manipulation of either modality will lead to dis-harmony between the two modalities, e.g., loss of lip-sync, unnatural facial and lip movements, etc. MDS is computed as the mean aggregate of dissimilarity scores between audio and visual segments in a video. Discriminative features are learnt for the audio and visual channels in a chunk-wise manner, employing the cross-entropy loss for individual modalities, and a contrastive loss that models inter-modality similarity. Extensive experiments on the DFDC and DeepFake-TIMIT Datasets show that our approach outperforms the state-of-the-art by up to 7%. We also demonstrate temporal forgery localization, and show how our technique identifies the manipulated video segments.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The multimodal MDS method is not designed for temporal forgery localization tasks.
“The multimodal MDS method is not designed for temporal forgery localization tasks”
From Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization (2022) · cites this paper as [28]
The Introduction identifies temporal forgery localization as a key contribution and says chunk-wise learning enables localization of transient video forgeries. Section 4.6 demonstrates the MDS method assigning segment-wise scores to locate forged portions of videos, contrary to the claim. (LLM, conf=0.92)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredUsing only spatio-temporal audio and visual information is insufficient for the task of multimodal forgery detection.
“Chung et al. [18] proposed an audio-visual dissonance-based model for deepfake video detection. However, the approach showed an accuracy of 69% on the FakeAVCeleb dataset [75], demonstrating that using only spatio-temporal audio and visual information is insufficient for the task of multimodal forgery detection.”
From Attention-based Multimodal learning framework for Generalized Audio-Visual Deepfake Detection (2023) · cites this paper as [18]
The Introduction identifies the cited work as an audio-visual dissonance-based deepfake detector. But Table 2 reports its performance on DFDC and DFTIMIT, not 69% accuracy on FakeAVCeleb. The Conclusion reports state-of-the-art DFDC performance; it does not conclude that spatio-temporal audio and visual information is insufficient for multimodal forgery detection. (LLM, conf=0.88)
Experimental — accuracy not yet measuredChung et al. [18] proposed an audio-visual dissonance-based model for deepfake video detection, and the approach showed an accuracy of 69% on the FakeAVCeleb dataset.
“Chung et al. [18] proposed an audio-visual dissonance-based model for deepfake video detection. However, the approach showed an accuracy of 69% on the FakeAVCeleb dataset [75], demonstrating that using only spatio-temporal audio and visual information is insufficient for the task of multimodal forgery detection.”
From Attention-based Multimodal learning framework for Generalized Audio-Visual Deepfake Detection (2023) · cites this paper as [18]
The Introduction describes an audio-visual dissonance model for deepfake detection. Section 4.1 says the experiments used DeepFake-TIMIT and DFDC, and the results report AUC rather than 69% accuracy on FakeAVCeleb. The cited paper is by Chugh et al., not Chung et al.; it does not establish the sentence’s conclusion about multimodal detection on FakeAVCeleb. (LLM, conf=0.90)
Experimental — accuracy not yet measuredAll previous deepfake datasets provide face manipulations that occur in most of the frames of the video.
“All previous datasets provide face manipulations that occur in most of the frames of the video [28].”
From Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization (2022) · cites this paper as [28]
Section 4.6, “Temporal Forgery Localization” (page not available), says that most datasets, including those used in the paper, have manipulation across practically the entire video. It does not say this is true of all previous datasets or that every such manipulation is facial; Section 4.1 notes that the DFDC dataset can contain audio manipulation. (LLM, conf=0.88)
Experimental — accuracy not yet measuredOn top of the contrastive loss, unimodal cross-entropy losses provide additional benefits to improve fake detection performance.
“and (b) on top of the contrastive loss, the unimodal cross-entropy losses provide additional benefits to improve fake detection performance.”
From Deepfake detection using deep learning methods: A systematic and comprehensive review (2024) · cites this paper as Chugh et al. (2020)
Simulations indicate that on the DFDC dataset, the MDS-based fake detection framework can attain state-of-the-art performance.
“Simulations indicate that (a) on the DFDC dataset, the MDS-based fake detection framework can attain state-of-the-art performance”
From Deepfake detection using deep learning methods: A systematic and comprehensive review (2024) · cites this paper as Chugh et al. (2020)
The suggested approach's explainability and interpretability are proven using audio-visual distance distributions for actual and fraudulent films, Grad-CAM results representing MDS network attention regions, and forgery localization results.
“The suggested approach's explainability and interpretability are proven using audio-visual distance distributions for actual and fraudulent films, Grad-CAM results representing MDS network attention regions, and forgery localization results.”
From Deepfake detection using deep learning methods: A systematic and comprehensive review (2024) · cites this paper as Chugh et al. (2020)
Existing deepfake generation algorithms mainly focus on face regions, but maintaining voice-facial consistency is difficult.
“Existing deepfake generation algorithms mainly focus on the face regions, but maintaining voice-facial consistency is difficult [18].”
From Attention-based Multimodal learning framework for Generalized Audio-Visual Deepfake Detection (2023) · cites this paper as [18]
Chugh et al. proposed an approach based on the modality dissonance score (MDS), which classifies forgery in DeepFake video between audio and visual modalities through their dissimilarities.
“Chugh et al. [23] proposed an approach based on the modality dissonance score (MDS), which classifies forgery in DeepFake video between audio and visual modalities through their dissimilarities.”
From A Comprehensive Review of DeepFake Detection Using Advanced Machine Learning and Fusion Methods (2023) · cites this paper as [23]
Fake content might constitute only a small part of an otherwise long real video.
“However, fake content might constitute only a small part of an otherwise long real video, as was initially suggested in [28].”
From Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization (2022) · cites this paper as [28]
MDS is the only exception shown to work for localization tasks, but it is designed primarily for classification.
“The only exception is the MDS, shown to work for localization tasks, however, the method is designed primarily for classification.”
From Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization (2022) · cites this paper as [28]
The multimodal MDS method predicts only fixed length segments and cannot predict precise boundaries.
“and predicts only fixed length segments (i.e. cannot predict the precise boundaries)”
From Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization (2022) · cites this paper as [28]
Contrastive loss has been shown to be a powerful objective for similar tasks.
“contrastive loss has been shown [66], [28] to be a powerful objective for similar tasks.”
From Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization (2022) · cites this paper as [28]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513 · see citing works
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,268 · see citing works
Extracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436 · see citing works
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · see citing works
MesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872 · see citing works
Face2Face: Real-Time Face Capture and Reenactment of RGB Videos
Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., & Niessner, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.262 · Cited by 1,747 · see citing works
Deepfake Video Detection Using Recurrent Neural Networks
Guera, D., & Delp, E. J. (2018) · 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2018.8639163 · Cited by 1,285 · see citing works
Exposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196 · see citing works
In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking
Li, Y., Chang, M.-C., & Lyu, S. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630787 · Cited by 1,089 · see citing works
Capsule-forensics: Using Capsule Networks to Detect Forged Images and Videos
Nguyen, H. H., Yamagishi, J., & Echizen, I. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682602 · Cited by 951 · see citing works
Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations
Matern, F., Riess, C., & Stamminger, M. (2019) · 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw.2019.00020 · Cited by 824 · see citing works
Exposing DeepFake Videos By Detecting Face Warping Artifacts
Li, Y., & Lyu, S. (2018) · arXiv · doi:10.48550/arxiv.1811.00656 · Cited by 785 · see citing works
Out of Time: Automated Lip Sync in the Wild
Chung, J. S., & Zisserman, A. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54427-4_19 · Cited by 689 · see citing works
S^3FD: Single Shot Scale-Invariant Face Detector
Zhang, S., Zhu, X., Lei, Z., Shi, H., Wang, X., & Li, S. Z. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.30 · Cited by 662 · see citing works
Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos
Nguyen, H. H., Fang, F., Yamagishi, J., & Echizen, I. (2019) · arXiv · doi:10.48550/arxiv.1906.06876 · Cited by 551 · see citing works
Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos
Nguyen, H. H., Fang, F., Yamagishi, J., & Echizen, I. (2019) · 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS) · doi:10.1109/btas46853.2019.9185974 · Cited by 549 · see citing works
DeepFakes: a New Threat to Face Recognition? Assessment and Detection
Korshunov, P., & Marcel, S. (2018) · arXiv · doi:10.48550/arxiv.1812.08685 · Cited by 497 · see citing works
(2019) · Cited by 415 · see citing works
The Deepfake Detection Challenge (DFDC) Preview Dataset
Dolhansky, B., Howes, R., Pflaum, B., Baram, N., & Ferrer, C. C. (2019) · arXiv · doi:10.48550/arxiv.1910.08854 · Cited by 413 · see citing works
BusterNet: Detecting Copy-Move Image Forgery with Source/Target Localization
Wu, Y., AbdAlmageed, W., & Natarajan, P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01231-1_11 · Cited by 356 · see citing works
Emotions Don't Lie: An Audio-Visual Deepfake Detection Method using Affective Cues
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., & Manocha, D. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413570 · Cited by 319 · see citing works
Multi-Region Probabilistic Histograms for Robust and Scalable Identity Inference
Sanderson, C., & Lovell, B. C. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-01793-3_21 · Cited by 303 · see citing works
Artificial Intelligence, Deepfakes and a Future of Ectypes
Floridi, L. (2018) · Philosophy & Technology · doi:10.1007/s13347-018-0325-3 · Cited by 187 · see citing works
Martinez, J., Perez-Meana, H. M., Escamilla, E., et al. (2012) · doi:10.1109/conielecomp.2012.6189918 · Cited by 132 · see citing works
Mild Auditory-Visual Dissonance in Television News May Exceed Viewer Attentional Capacity
Grimes, T. (1991) · Human Communication Research · doi:10.1111/j.1468-2958.1991.tb00546.x · Cited by 117 · see citing works
Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?Preprint
Hara, K., Kataoka, H., & Satoh, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.09577 · Cited by 114 · see citing works
Whose dystopia is it anyway? Deepfakes and social media regulation
Yadlin‐Segal, A., & Oppenheim, Y. (2020) · Convergence The International Journal of Research into New Media Technologies · doi:10.1177/1354856520923963 · Cited by 96 · see citing works
Classification and weakly supervised pain localization using multiple segment representation
Sikka, K., Dhall, A., & Bartlett, M. S. (2014) · Image and Vision Computing · doi:10.1016/j.imavis.2014.02.008 · Cited by 58 · see citing works
Self-citationTwo-Stream Neural Networks for Tampered Face DetectionPreprint
Zhou, P., Han, X., Morariu, V. I., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.11276 · Cited by 57 · see citing works
Automatic Speech Recognition: An Auditory Perspective
Mogran, N., Bourlard, H. A., & Heřmanský, H. (2006) · Springer eBooks · doi:10.1007/0-387-21575-1_6 · Cited by 47 · see citing works
Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective CuesPreprint
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., & Manocha, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.06711 · Cited by 33 · see citing works
Subramanian, R., Katti, H., Huang, R. Z., Chua, T., & Kankanhalli, M. (2009) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1631272.1631399 · Cited by 28 · see citing works
Self-citationGAZED Gaze-guided Cinematic Editing of Wide-Angle Monocular Video Recordings
Moorthy, K. L. B., Kumar, M., Subramanian, R., et al. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376544 · Cited by 26 · see citing works
Self-citationEEG-based Evaluation of Cognitive Workload Induced by Acoustic Parameters for Data Sonification
Bilalpur, M., Kankanhalli, M., WINKLER, S. S., et al. (2018) · ACM International Conference on Multimodal Interaction (ICMI) · doi:10.1145/3242969.3243016 · Cited by 23 · see citing works
Emotions Don't Lie: A Deepfake Detection Method using Audio-Visual Affective Cues
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., & Manocha, D. (2020) · Cited by 15 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [7]7. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger (Eds.). Curran Associates, Inc., 2672-2680.
- [29]29. Shifeng Zhang, Xiangyu Zhu, Zhen Lei, Hailin Shi, Xiaobo Wang, and Stan Z Li. 2017. S3fd: Single shot scale-invariant face detector. In Proceedings of the IEEE International Conference on Computer Vision. 192-201.
Cited by
Sorted by citations
Deepfake Detection: A Systematic Literature Review
Rana, M. S., Nobi, M. N., Murali, B., & Sung, A. H. (2022) · IEEE Access · doi:10.1109/access.2022.3154404 · Cited by 573 · see citing works
Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection
Haliassos, A., Vougioukas, K., Petridis, S., & Pantic, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00500 · Cited by 537 · see citing works
Masood, M., Nawaz, M., Malik, K. M., Javed, A., Irtaza, A., & Malik, H. (2023) · Applied Intelligence · doi:10.1007/s10489-022-03766-z · Cited by 543 · see citing works
Deepfake detection using deep learning methods: A systematic and comprehensive review
Heidari, A., Jafari Navimipour, N., Dag, H., & Unal, M. (2024) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1520 · Cited by 349 · see citing works
“Chugh et al. (2020) introduced a bimodal deepfake detecting model based on the Modality Dissonance Score (MDS), it measures the similarity between audio and visual streams for real and fake videos, allowing them to be distinguished.”
MethodCountering Malicious DeepFakes: Survey, Battleground, and Horizon
Juefei-Xu, F., Wang, R., Huang, Y., Guo, Q., Ma, L., & Liu, Y. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01606-8 · Cited by 191 · see citing works
“Then, some studies are working on how to measure the similarity between visual and audio (Mittal et al., 2020; Chugh et al., 2020) and further explore strong visual-audio inconsistency signals for DeepFake detection (Agarwal et al., 2020b).”
BackgroundAVoiD-DF: Audio-Visual Joint Learning for Detecting Deepfake
Yang, W., Zhou, X., Chen, Z., Guo, B., Ba, Z., Xia, Z., Cao, X., & Ren, K. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3262148 · Cited by 198 · see citing works
Ilyas, H., Javed, A., & Malik, K. M. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110124 · Cited by 186 · see citing works
Deepfake generation and detection: Case study and challenges
Patel, Y., Tanwar, S., Gupta, R., Bhattacharya, P., Davidson, I. E., Nyameko, R., Aluvala, S., & Vimal, V. (2023) · IEEE Access · doi:10.1109/access.2023.3342107 · Cited by 186 · see citing works
Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection
Haliassos, A., Mira, R., Petridis, S., & Pantic, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01453 · Cited by 180 · see citing works
Zhou, T., Wang, W., Liang, Z., & Shen, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00572 · Cited by 156 · see citing works
The effect of deep learning methods on deepfake audio detection for digital investigation
Mcuba, M., Singh, A., Ikuesan, R. A., & Venter, H. (2023) · Procedia Computer Science · doi:10.1016/j.procs.2023.01.283 · Cited by 117 · see citing works
Deepfakes generation and detection: a short survey
Akhtar, Z. (2023) · Journal of Imaging · doi:10.3390/jimaging9010018 · Cited by 106 · see citing works
Self-Supervised Video Forensics by Audio-Visual Anomaly Detection
Feng, C., Chen, Z., & Owens, A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01011 · Cited by 101 · see citing works
Audio-Visual Person-of-Interest DeepFake Detection
Cozzolino, D., Pianese, A., Nießner, M., & Verdoliva, L. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00101 · Cited by 104 · see citing works
A Comprehensive Review of DeepFake Detection Using Advanced Machine Learning and Fusion Methods
Gupta, G., Raja, K., Gupta, M., Jan, T., Whiteside, S. T., & Prasad, M. (2023) · Electronics · doi:10.3390/electronics13010095 · Cited by 99 · see citing works
Cai, Z., Stefanov, K., Dhall, A., & Hayat, M. (2022) · 2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta56598.2022.10034605 · Cited by 99 · see citing works
“We have performed extensive benchmarking of the proposed dataset via several state-of-the-art methods including, BMN [52], AGT [45], AVFusion [46], and MDS [28].”
BackgroundMethodInfluentialSelf-citationDeepfakes and scientific knowledge dissemination
Doss, C., Mondschein, J., Shu, D., Wolfson, T., Kopecky, D., Fitton-Kane, V. A., Bush, L., & Tucker, C. (2023) · Scientific Reports · doi:10.1038/s41598-023-39944-3 · Cited by 90 · see citing works
Multimodaltrace: Deepfake Detection using Audiovisual Representation Learning
Anas Raza, M., & Mahmood Malik, K. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00106 · Cited by 63 · see citing works
TIMIT-TTS: A Text-to-Speech Dataset for Multimodal Synthetic Media Detection
Salvi, D., Hosler, B., Bestagini, P., Stamm, M. C., & Tubaro, S. (2023) · IEEE Access · doi:10.1109/access.2023.3276480 · Cited by 65 · see citing works
PVASS-MDD: Predictive Visual-Audio Alignment Self-Supervision for Multimodal Deepfake Detection
Yu, Y., Liu, X., Ni, R., Yang, S., Zhao, Y., & Kot, A. C. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3309899 · Cited by 63 · see citing works
Depth map guided triplet network for deepfake face detection
Liang, B., Wang, Z., Huang, B., Zou, Q., Wang, Q., & Liang, J. (2023) · Neural Networks · doi:10.1016/j.neunet.2022.11.031 · Cited by 54 · see citing works
Exploring Disentangled Content Information for Face Forgery Detection
Liang, J., Shi, H., & Deng, W. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19781-9_8 · Cited by 58 · see citing works
AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset
Cai, Z., Ghosh, S., Adatia, A. P., Hayat, M., Dhall, A., Gedeon, T., & Stefanov, K. (2024) · Proceedings of the 32nd ACM International Conference on Multimedia · doi:10.1145/3664647.3680795 · Cited by 59 · see citing works
“To investigate the impact of the level of label access, we designed 3 different label access levels for training: frame-level labels, segment-level labels only, and video-level labels only.”
MethodSelf-citationJoint Audio-Visual Attention with Contrastive Learning for More General Deepfake Detection
Zhang, Y., Lin, W., & Xu, J. (2024) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/3625100 · Cited by 40 · see citing works
Combining Automatic Speaker Verification and Prosody Analysis for Synthetic Speech Detection
Attorresi, L., Salvi, D., Borrelli, C., Bestagini, P., & Tubaro, S. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-37742-6_21 · Cited by 30 · see citing works
Audio–visual deepfake detection using articulatory representation learning
Wang, Y., & Huang, H. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.104133 · Cited by 28 · see citing works
Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion
Agarwal, S., Hu, L., Ng, E., Darrell, T., Li, H., & Rohrbach, A. (2023) · 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv56688.2023.00469 · Cited by 28 · see citing works
Singh, S., & Dhumane, A. (2025) · MethodsX · doi:10.1016/j.mex.2025.103632 · Cited by 24 · see citing works
Fine-Grained Multimodal DeepFake Classification via Heterogeneous Graphs
Yin, Q., Lu, W., Cao, X., Luo, X., Zhou, Y., & Huang, J. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02128-1 · Cited by 23 · see citing works
Contextual Cross-Modal Attention for Audio-Visual Deepfake Detection and Localization
Katamneni, V. S., & Rattani, A. (2024) · 2024 IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb62174.2024.10744480 · Cited by 20 · see citing works
Attention-based Multimodal learning framework for Generalized Audio-Visual Deepfake Detection
Masood, M., Javed, A., & Irtaza, A. (2023) · Preprint (Research Square) · doi:10.21203/rs.3.rs-3415144/v1 · Cited by 6 · see citing works
Mining Generalized Multi-timescale Inconsistency for Detecting Deepfake Videos
Yu, Y., Ni, R., Yang, S., Ni, Y., Zhao, Y., & Kot, A. C. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-024-02249-7 · Cited by 7 · see citing works
LayLens: Improving Deepfake Understanding through Simplified Explanations
Narang, A., Gupta, P., Su, L., & Dhall, A. (2025) · Proceedings of the 27th International Conference on Multimodal Interaction · doi:10.1145/3716553.3757093 · Cited by 1
Self-citationAnshul, A., Gopal, S., Rajan, D., & Chng, E. S. (2025) · 2025 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01283 · Cited by 1 · see citing works
- Authors
- Chugh, K., Gupta, P., Dhall, A., & Subramanian, R.
- Affiliations
- Indian Institute of Technology Ropar (IN)
- Venue
- Proceedings of the 28th ACM International Conference on Multimedia
- Publisher
- ACM
- Published
- 2020-10-12
- Pages
- 439-447
- OpenAlex
- W3031369641
- Cited by
- 225
- References
- 36
- Citations by year
- 2026: 29 · 2025: 61 · 2024: 44 · 2023: 56 · 2022: 23 · 2021: 11 · 2020: 1
- FWCI
- 7.02
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM International Conference on Multimedia (ACM MM) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- deepfake detection, lip synchronization, cross-modal contrastive learning, video manipulation detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
