Self-Supervised Video Forensics by Audio-Visual Anomaly Detection
Open accessAbstract
Manipulated videos often contain subtle inconsistencies between their visual and audio signals. We propose a video forensics method, based on anomaly detection, that can identify these inconsistencies, and that can be trained solely using real, unlabeled data. We train an autoregressive model to generate sequences of audio-visual features, using feature sets that capture the temporal synchronization between video frames and sound. At test time, we then flag videos that the model assigns low probability. Despite being trained entirely on real videos, our model obtains strong performance on the task of detecting manipulated speech videos. Project site: https://cfeng16.github.io/audio-visual-forensics.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The performance of any deepfake detection method is highly dependent on the quantitative and qualitative aspects of the datasets used for development.
“The performance of any deepfake detection method is highly dependent on the quantitative and qualitative aspects of the datasets used for development [18, 21, 25, 40, 44, 47, 49, 51, 52, 61, 63, 69, 72, 75].”
From AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset (2024) · cites this paper as [18]
The paper’s §4.7 ablation study reports no significant performance change when its model is trained on different combinations of LRS2 and LRS3. Section 4.4 likewise says its performance is not tied to a single training set. These findings run counter to the claim that the performance of any deepfake detector is highly dependent on the datasets used for development; the paper does not establish that broader claim about dataset quantity or quality. (LLM, conf=0.86)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredExisting surveys do not examine how detection methods withstand adversarial attacks designed to evade detection or postprocessing operations that degrade detection performance.
“Existing surveys do not examine how detection methods withstand adversarial attacks designed to evade detection [17, 21] or postprocessing operations that degrade detection performance [22-24].”
From Deepfake detection across image, video, and audio: a comprehensive survey with empirical evaluation of generalization and robustness (2026) · cites this paper as [22]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 6× · see citing works
“To detect anomalies, we fit an autoregressive generative model [84, 99] to sequences of synchronization features extracted from real videos, and identify low probability examples.”
MethodInfluentialAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
(2014) · Cited by 33,419 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Labaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,100 · see citing works
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242 · see citing works
SGDR: Stochastic Gradient Descent with Warm Restarts
Loshchilov, I., & Hutter, F. (2016) · arXiv · doi:10.48550/arxiv.1608.03983 · Cited by 5,309 · see citing works
The generalized correlation method for estimation of time delay
Knapp, C., & Carter, G. (1976) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1976.1162830 · Cited by 4,433 · see citing works
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.07733 · Cited by 3,846 · cited 8× · see citing works
“U ns up er vi se d AVBYOL [38, 40] AV LRW [22] 50.”
BackgroundMethodInfluentialThe Kinetics Human Action Video DatasetPreprint
Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., Natsev, P., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.06950 · Cited by 3,711 · see citing works
A Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,451 · see citing works
Return of the Devil in the Details: Delving Deep into Convolutional Nets
Chatfield, K., Simonyan, K., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.6 · Cited by 3,335 · see citing works
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · see citing works
Pixel Recurrent Neural NetworksPreprint
van den Oord, A., Kalchbrenner, N., & Kavukcuoglu, K. (2016) · International Conference on Machine Learning · doi:10.48550/arxiv.1601.06759 · Cited by 3,111 · see citing works
Conditional Image Generation with PixelCNN DecodersPreprint
van den Oord, A., Kalchbrenner, N., Vinyals, O., Espeholt, L., Graves, A., & Kavukcuoglu, K. (2016) · Neural Information Processing Systems · doi:10.48550/arxiv.1606.05328 · Cited by 3,066 · see citing works
Neural Discrete Representation Learning
Oord, A. V. D., Vinyals, O., & Kavukcuoglu, K. (2017) · arXiv · doi:10.48550/arxiv.1711.00937 · Cited by 2,729 · cited 1× · see citing works
“We first consider a simple model that uses discrete time delay as our features, following the success of autoregressive models for fitting discrete data [32, 86, 97].”
MethodUnsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery
Schlegl, T., Seeböck, P., Waldstein, S. M., Schmidt-Erfurth, U., & Langs, G. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-59050-9_12 · Cited by 2,627 · see citing works
Taming Transformers for High-Resolution Image Synthesis
Esser, P., Rombach, R., & Ommer, B. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01268 · Cited by 2,514 · see citing works
Hierarchical Text-Conditional Image Generation with CLIP Latents
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06125 · Cited by 2,511 · cited 1× · see citing works
“To provide a straightforward comparison with the time delay distribution model, we reduce the dimensionality of the features by projecting them onto the top 2τ + 1 principal components, following other work in autoregressive models of features [85].”
MethodCan Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?
Hara, K., Kataoka, H., & Satoh, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00685 · Cited by 2,300 · see citing works
VoxCeleb2: Deep Speaker Recognition
Chung, J. S., Nagrani, A., & Zisserman, A. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-1929 · Cited by 2,256 · see citing works
The kinetics human action video dataset
Kay, W. (2024) · TIB Data Manager · doi:10.57702/jtuxnt5m · Cited by 1,646 · see citing works
Bulat, A., & Tzimiropoulos, G. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.116 · Cited by 1,464 · cited 1× · see citing works
“We extract faces by using face detection [107] and alignment [15].”
MethodFace X-Ray for More General Face Forgery Detection
Li, L., Bao, J., Zhang, T., Yang, H., Chen, D., Wen, F., & Guo, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00505 · Cited by 1,211 · see citing works
CNN-Generated Images Are Surprisingly Easy to Spot… for Now
Wang, S.-Y., Wang, O., Zhang, R., Owens, A., & Efros, A. A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00872 · Cited by 1,210 · see citing works
Self-citationEnhancing The Reliability of Out-of-distribution Image Detection in Neural NetworksPreprint
Liang, S., Li, Y., & Srikant, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02690 · Cited by 1,117 · see citing works
Thinking in Frequency: Face Forgery Detection by Mining Frequency-Aware Clues
Qian, Y., Yin, G., Sheng, L., Chen, Z., & Shao, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58610-2_6 · Cited by 1,058 · see citing works
Self-citationDeep Anomaly Detection with Outlier Exposure
Hendrycks, D., Mazeika, M., & Dietterich, T. (2018) · arXiv · doi:10.48550/arxiv.1812.04606 · Cited by 932 · cited 1× · see citing works
“Other work addresses goals similar to anomaly detection by creating methods that model uncertainty [64] or that leverage outlier exposure [30, 44, 88].”
BackgroundA Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild
Prajwal, K. R., Mukhopadhyay, R., Namboodiri, V. P., & Jawahar, C. V. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413532 · Cited by 897 · see citing works
Deep Audio-Visual Speech Recognition
Afouras, T., Chung, J. S., Senior, A., Vinyals, O., & Zisserman, A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2889052 · Cited by 801 · cited 5× · see citing works
“Second, we use a generative model VQGAN [32], trained on LRS2 [3], for anomaly detection.”
MethodInfluentialExposing DeepFake Videos By Detecting Face Warping Artifacts
Li, Y., & Lyu, S. (2018) · arXiv · doi:10.48550/arxiv.1811.00656 · Cited by 785 · see citing works
Adversarially Learned One-Class Classifier for Novelty Detection
Sabokrou, M., Khalooei, M., Fathy, M., & Adeli, E. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00356 · Cited by 765 · see citing works
Generating diverse high-fidelity images with vq-vae-2
Razavi, A., Oord, A. V. D., & Vinyals, O. (2019) · arXiv · doi:10.48550/arxiv.1906.00446 · Cited by 709 · see citing works
FSGAN: Subject Agnostic Face Swapping and Reenactment
Nirkin, Y., Keller, Y., & Hassner, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00728 · Cited by 706 · see citing works
Audio-Visual Scene Analysis with Self-Supervised Multisensory Features
Owens, A., & Efros, A. A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_39 · Cited by 704 · see citing works
Self-citationWu, Y., AbdAlmageed, W., & Natarajan, P. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00977 · Cited by 700 · see citing works
Out of Time: Automated Lip Sync in the Wild
Chung, J. S., & Zisserman, A. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54427-4_19 · Cited by 689 · cited 4× · see citing works
“Unfortunately, videos in the wild are often “naturally” misaligned due to errors in encoding or recording, such as by having a single, consistent shift by a few frames [2, 23].”
BackgroundMethodInfluentialReturn of the Devil in the Details: Delving Deep into Convolutional NetsPreprint
Chatfield, K., Simonyan, K., Vedaldi, A., et al. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.3531 · Cited by 663 · see citing works
S^3FD: Single Shot Scale-Invariant Face Detector
Zhang, S., Zhu, X., Lei, Z., Shi, H., Wang, X., & Li, S. Z. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.30 · Cited by 662 · see citing works
DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection
Jiang, L., Li, R., Wu, W., Qian, C., & Loy, C. C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00296 · Cited by 599 · see citing works
First Order Motion Model for Image Animation
Siarohin, A., Lathuilière, S., Tulyakov, S., Ricci, E., & Sebe, N. (2020) · arXiv · doi:10.48550/arxiv.2003.00196 · Cited by 585 · see citing works
Salimans, T., Karpathy, A., Chen, X., & Kingma, D. P. (2017) · arXiv · doi:10.48550/arxiv.1701.05517 · Cited by 584 · see citing works
Generating Wikipedia by Summarizing Long SequencesPreprint
Liu, P. J., Saleh, M., Pot, E., Goodrich, B., Sepassi, R., Kaiser, Ł., & Shazeer, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.10198 · Cited by 578 · see citing works
PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
Song, Y., Kim, T., Nowozin, S., Ermon, S., & Kushman, N. (2017) · arXiv · doi:10.48550/arxiv.1710.10766 · Cited by 562 · cited 1× · see citing works
“There have been a variety of methods proposed for learning this distribution, such as GAN discriminators [37,56,66,81,89,91,104,113], flow-based models [106], and autoregressive models [93].”
MethodLips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection
Haliassos, A., Vougioukas, K., Petridis, S., & Pantic, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00500 · Cited by 537 · see citing works
Chung, J. S., & Zisserman, A. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54184-6_6 · Cited by 537 · cited 2× · see citing works
“U ns up er vi se d AVBYOL [38, 40] AV LRW [22] 50.”
BackgroundTransfer learning from speaker verification to multispeaker text-to-speech synthesis
Jia, Y., Zhang, Y., Weiss, R. J., Wang, Q., Shen, J., Ren, F., Chen, Z., Nguyen, P., Pang, R., Moreno, I. L., et al. (2018) · arXiv · doi:10.48550/arxiv.1806.04558 · Cited by 519 · see citing works
Self-citationFast Face-Swap Using Convolutional Neural Networks
Korshunova, I., Shi, W., Dambre, J., & Theis, L. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.397 · Cited by 501 · see citing works
Noiseprint: A CNN-Based Camera Model Fingerprint
Cozzolino, D., & Verdoliva, L. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2019.2916364 · Cited by 499 · see citing works
Fighting Fake News: Image Splice Detection via Learned Self-Consistency
Huh, M., Liu, A., Owens, A., & Efros, A. A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_7 · Cited by 465 · see citing works
Self-citationImage Forgery Localization via Block-Grained Analysis of JPEG Artifacts
Bianchi, T., & Piva, A. (2012) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2012.2187516 · Cited by 458 · see citing works
Adversarially Learned Anomaly Detection
Zenati, H., Romain, M., Foo, C.-S., Lecouat, B., & Chandrasekhar, V. (2018) · Proceedings · doi:10.1109/icdm.2018.00088 · Cited by 450 · see citing works
Arandjelović, R., & Zisserman, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_27 · Cited by 448 · see citing works
Global Texture Enhancement for Fake Face Detection in the Wild
Liu, Z., Qi, X., & Torr, P. H. S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00808 · Cited by 401 · see citing works
Leveraging Frequency Analysis for Deep Fake Image Recognition
Frank, J., Eisenhofer, T., Schönherr, L., Fischer, A., Kolossa, D., & Holz, T. (2020) · arXiv · doi:10.48550/arxiv.2003.08685 · Cited by 390 · cited 1× · see citing works
“Other work has proposed to use deep learning to inspect specific artifacts, such as blending [62], frequency domain [31, 35, 83], or texture [67].”
BackgroundDeep Semi-Supervised Anomaly Detection
Ruff, L., Vandermeulen, R. A., Görnitz, N., Binder, A., Müller, E., Müller, K.-R., & Kloft, M. (2019) · arXiv · doi:10.48550/arxiv.1906.02694 · Cited by 388 · cited 1× · see citing works
“Other work addresses goals similar to anomaly detection by creating methods that model uncertainty [64] or that leverage outlier exposure [30, 44, 88].”
BackgroundLearning Self-Consistency for Deepfake Detection
Zhao, T., Xu, X., Xu, M., Ding, H., Xiong, Y., & Xia, W. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01475 · Cited by 383 · see citing works
Scaling Autoregressive Models for Content-Rich Text-to-Image Generation
Yu, J., Xu, Y., Koh, J. Y., Luong, T., Baid, G., Wang, Z., Vasudevan, V., Ku, A., Yang, Y., Ayan, B. K., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.10789 · Cited by 371 · see citing works
What Makes Fake Images Detectable? Understanding Properties that Generalize
Chai, L., Bau, D., Lim, S.-N., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58574-7_7 · Cited by 357 · see citing works
Generative Adversarial Active Learning for Unsupervised Outlier Detection
Liu, Y., Li, Z., Zhou, C., Jiang, Y., Sun, J., Wang, M., & He, X. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2905606 · Cited by 338 · see citing works
Cooperative Learning of Audio and Video Models from Self-Supervised SynchronizationPreprint
Korbar, B., Tran, D., & Torresani, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.00230 · Cited by 331 · cited 1× · see citing works
“Other work studies temporal synchronization between audio and visual signals to learn audio-visual features [24, 60, 79], which can be used for active speaker detection [8, 59, 96], source separation [37, 71, 112], lip reading [5, 70, 73] and so on.”
BackgroundExploring Temporal Coherence for More General Video Face Forgery Detection
Zheng, Y., Bao, J., Chen, D., Zeng, M., & Wen, F. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01477 · Cited by 320 · see citing works
Emotions Don't Lie: An Audio-Visual Deepfake Detection Method using Affective Cues
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., & Manocha, D. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413570 · Cited by 319 · see citing works
Lipreading Using Temporal Convolutional Networks
Martínez, B., Ma, P., Petridis, S., & Pantić, M. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053841 · Cited by 293 · see citing works
LRS3-TED: a large-scale dataset for visual speech recognitionPreprint
Afouras, T., Chung, J. S., & Zisserman, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.00496 · Cited by 263 · cited 4× · see citing works
“We train our models on Lip Reading Sentences 2 (LRS2, 97k videos) [3] and Lip Reading Sentences 3 (LRS3, 120k videos) [4].”
MethodInfluentialGenerative Probabilistic Novelty Detection with Adversarial Autoencoders
Pidhorskyi, S., Almohsen, R., Adjeroh, D. A., & Doretto, G. (2018) · arXiv · doi:10.48550/arxiv.1807.02588 · Cited by 255 · cited 2× · see citing works
“There have been a variety of methods proposed for learning this distribution, such as GAN discriminators [37,56,66,81,89,91,104,113], flow-based models [106], and autoregressive models [93].”
BackgroundMethodReducing Network AgnostophobiaPreprint
Dhamija, A. R., Günther, M., & Boult, T. E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.04110 · Cited by 251 · see citing works
Detecting Deep-Fake Videos from Phoneme-Viseme Mismatches
Agarwal, S., Farid, H., Fried, O., & Agrawala, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00338 · Cited by 234 · see citing works
OpenGAN: Open-Set Recognition via Open Data Generation
Kong, S., & Ramanan, D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00085 · Cited by 225 · see citing works
Not made for each other- Audio-Visual Dissonance-based Deepfake Detection and Localization
Chugh, K., Gupta, P., Dhall, A., & Subramanian, R. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413700 · Cited by 225 · see citing works
Self-supervised Learning of Audio-Visual Objects from Video
Afouras, T., Owens, A., Chung, J. S., & Zisserman, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58523-5_13 · Cited by 221 · see citing works
Self-citationAudio-Visual Instance Discrimination with Cross-Modal Agreement
Morgado, P. M., Vasconcelos, N. M., & Misra, I. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01229 · Cited by 218 · see citing works
Tampering Detection and Localization Through Clustering of Camera-Based CNN Features
Bondi, L., Lameri, S., Güera, D., Bestagini, P., Delp, E. J., & Tubaro, S. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.232 · Cited by 217 · cited 1× · see citing works
“Some work [13, 26, 28, 47, 50, 53, 79] has used specialpurpose anomaly detection methods for image/video forensics.”
MethodJoint Audio-Visual Deepfake Detection
Zhou, Y., & Lim, S.-N. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01453 · Cited by 217 · see citing works
OC-FakeDect: Classifying Deepfakes Using One-class Variational Autoencoder
Khalid, H., & Woo, S. S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00336 · Cited by 197 · cited 1× · see citing works
“Some work [13, 26, 28, 47, 50, 53, 79] has used specialpurpose anomaly detection methods for image/video forensics.”
MethodVisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistency
Gao, R., & Grauman, K. L. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01524 · Cited by 193 · see citing works
Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection
Haliassos, A., Mira, R., Petridis, S., & Pantic, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01453 · Cited by 180 · see citing works
Tao, R., Pan, Z., Das, R. K., Qian, X., Shou, M. Z., & Li, H. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475587 · Cited by 157 · see citing works
Zhou, T., Wang, W., Liang, Z., & Shen, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00572 · Cited by 156 · see citing works
Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection
Tao, R., Pan, Z., Das, R. K., Qian, X., Shou, M. Z., & Li, H. (2021) · arXiv · doi:10.48550/arxiv.2107.06592 · Cited by 156 · see citing works
Towards Automatic Face-to-Face Translation
R, P. K., Mukhopadhyay, R., Philip, J., Jha, A., Namboodiri, V. P., & Jawahar, C. V. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3351066 · Cited by 151 · see citing works
Hybrid Models for Open Set Recognition
Zhang, H., Li, A., Guo, J. J., & Guo, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_7 · Cited by 151 · see citing works
KoDF: A Large-scale Korean DeepFake Detection Dataset
Kwon, P., You, J., Nam, G., Park, S., & Chae, G. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01057 · Cited by 145 · see citing works
Single-image splicing localization through autoencoder-based anomaly detection
Cozzolino, D., & Verdoliva, L. (2016) · IEEE International Workshop on Information Forensics and Security · doi:10.1109/wifs.2016.7823921 · Cited by 122 · see citing works
FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset
Khalid, H., Tariq, S., Kim, M., & Woo, S. S. (2021) · arXiv · doi:10.48550/arxiv.2108.05080 · Cited by 116 · cited 10× · see citing works
“Thus, following the standard procedure used in [40, 41, 109, 112], we conduct the experiment to evaluate the cross-manipulation generalization ability of our model on the FakeAVCeleb dataset [52] which videos are manipulated in various ways.”
BackgroundMethodInfluentialTowards Practical Lipreading with Distilled and Efficient Models
Ma, P., Martínez, B., Petridis, S., & Pantić, M. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9415063 · Cited by 115 · see citing works
Autoencoder with recurrent neural networks for video forgery detection
D'Avino, D., Cozzolino, D., Poggi, G., & Verdoliva, L. (2017) · Electronic Imaging · doi:10.2352/issn.2470-1173.2017.7.mwsf-330 · Cited by 109 · see citing works
Perfect Match: Improved Cross-modal Embeddings for Audio-visual Synchronisation
Chung, S.-W., Chung, J. S., & Kang, H.-G. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682524 · Cited by 108 · see citing works
DeepFaceLab: A simple, flexible and extensible face swapping framework.
Perov, I., Gao, D., Chervoniy, N., Liu, K., Marangonda, S., Umé, C., Dpfks, Facenheim, C. S., Rp, L., Jiang, J., et al. (2020) · Cited by 106 · see citing works
Audio-Visual Person-of-Interest DeepFake Detection
Cozzolino, D., Pianese, A., Nießner, M., & Verdoliva, L. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00101 · Cited by 104 · cited 3× · see citing works
“First, we consider an audio-visual contrastive learning model, which we call AVBYOL, that learns to determine whether the visual and audio streams of a video do (or do not) match, an approach that has been used as a part of other audio-visual forensics models [25, 40].”
MethodResultEyes Tell All: Irregular Pupil Shapes Reveal GAN-Generated Faces
Hui, G., Hu, S., Wang, X., Chang, M., & Lyu, S. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746597 · Cited by 100 · see citing works
Labelling unlabelled videos from scratch with multi-modal self-supervision
Asano, Y. M., Patrick, M., Rupprecht, C., & Vedaldi, A. (2020) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.48550/arxiv.2006.13662 · Cited by 94 · cited 1× · see citing works
“between frames and audio tracks [10, 75, 105] to learn multimodal features and applied them to downstream tasks such as sound localization [9, 45, 74].”
BackgroundLearning Second Order Local Anomaly for General Face Forgery Detection
Fei, J., Dai, Y., Yu, P., Shen, T., Xia, Z., & Weng, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01963 · Cited by 88 · see citing works
Audio-driven Talking Face Video Generation with Learning-based Personalized Head PosePreprint
Yi, R., Ye, Z., Zhang, J., Bao, H., & Liu, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.10137 · Cited by 83 · see citing works
Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation
Zhou, H., Xu, X., Lin, D., Wang, X., & Liu, Z. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58610-2_4 · Cited by 80 · see citing works
Audio forensics from acoustic reverberation
Malik, H. M. A., & Farid, H. (2010) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2010.5495479 · Cited by 68 · see citing works
Localizing Visual Sounds the Easy Way
Mo, S., & Morgado, P. M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_13 · Cited by 67 · see citing works
Satellite image forgery detection and localization using GAN and One-Class classifier
Yarlagadda, S. K., Güera, D., Bestagini, P., Zhu, F. M., Tubaro, S., & Delp, E. J. (2018) · Virtual Community of Pathological Anatomy (University of Castilla La Mancha) · doi:10.2352/issn.2470-1173.2018.07.mwsf-214 · Cited by 67 · see citing works
Deep Anomaly Detection for Generalized Face Anti-Spoofing
Pérez‐Cabo, D., Jiménez‐Cabello, D., Costa‐Pazo, A., & López-Sastre, R. J. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00201 · Cited by 65 · see citing works
Self-supervised object detection from audio-visual correspondence
Afouras, T., Asano, Y. M., Fagan, F., Vedaldi, A., & Metze, F. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01032 · Cited by 56 · see citing works
Mix and Localize: Localizing Sound Sources in Mixtures
Hu, X., Chen, Z., & Owens, A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01023 · Cited by 53 · see citing works
Self-citationHow to Design a Three-Stage Architecture for Audio-Visual Active Speaker Detection in the Wild
Köpüklü, O., Taseska, M., & Rigoll, G. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00123 · Cited by 51 · see citing works
MAAS: Multi-modal Assignation for Active Speaker Detection
Alcázar, J. L., Heilbron, F. C., Thabet, A., & Ghanem, B. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00033 · Cited by 50 · see citing works
Dynamic Inconsistency-aware DeepFake Video Detection
Hu, Z., Xie, H., Wang, Y., Li, J., Wang, Z., & Zhang, Y. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/102 · Cited by 49 · see citing works
Move2Hear: Active Audio-Visual Source Separation
Majumder, S., Al-Halah, Z., & Grauman, K. L. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00034 · Cited by 40 · see citing works
Durall, R., Keuper, M., & Keuper, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00791 · Cited by 26 · see citing works
ASR is All You Need: Cross-Modal Distillation for Lip Reading
Afouras, T., Chung, J. S., & Zisserman, A. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054253 · Cited by 25 · see citing works
Contrastive Learning of Global and Local Video Representations
Ma, S., Zeng, Z., McDuff, D., & Song, Y. (2021) · neural information processing systems · Cited by 23 · see citing works
Audio-Visual Synchronisation in the wildPreprint
Chen, H., Xie, W., Afouras, T., Nagrani, A., Vedaldi, A., & Zisserman, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.04432 · Cited by 21 · cited 3× · see citing works
“[18], we use ResNet-18 2D+3D [42, 43] as the visual encoder, using 5 frames (25 fps) as input.”
MethodVideoforensicshq: Detecting High-Quality Manipulated Face Videos
Fox, G., Liu, W., Kim, H., Seidel, H.-P., Elgharib, M., & Theobalt, C. (2021) · 2021 IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme51207.2021.9428101 · Cited by 13 · see citing works
Sound Localization by Self-supervised Time Delay Estimation
Chen, Z., Fouhey, D. F., & Owens, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19809-0_28 · Cited by 12 · see citing works
Self-citationMAAS: Multi-modal Assignation for Active Speaker DetectionPreprint
Alcázar, J. L., Heilbron, F. C., Thabet, A., & Ghanem, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.03682 · Cited by 8 · see citing works
Audio-Visual Person-of-Interest DeepFake DetectionPreprint
Cozzolino, D., Pianese, A., Nießner, M., & Verdoliva, L. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.03083 · Cited by 3 · cited 3× · see citing works
“First, we consider an audio-visual contrastive learning model, which we call AVBYOL, that learns to determine whether the visual and audio streams of a video do (or do not) match, an approach that has been used as a part of other audio-visual forensics models [25, 40].”
MethodResultDeep Audio-Visual Speech Recognition
Afouras, T., Chung, J. S., Senior, A., Vinyals, O., & Zisserman, A. (2018) · arXiv · doi:10.48550/arxiv.1809.02108 · cited 5×
“Second, we use a generative model VQGAN [32], trained on LRS2 [3], for anomaly detection.”
MethodInfluentialAutoencoder with recurrent neural networks for video forgery detection
D'Avino, D., Cozzolino, D., Poggi, G., & Verdoliva, L. (2017) · arXiv · doi:10.48550/arxiv.1708.08754 · cited 1×
“Some work [13, 26, 28, 47, 50, 53, 79] has used specialpurpose anomaly detection methods for image/video forensics.”
Method
Flagged first, then by citations
Research works printed but not linked: 3
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsu- pervised multitask learners. OpenAI blog, 1(8):9, 2019. 1, 2, 4
- Zhaoyang Zeng, Daniel McDuff, Yale Song, et al. Con- trastive learning of global and local video representa- tions. Advances in Neural Information Processing Systems, 34:7025-7040, 2021. 2
- Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cris- tian Lumezanu, Daeki Cho, and Haifeng Chen. Deep autoen- coding gaussian mixture model for unsupervised anomaly detection. In International conference on learning represen- tations, 2018. 2
Cited by
Sorted by citations
Binding Touch to Everything: Learning Unified Multimodal Tactile Representations
Yang, F., Feng, C., Chen, Z., Park, H., Wang, D., Dou, Y., Zeng, Z., Chen, X., Gangopadhyay, S., Owens, A., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02488 · Cited by 66 · see citing works
“Thus, we minimize the overall loss: Naturally, minimizing the contrastive objective [31, 113, 127, 145] will “pull” a visuo-tactile pair close together and “push” it away from other pairs, achieving the alignment between touch and visual embedding.”
Self-citationPVASS-MDD: Predictive Visual-Audio Alignment Self-Supervision for Multimodal Deepfake Detection
Yu, Y., Liu, X., Ni, R., Yang, S., Zhao, Y., & Kot, A. C. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3309899 · Cited by 63 · see citing works
“The SS Auxiliary methods [46], [47], [58] and multimodal detection method [43] are not compared in [44], thus the results of these methods are reproduced by ourselves.”
BackgroundMethodInfluentialAV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset
Cai, Z., Ghosh, S., Adatia, A. P., Hayat, M., Dhall, A., Gedeon, T., & Stefanov, K. (2024) · Proceedings of the 32nd ACM International Conference on Multimedia · doi:10.1145/3664647.3680795 · Cited by 59 · see citing works
SeeABLE: Soft Discrepancies and Bounded Contrastive Learning for Exposing Deepfakes
Larue, N., Vu, N., Štruc, V., Peer, P., & Christophides, V. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01921 · Cited by 51 · see citing works
“Another category of models [22,27,78,80] utilize temporal features for deepfake detection.”
BackgroundLiz, H., Keïta, M., Ahmed, A. T., Hadid, A., Huertas‐Tato, J., & Camacho, D. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102103 · Cited by 42 · see citing works
Integrating Audio-Visual Features For Multimodal Deepfake Detection
Muppalla, S., Jia, S., & Lyu, S. (2023) · 2023 IEEE MIT Undergraduate Research Technology Conference (URTC) · doi:10.1109/urtc60662.2023.10534969 · Cited by 46 · see citing works
“multimodal detectors that extract the audio-visual inconsistencies in deepfakes [14, 21, 12, 22].”
BackgroundDistributed intelligence for IoT-based smart cities: a survey
Hashem, M., Siddiqa, A., Alaba, F. A., Bilal, M., & Alhashmi, S. M. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10136-y · Cited by 33 · see citing works
“T h e fr a m e w o r k t o h a nd l e e n e r gy c on s u m p ti on a nd c o m pu t a ti on a l r e s ou r ce s A no m a l y d e t ec ti on [ 135 ] C oo r d i n a ti ng m od e l s ac r o ss d i s t r i bu t e d nod e s a nd c o m b i n i ng t h e i r r e s u lt s ca n b e c h a ll e ng i ng M od e…”
Audio–visual deepfake detection using articulatory representation learning
Wang, Y., & Huang, H. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.104133 · Cited by 28 · see citing works
Generating Visual Scenes from Touch
Yang, F., Zhang, J., & Owens, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02017 · Cited by 25 · see citing works
“Following other work in cross-modal synthesis [48, 50], we provide conditional information to our generation models through multimodal embeddings via contrastive learning [14, 66, 62, 59].”
Self-citationMIS-AVoiDD: Modality Invariant and Specific Representation for Audio-Visual Deepfake Detection
Katamneni, V. S., & Rattani, A. (2023) · International Conference on Machine Learning and Applications · doi:10.1109/icmla58977.2023.00207 · Cited by 28 · see citing works
“…[16] Audio-Visual 94.4 - Lip-Sync [17] Audio-Visual 83.3 0.976 AV-POI [18] Audio-Visual 86.6 0.942 VFD [19] Audio-Visual 81.5 0.861 AVForensics [20] Audio-Visual - 0.900 MultimodalTrace [21] Audio-Visual 92.9 - MIS-AVoiDD (Ours) Audio-Visual 96.2 0.973 Table II compares MIS-AVoiDD with SOTA…”
InfluentialExposing Lip-syncing Deepfakes from Mouth Inconsistencies
Datta, S. K., Jia, S., & Lyu, S. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10687902 · Cited by 27 · see citing works
Learning Natural Consistency Representation for Face Forgery Video Detection
Zhang, D., Xiao, Z., Li, S., Lin, F., Li, J., & Ge, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73010-8_24 · Cited by 24 · see citing works
FRADE: Forgery-aware Audio-distilled Multimodal Learning for Deepfake Detection
Nie, F., Ni, J., Zhang, J., Zhang, B., & Zhang, W. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681672 · Cited by 19 · see citing works
“Meanwhile, similar to [8], we resample the audio signal to 16 𝑘𝐻𝑧 and then transform it into the log-mel spectrogram by short-time Fourier Transform with 80 mel filter banks, a hop length of 160, and a window size of 320.”
InfluentialDeCLIP: Decoding CLIP Representations for Deepfake Localization
Smeu, S., Oneaţă, E., & Oneaţă, D. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00025 · Cited by 18 · see citing works
“Our code is available at: https://github.com/bit-ml/DeCLIP .”
A review of deep learning based multimodal forgery detection for video and audio
Tan, D., Yang, Y., Niu, C., Li, S., Yang, D., & Tan, B. (2025) · Discover Applied Sciences · doi:10.1007/s42452-025-07629-3 · Cited by 17 · see citing works
“…VFD [73] 86.11% KoDF AVFF [76] 95.50% POI-Forensics [71] 89.90% AVAD [77] 86.90% AV-Deepfake1M MDS [75] 56.57% AP FakeAVCeleb AVFF [76] 98.50% AVAD [77] 91.30% KoDF AVFF [76] 93.10% AVAD [77] 87.60% Lav-DF UMMAFormer [78] 92.48% EER ASVspoof 2021 (LA) SSL_Anti-spoofing [79] 0.82% T23 [80] 1.32%…”
InfluentialSoK: Systematization and Benchmarking of Deepfake Detectors in a Unified Framework
Le, B. M., Kim, J., Woo, S. S., Moore, K., Abuadbba, A., & Tariq, S. (2025) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp63326.2025.00055 · Cited by 23 · see citing works
“However, a systematic evaluation of recent methods with unified criteria is lacking.”
InfluentialContextual Cross-Modal Attention for Audio-Visual Deepfake Detection and Localization
Katamneni, V. S., & Rattani, A. (2024) · 2024 IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb62174.2024.10744480 · Cited by 20 · see citing works
“…0.962 Unsupervised Multi-modal Deepfake Detection [57] 0.968 - Audio-visual person-of-interest [18] 0.946 0.850 Audio-visual anomaly detection [20] 0.942 0.945 Multi-modal contrastive learning [36] 0.978 0.965 PVAS-MDD [62] 0.965 0.948 Novel Smart Deepfake Detection [19] 0.954 0.960…”
BackgroundInfluentialSemantics-Oriented Multitask Learning for DeepFake Detection: A Joint Embedding Approach
Zou, M., Yu, B., Zhan, Y., Lyu, S., & Ma, K. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3572508 · Cited by 15 · see citing works
“…[70] and vision Transformers [32], [71], [72], along with advanced training strategies such as adversarial training [25], [73], test-time training [74], graph learning [34], [36], contrastive learning [33], [38], and one-class classification [75], have also been explored for DeepFake detection.”
Increasing Importance of Joint Analysis of Audio and Video in Computer Vision: A Survey
Shahabaz, A., & Sarkar, S. (2024) · IEEE Access · doi:10.1109/access.2024.3391817 · Cited by 13 · see citing works
“…[186], [187], [188], [189], [190], [191], saliency prediction [59], [66], [76], [192] and other tasks that uses information from audio [4], [75], [79], [122], [158], [193], [194], [195], [196] to enhance pure-video-based classification, such as for a moving car, dog barking, or cooking an omelet.”
Background1M-Deepfakes Detection Challenge
Cai, Z., Dhall, A., Ghosh, S., Hayat, M., Kollias, D., Stefanov, K., & Tariq, U. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3689145 · Cited by 12 · see citing works
“The latest work on deepfake detection methodologies further supports this trend [12, 16, 31, 34, 39, 52, 54].”
MethodAudio-Visual Deepfake Detection With Local Temporal Inconsistencies
Astrid, M., Ghorbel, E., & Aouada, D. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889087 · Cited by 14 · see citing works
Shahzad, S. A., Hashmi, A., Peng, Y., Tsao, Y., & Wang, H. (2025) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2025.3618409 · Cited by 16 · see citing works
Circumventing shortcuts in audio-visual deepfake detection datasets with unsupervised learning
Smeu, S., Boldisor, D.-A., Oneaţă, D., & Oneaţă, E. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01753 · Cited by 13 · see citing works
“Among the first class, prior work proposed to verify that audio and visual streams align at a semantic level, for example, from the point of view of spoken content [2, 29], or at a representation level, based on the alignment of audio and video features [17, 44].”
InfluentialHiDF: A Human-Indistinguishable Deepfake Dataset
Kang, C., Jeong, S., Lee, J., Choi, D., Woo, S. S., & Han, J. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737399 · Cited by 10 · see citing works
“Extensive research has been conducted on the inconsistencies between the audio-visual modalities in deepfake videos, which demonstrates the effectiveness of utilizing multiple modalities rather than relying only on a single audio or visual feature in deepfake video detection [13, 22, 27, 80, 84].”
Systematic analysis of video tampering and detection techniques
Diwan, A., Dixit, S., Subbiah, R., & Mahadeva, R. (2024) · Cogent Engineering · doi:10.1080/23311916.2024.2424466 · Cited by 8 · see citing works
“By jointly analyzing multiple modalities, researchers can detect sophisticated tampering techniques that involve manipulation across different modalities (Diwan et al., 2024; Feng et al., 2023).”
Sound Localization from Motion: Jointly Learning Sound Direction and Camera Rotation
Chen, Z., Qian, S., & Owens, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00726 · Cited by 9 · see citing works
“Other approaches study audio-visual sound localization [39, 61, 16, 62], source separation [56, 31, 58, 87], active speaker detection [3, 86, 5], navigation [12, 13, 10] and forensics [105, 34, 24].”
BackgroundSelf-citationEnhancing Deepfake Detection: Proactive Forensics Techniques Using Digital Watermarking
Lai, Z. (2024) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2024.059370 · Cited by 7 · see citing works
“Furthermore, passive detection methods are reactive and cannot prevent the advanced spread of forged facial images or videos, and their results lack robust evidence to validate facial forgery [16–20].”
Localization and detection of deepfake videos based on self-blending method
Xu, J., Liu, X., Lin, W., Shang, W., & Wang, Y. (2025) · Scientific Reports · doi:10.1038/s41598-025-88523-1 · Cited by 10 · see citing works
Multi-modal Deepfake Detection via Multi-task Audio-Visual Prompt Learning
Miao, H., Guo, Y., Liu, Z., & Wang, Y. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i1.32042 · Cited by 8 · see citing works
“…multi-modal deepfake detection methods are selected, including MDS (Chugh et al. 2020), AV-DFD (Zhou and Lim 2021), BA-TFD (Cai et al. 2022), MMtrace (Raza and Malik 2023), AVAD (Feng, Chen, and Owens 2023), MRDF-CE (Zou et al. 2024), AVFF (Oorloff et al. 2024), and AVGraph (Yin et al. 2024).”
InfluentialForensics-Bench: A Comprehensive Forgery Detection Benchmark Suite for Large Vision Language Models
Wang, J., Lv, C., Li, X., Dong, S., Li, H., Yao, K., Li, C., Shao, W., & Luo, P. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00400 · Cited by 7 · see citing works
“[20, 25, 30, 44, 45, 78, 81, 82, 96, 97, 111, 112] have been proposed, expecting to increase the generalization capabilities of current forgery detectors.”
Statistics-Aware Audio-Visual Deepfake Detector
Astrid, M., Ghorbel, E., & Aouada, D. (2024) · Proceedings - International Conference on Image Processing · doi:10.1109/icip51287.2024.10648224 · Cited by 7 · see citing works
Explicit Correlation Learning for Generalizable Cross-Modal Deepfake Detection
Yu, C., Jia, S., Fu, X., Liu, J., Tian, J., Dai, J., Wang, X., Lyu, S., & Han, J. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10687814 · Cited by 8 · see citing works
“AVAD [7] detects cross-modal deep-fakes by predicting synchronization features pre-learned via an external audio-visual synchronization model.”
BackgroundMethodX3A: Efficient Multimodal Deepfake Detection with Score-Level Fusion
Park, C. S., Moon, B., Jeon, M., Jung, J.-W., & Woo, S. S. (2025) · ACM Symposium on Applied Computing · doi:10.1145/3672608.3707934 · Cited by 8 · see citing works
“Intermediate fusion models [19, 56] merge features from both modalities at the feature level, enabling better interaction be-tween data streams.”
Modality-Agnostic Deepfakes Detection
Cai, Y., Chen, P., Tian, J., Liu, J., Dai, J., Wang, X., Jia, S., Lyu, S., & Han, J. (2025) · ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec) · doi:10.1145/3733102.3733133 · Cited by 6 · see citing works
“…[25] 2022 Cross-modal Self-supervised Learning Binary × × BA-TFD [7] 2022 Temporal Synchronization Binary × × AVoiD-DF [64] 2023 Multi-modal Joint-decoder Binary × × AVFakeNet [30] 2023 Swin Transformer Dual-Classifier ✓ × AVAD [19] 2023 Autoregressive Synchronization Binary × × PVASS [68] 2023”
Exploring the Advancements and Challenges of Deepfake Face-swap: A Survey
Bakar, S. A. R. B. S. A., Waseem, S., Omar, Z., & Bilalashfaqahmed (2026) · Multimedia Tools and Applications · doi:10.1007/s11042-026-21285-8 · Cited by 11 · see citing works
A Review on Generative Intelligence in Deep Learning based Network Intrusion Detection
Ali, M., Udoidiok, I., Li, F., & Zhang, J. (2024) · Cyber Awareness and Research Symposium (CARS) · doi:10.1109/cars61786.2024.10778836 · Cited by 6 · see citing works
“Beyond their strong performance in modeling sequential data, autoregressive models can also contribute to IDS through their generative capabilities [44].”
Audio–visual self-supervised representation learning: A survey
Alsuwat, M. S., Al-Shareef, S., & Alghamdi, M. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129750 · Cited by 7 · see citing works
Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges
Liu, P., Tao, Q., & Zhou, J. T. (2024) · arXiv · doi:10.48550/arxiv.2406.06965 · Cited by 6 · see citing works
“Feng et al. [236] formulated detection as an anomaly detection task, training an audio-visual synchronization model on pristine data.”
Aravinda Reddy, P. N., Ramachandra, R., Rao, K. S., Mitra, P., & Rathod, V. (2024) · IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb62174.2024.10744460 · Cited by 5 · see citing works
“Current studies [16, 4, 9, 7] use the majority rule, ensemble voting, and fusion module to fuse audio and visual features for accurate audio-visual deepfake detection; however, these methods are unable to adapt to changing data characteristics, limited capacity to learn complex patterns, and…”
BackgroundMethodDo Deepfake Attribution Models Generalize?
Baxevanakis, S., Schinas, M., & Papadopoulos, S. (2025) · ACM International Workshop on Multimedia AI against Disinformation (MAD) · doi:10.1145/3733567.3735572 · Cited by 5 · see citing works
Bi-Stream Coteaching Network for Weakly-Supervised Deepfake Localization in Videos
Li, Z., Teng, Z., Zhang, B., & Jianping Fan, B. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3533906 · Cited by 5 · see citing works
“For cross-dataset detection, we follow established evaluation protocols by training on a partially manipulated FF ++ (c40) dataset, with the exception of AVF [50], which is pre-trained on Lip Reading datasets.”
Xiao, M., Ye, J., Liu, B., Gao, Z., Xu, K., & Wang, X. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733429 · Cited by 4 · see citing works
“Unlike supervised-based methods, existing self-supervised approaches has focused predominantly on extracting intrinsic features of authentic videos for more general deepfake detection [13, 18, 29].”
InfluentialFauForensics: Boosting Audio-Visual Deepfake Detection With Facial Action Units
Wang, J., Wu, B., Liu, L., & Liu, Q. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3674401 · Cited by 7 · see citing works
“To capture inter-modality temporal forgery traces, most research e ff orts were devoted to exploring audio-visual dissonance [33], [43], [44], [45], [46] and joint audio-visual feature learning [4], [47], [48], [49] for improving generalization across unseen manipulation methods.”
Review of AIGC passive detection and proactive protection
Liu, X.-L., Liu, H.-C., Zhao, Y., Ni, R.-R., Li, X.-L., & Guo, M.-Z. (2025) · Scientia Sinica Informationis · doi:10.1360/ssi-2025-0043 · Cited by 3 · see citing works
Analytic Synaptic Dynamic Scaling Balancer for Multimodal Deepfake Continual Detection
Xiao, M., Ye, J., Liu, B., Gao, Z., Xu, K., & Wang, X. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755680 · Cited by 3 · see citing works
PIA: Deepfake Detection Using Phoneme-Temporal and Identity-Dynamic Analysis
Datta, S. K., Ranga, T., Sun, C., & Lyu, S. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00170 · Cited by 4 · see citing works
“The audio–visual detectors exploit the lip–speech mis-matches to detect deepfakes [17, 52, 54].”
Multimodal VAD: Visual Anomaly Detection in Intelligent Monitoring System via Audio-Vision-Language
Wang, D., Wang, Q., Hu, Q., & Wu, K. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3578702 · Cited by 3 · see citing works
“These modalities’ complementarity e ff ectively compensates for the limitations that may exist in single modality, enabling capturing anomalous behaviors and better interpreting scenes through cross-modal fusion [24], [26], [33], [34].”
A Review of Tools and Technologies to Combat Deepfakes
Erokhin, D., & Komendantova, N. (2026) · Information · doi:10.3390/info17040347 · Cited by 2 · see citing works
Modal-Guided Multi-Domain Inconsistency Learning for Face Forgery Detection
Guo, Z., Zhang, B., Fan, J., Teng, Z., & Jianping Fan, B. (2024) · Applied Sciences · doi:10.3390/app15010229 · Cited by 2 · see citing works
Wang, W.-C., Coninck, S. D., Leroux, S., & Simoens, P. (2025) · Frontiers in Robotics and AI · doi:10.3389/frobt.2024.1490718 · Cited by 2 · see citing works
Domain-Adaptive Representation Learning for Multimodal Deepfake Detection
Haq, I. U., & Malik, K. M. (2025) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2025.3637242 · Cited by 2 · see citing works
Multiple contexts and frequencies aggregation network for deepfake detection
Li, Z., Tang, W., Gao, S., Wang, Y., & Wang, S. (2026) · PLoS ONE · doi:10.1371/journal.pone.0337409 · Cited by 4 · see citing works
Anshul, A., Chng, E. S., & Rajan, D. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00251 · Cited by 2 · see citing works
“Methods such as AVAD [14] and AVTS [44] use audio-visual synchronization tasks for pre-training, while AVFF [36] employs CV-MAE [16] to jointly learn audio-visual features.”
InfluentialSelf-Supervised Audio-Visual Soundscape Stylization
Li, T., Wang, R., Huang, P.-Y., Owens, A., & Anumanchipalli, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72989-8_2 · Cited by 2 · see citing works
Self-citationNguyen-Le, H.-H., Tran, V.-T., Nguyen, D.-T., & Le-Khac, N.-A. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11608-4 · Cited by 1 · see citing works
Anshul, A., Gopal, S., Rajan, D., & Chng, E. S. (2025) · 2025 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01283 · Cited by 1 · see citing works
“[10], we selected the maximum offset for a temporal shift, τ to be 15, thus generating 31 scores for each frame.”
GrDT: Towards Robust Deepfake Detection Using Geometric Representation Distribution and Texture
Xie, H., He, H., Fu, B., & Sanchez, V. (2025) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw65960.2025.00083 · Cited by 1 · see citing works
HTMDF-DD: Hybrid triple modality based spatial–temporal features early fusion for deepfake detection
Das, A. K., & Naskar, R. (2025) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2025.110860 · Cited by 1 · see citing works
Self-Supervised CLIP-Based Image Recognition and Analysis for Electronic Data Forensics
Wang, Y. (2025) · IEEE Access · doi:10.1109/access.2025.3648849 · Cited by 1 · see citing works
“In addition, Feng et al. modeled audio visual temporal inconsistencies [41], Chen et al. incorporated forgery diversity modeling and configuration prediction [42], and Wei et al.’s MaskFeat introduced a masked feature prediction mechanism—together illustrating the diverse directions of SSL research…”
A Hybrid Deep Learning Framework for Real and Deepfake Voice Detection
Murty, M., Tomar, S., & Koolagudi, S. G. (2026) · Circuits Systems and Signal Processing · doi:10.1007/s00034-025-03464-4 · Cited by 2 · see citing works
“Additionally, [13] presented a self-supervised method for video forensics by identifying audio-visual irregularities and utilizing discrepancies between modalities to detect modified information without requiring labeled data.”
Nie, F., Ni, J., Zhang, J., Zhang, B., Zhang, W. J., & Li, B. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3673104 · Cited by 1 · see citing works
“Following [14], [20], facial regions are extracted from the sampled frames by MTCNN [41].”
InfluentialVTFAformer: A visual–tactile fusion network for object attribute recognition
Yi, Z., Chen, G., Wang, H., Li, L., Li, B., & Qi, F. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.104813 · Cited by 1 · see citing works
AVIR: Audio-Visual Information Refinement for Robust Multimodal Deepfake Detection
Liu, X., Yang, Y. Y., Li, X., Zhao, G., & Zhao, Y. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3691216 · Cited by 2 · see citing works
“In order to ensure a fair comparison, we conduct training for the open-source methods [17] on our experimental datasets, refraining from the direct utilization of publicly available pre-trained models.”
InfluentialAnomaly Detection in Voice Conversations Through Spectral Features
Akilandeswari., T., Balaji, N., Paidimarri, N., & Yadav, G. N. (2024) · International Conference on Sustainable Computing and Smart Systems (ICSCSS) · doi:10.1109/icscss60660.2024.10625647 · Cited by 1 · see citing works
AI vs. AI: Deep Learning Approaches for Detecting Deepfake Content
Kulkarni, S., Vishwakarma, D. K., & Ranga, V. (2024) · International Conference on Advances in Computing, Communication and Networking (ICAC2N) · doi:10.1109/icac2n63387.2024.10895362 · Cited by 1 · see citing works
“Overall, the paper presents a huge step forward in video forensics by offering a robust and generalizable solution for detecting video manipulations from accurate, unlabeled data [9].”
Exposing Audio-Visual Forgeries in Frequency Domain
Wan, Y., Wang, J., Cui, J., & Sun, Y. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-1068-6_23
Adaptive Contribution Modulation For Multi-Modal Manipulation Media Detection and Grounding
Li, Y., & Leng, B. (2025) · IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp49660.2025.10890433 · Cited by 1 · see citing works
“Previous image manipulation detection methods [15], [16] and text manipulation detection methods [17], [18] have focused on a single modality, resulting in difficult to cope with complex multi-modal manipulations [19]–[25].”
Attention-Guided Contrastive Masked Autoencoders for Self-supervised Cross-Modal Biometric Matching
Zhang, H., Wang, J., Shi, H., Yu, Z., & Yin, L. (2025) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-89360-5_10
Representation Alignment For Deepfake Detection
Li, Z., Tang, W., Gao, S., Wang, Y., & Wang, S. (2025) · International Conference on Computing and Artificial Intelligence (ICCAI) · doi:10.1109/iccai66501.2025.00109
Convolutional Neural Networks for Robust Deepfake Detection in Digital Photography
Kundu, S., Bera, A., Neogi, S. K., Jaiswal, N., Ghosh, P., Das, S., & Mondal, M. A. (2025) · International Conference on Computing Technologies (ICOCT) · doi:10.1109/icoct64433.2025.11118479
DualStreamNet : Robust Audio-Video Deep Fake Media Detection Using Complimentary Information Fusion
Sheeranali, S., Ramachandra, R., & Venkatesh, S. K. (2025) · International Conference on Information Fusion (FUSION) · doi:10.23919/fusion65864.2025.11124073
“In [13], a self-supervised approach was proposed using an autoregressive model to capture the temporal synchronization between the frames and the sound of the sample.”
Improving Generalization in Deepfake Detection with Face Foundation Models and Metric Learning
Mylonas, S., & Papadopoulos, S. (2025) · doi:10.1145/3746275.3762208
“Although some recent works have explored temporal modeling [17, 18, 58, 64] or audio-visual cues [14, 28, 43] to improve robustness, our approach focuses on a more efficient, frame-based and vision-only setup.”
Tell Fake from Real: Temporal Forgery Localization with Fake-Real Guided Cross-Modal-Reconstruction
Yang, X., Zhou, Y., Xu, Y., Zou, Z., Ge, C., & He, Z. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-4381-6_20
Boosting Video Deepfake Detection via Quality-Aware and Multi-scale Audio-Visual Alignment
Chen, J., Che, X., Li, Q., & Zhao, Q. (2025) · Communications in computer and information science · doi:10.1007/978-981-95-4109-6_18
Frequency-Enhanced Multi-Modal Consistency Learning for Audio-Visual Deepfake Detection
Zheng, X., Zhang, D., & Sun, C. (2025) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3743093.3771003
“Following the prior work [11], we partition the FakeAVCeleb dataset into five categories based on the manipulation method: RVFA (Real Video, Fake Audio), FVRA-WL (Fake Video, Real Audio), FVFA-WL (Fake Video, Fake Audio, Wav2Lip), FVFA-FS (Fake Video, Fake Audio, FaceSwap), and FVFA-GAN (Fake…”
Audio-Visual Deepfake Detection via Multi-Teacher Distillation of Content and Semantic Consistency
Sun, B., Du, H., Fu, Y., Dong, T., & Lu, W. (2025) · IEEE International Conference on Visual Communications and Image Processing (VCIP) · doi:10.1109/vcip67698.2025.11396833
“AVoiD-DF [14] 83.70 89.20 AVAD [16] 94.20 94.50 MRDF-CE [31] 94.05 92.43 PVASS-MDD [15] 95.70 97.30 Baseline [18] – 98.86 MMMS-BA [33] 97.90 98.90 AVFF [17] 98.60 99.10 AVCS 98.26 99.64”
InfluentialCMRRL-Net: Cross-Modal Reconstruction Residual Learning for Audio–Visual Deepfake DetectionPreprint
Pandey, M., Koundal, D., & Kumar, D. S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6597781
Audio-Visual Deepfake Generation and Detection: An Exploratory Survey
Xu, H., An, Y., Fu, P., Fan, Z., Li, B., Liu, J., Liang, Y., & Yu, M. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463954
Are DeepFakes Realistic Enough? Exploring Semantic Mismatch as a Novel Challenge
Deshmukh, S. N., Hambarde, K., Costa, J. C., Proença, H. P. M. C., & Roxo, T. (2026) · IEEE Access · doi:10.1109/access.2026.3730281
“…of forgeries requiring simultaneous audio-video manipulation [11, 22] motivated the shift to joint audio-visual learning, where methods exploit synchronization patterns, cross-modal attention, and contrastive objectives [33, 34, 12, 7] to capture forgery traces that neither modality reveals alone.”
Video Deepfake Detection: Challenges and Recent Trends
Koutlis, C., Pianese, A., Cozzolino, D., Schinas, M., Mylonas, S., Verdoliva, L., & Papadopoulos, S. (2026) · Springer eBooks · doi:10.1007/978-3-032-11782-3_8
Visual Surface Wave Elastography: Revealing Subsurface Physical Properties via Visible Surface Waves
Ogren, A. C., Feng, B. T., Ahn, J., Bouman, K. L., & Daraio, C. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02454
“For example, previous work has leveraged surface vibrations [14] or sound [13, 19] in video to recover unknown information.”
LN_Net: Lightweight Non-interventionist Network for Weakly-Supervised Video Anomaly Detection
Shu, L., Zhu, T., Jiang, J., Yu, Q., Li, S. M., & Peng, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-7084-3_26
Zhu, J., Fan, Y., Han, G., Shi, X., Deng, F., & Chen, J. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11591-w
RA-FD: Resource-Adaptive Federated Distillation for Robust Audio-Visual Anomaly Detection
Qu, B. (2026) · International Conference on Digital Management and Information Technology (DMIT) · doi:10.1145/3808707.3808862
“For instance, Feng et al. [2] successfully utilized audio-visual synchronization features for forensics, demonstrating the critical value of audio-visual complementarity.”
Liu, W., Fu, C., Peng, H., Wu, J., Chen, X., Huang, L., Jing, X., Tian, Y., Liang, M., Wang, X., et al. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810582
Multimodal Deepfake Detection with Quantum State Inspired Analytic Incremental Adaptability Learning
Ye, J., Xiao, M., Liu, B., Hu, H., Gao, Z., Fu, S., Xu, K., & Wang, H. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810739
“Additionally, [10, 40] presents a self-supervised method for detecting manipulated videos through anomaly detection.”
TCFFNet: Temporal and Cross-Modal Feature Fusion Network for Deepfake Detection
Wu, Z., Yang, R., Lan, R., & Ghazali, K. H. (2026) · International Conference on Image, Signal Processing and Pattern Recognition (ISPP) · doi:10.1109/ispp69262.2026.11542863
“Rep-resentative studies model temporal coherence, identity inconsistency [18], and semantic spatiotemporal forgery patterns [19]–[21], while self-supervised pre-training has been explored to improve robustness in degraded scenarios [22], [23].”
SRG-OSyNet: A Neuro-Oscillation Inspired Audio-Visual Network for Deepfake Detection
Amin, Z., Subudhi, B. N., Nathwani, K., & Jakhetiya, V. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3705934
“Complementary developments in self-supervised learning [33] and contrastive learning and [16] improved the generalization of multimodal embed-dings by leveraging unlabeled data.”
Multimodal Consistency Anomaly Detection Method Based on MLP
Wang, K., Ma, Y., Lu, S., Li, X., & Dan, F. (2026) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc69976.2026.11560279
“Therefore, effectively detecting the consistency relationship between audio and video has become an important research problem in the fields of multimodal learning and anomaly detection[3].”
Adaptive Affinity Memorization With Layer Mutation for Multimodal Deepfake Continual Detection
Xiao, M., Ye, J., Liu, B., Gao, Z., Chen, W., Li, T., Wang, H., & Xu, K. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3707760
“Feng et al. [24] reformulate detection as a temporal anomaly task.”
Transfer Learning with Vision Transformer for Human-Indistinguishable Deepfake Detection
Amin, U., & Mateen, M. (2026) · International Conference on IT and Industrial Technologies (ICIT) · doi:10.1109/icit68548.2026.11577692
“AP AUC AVAD [14] 0.510 0.456 MARLIN-L [15] 0.511 0.491 FTCN [13] 0.631 0.697 EB4 (Previous SOTA) [22] 0.712 0.733 CADDM-Eb4 [12] 0.669 0.649 ViT-B/16 (Ours) 0.996 0.995”
InfluentialLLM as a forensic reasoner: Expert-guided video deepfake detection across modalities
Liu, J., Wang, J., Hou, S., Ren, M., Wu, H. B., Ma, L., & He, Z. (2026) · Pattern Recognition · doi:10.2139/ssrn.7051224
Wavelet Enhanced Lip Forgery Learning for Talking Face Generation Detection
Chai, Y., Xue, J., Ren, Y. Q., Huang, Y. N., & Liu, J. (2026) · IEEE Signal Processing Letters · doi:10.1109/lsp.2026.3716160
“This high deceptiveness challenges existing detectors [3], [4], [5], [6] designed for FS artifacts.”
Deepfake Detection Using Generative Artificial Intelligence Models
Shahapurkar, M. M., Sambrekar, K. P., & Kalghatgi, S. M. (2026) · International Conference on Intelligent Technologies (CONIT) · doi:10.1109/conit69683.2026.11620609
DATS-AV: A Dissonance-Aware Two-Stage Framework for Audio–Visual Deepfake Detection
Tonmoy, R. K., & Rattani, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31936-4_3
Coro, G., Cuttano, A., & Bardelli, S. (2026) · Neural Computing and Applications · doi:10.1007/s00521-026-12319-1
Dry-Type Transformers Fault Early Warning Based on SSTN-DBSCAN
Guan, X., Zhang, G., Cheng, Y., Duan, R., & Gong, Z. (2026) · Lecture notes in electrical engineering · doi:10.1007/978-981-92-2262-9_15
Zhang, W., Fan, X., Pan, Q., & Zhou, B. (2026) · IEEE Access · doi:10.1109/access.2026.3734244
“Different modalities may exhibit temporal misalignment, modality-specific noise, or missing information in practical surveillance systems [25].”
Zafar, A., Donepudi, H., Panda, R. K., Maliye, S., Sahni, N., & Nair, R. C. (2026) · IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT) · doi:10.1109/conecct68978.2026.11700622
“We compare StreamSentry against representative unimodal and multimodal baselines, including VFD [25], AVFakeNet [26], JointAV [27], AVAD [28], and AVFF.”
Leave No Stone Unturned: Uncovering Holistic Audio-Visual Intrinsic Coherence for Deepfake Detection
Peng, J., Wang, Y., Li, Y., Kong, L., & Hong, X. (2026) · doi:10.1109/cvprf71909.2026.00639
“2 in the main paper), we further conduct cross-manipulation experiments on the FakeAVCeleb dataset, following the protocols in [23, 56, 61].”
Influential
- Authors
- Feng, C., Chen, Z., & Owens, A.
- Affiliations
- University of Michigan (US)
- Venue
- 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2023-06
- Pages
- 10491-10503
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4386076652
- Cited by
- 101
- References
- 117
- Citations by year
- 2026: 35 · 2025: 39 · 2024: 20 · 2023: 7
- FWCI
- 10.80
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- U.S. Department of Defense
- Defense Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Mathematics › Statistics and Probability › Benford’s Law and Fraud Detection
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- video forensics, self-supervised learning, audio-video synchronization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
