CoChat
Sign Up
Cited by 85Digital Media Forensic DetectionGenerative Adversarial Networks and Image SynthesisMisinformation and Its Impacts

AVFF: Audio-Visual Feature Fusion for Video Deepfake Detection

Open access
Oorloff, Trevine, Koppisetti, Surya, Bonettini, Nicolò, Solanki, Divyaraj, Colman, Ben, Yacoob, Yaser, Shahriyari, Ali, Bharaj, Gaurav·2024·2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
DOI 10.1109/cvpr52733.2024.02559
Open this paper in CoChat — free accountOpen PDF

Abstract

With the rapid growth in deepfake video content, we re-quire improved and generalizable methods to detect them. Most existing detection methods either use uni-modal cues or rely on supervised training to capture the dissonance be-tween the audio and visual modalities. While the former disregards the audio-visual correspondences entirely, the lat-ter predominantly focuses on discerning audio-visual cues within the training corpus, thereby potentially overlooking correspondences that can help detect unseen deepfakes. We present Audio- Visual Feature Fusion (AVFF), a two-stage cross-modal learning method that explicitly captures the correspondence between the audio and visual modalities for improved deepfake detection. The first stage pursues representation learning via self-supervision on real videos to capture the intrinsic audio-visual correspondences. To extract rich cross-modal representations, we use contrastive learning and autoencoding objectives, and introduce a novel audio-visual complementary masking and feature fusion strategy. The learned representations are tuned in the second stage, where deepfake classification is pursued via super-vised learning on both real and fake videos. Extensive exper-iments and analysis suggest that our novel representation learning paradigm is highly discriminative in nature. We report 98.6% accuracy and 99.1% AUC on the FakeAVCeleb dataset, outperforming the current audio-visual state-of-the-art by 14.9% and 9.9%, respectively.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/cvpr52733.2024.02559

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The performance of any deepfake detection method is highly dependent on the quantitative and qualitative aspects of the datasets used for development.

    “The performance of any deepfake detection method is highly dependent on the quantitative and qualitative aspects of the datasets used for development [18, 21, 25, 40, 44, 47, 49, 51, 52, 61, 63, 69, 72, 75].”

    From AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset (2024) · cites this paper as [44]

    The Introduction (page not available in our copy) says training solely on a deepfake dataset can narrow a model’s focus to features of that training corpus and cause it to overlook correspondences useful for detecting unseen samples. The cross-manipulation evaluation also reports weaker performance for some baselines on particular manipulation categories. These passages support an effect of dataset composition, but do not establish that the quantity of development data strongly determines performance for any deepfake detection method. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Several researchers have explored transfer learning from self-supervised representations on generic datasets before fine-tuning for DF detection.

    “Several researchers have explored transfer learning from self-supervised representations on generic datasets before fine-tuning for DF detection [155, 181].”

    From Deepfake detection across image, video, and audio: a comprehensive survey with empirical evaluation of generalization and robustness (2026) · cites this paper as [181]

  • Recent multimodal deepfake detection research has begun to exploit richer behavioral and biometric correspondences between face, voice, and motion.

    “Beyond temporal synchronization, recent multimodal deepfake detection research has begun to exploit richer behavioral and biometric correspondences between face, voice, and motion [16-20].”

    From A Review of Tools and Technologies to Combat Deepfakes (2026) · cites this paper as [18]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

Cited by 85

Research works printed but not linked: 73

  1. Triantafyllos Afouras, Joon Son Chung, and Andrew Zis- serman. Lrs3-ted: a large-scale dataset for visual speech recognition. arXiv preprint arXiv:1809.00496, 2018. 2, 6, 9 · arXiv
  2. Shruti Agarwal, Hany Farid, Ohad Fried, and Maneesh Agrawala. Detecting deep-fake videos from phoneme-viseme mismatches. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 660-661, 2020. 1, 3
  3. Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasser- stein generative adversarial networks. In International con- ference on machine learning, pages 214-223. PMLR, 2017.
  4. Weiming Bai, Yufan Liu, Zhipeng Zhang, Bing Li, and Weim- ing Hu. Aunet: Learning relations between action units for face forgery detection. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24709-24719, 2023.
  5. Tadas Baltrušaitis, Chaitanya Ahuja, and Louis-Philippe Morency. Multimodal machine learning: A survey and tax- onomy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2):423-443, 2019. 2
  6. Zhixi Cai, Shreya Ghosh, Kalin Stefanov, Abhinav Dhall, Jianfei Cai, Hamid Rezatofighi, Reza Haffari, and Munawar Hayat. Marlin: Masked autoencoder for facial video represen- tation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1493- 1504, 2023. 4, 5, 6, 8, 9
  7. Harry Cheng, Yangyang Guo, Tianyi Wang, Qi Li, Xiaojun Chang, and Liqiang Nie. Voice-face homogeneity tells deep- fake. arXiv preprint arXiv:2203.02195, 2022. 3, 6 · arXiv
  8. Komal Chugh, Parul Gupta, Abhinav Dhall, and Ra- manathan Subramanian. Not made for each other-audio-visual dissonance-based deepfake detection and localization. In Proceedings of the 28th ACM international conference on multimedia, pages 439-447, 2020. 3, 6
  9. J Chung, A Nagrani, and A Zisserman. Voxceleb2: Deep speaker recognition. Interspeech 2018, 2018. 9
  10. J. S. Chung and A. Zisserman. Out of time: automated lip sync in the wild. In Workshop on Multi-view Lip-reading, ACCV, 2016. 1, 2
  11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transform- ers for language understanding. In North American Chapter of the Association for Computational Linguistics, 2019. 2
  12. Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, and Cristian Canton Ferrer. The deepfake detection challenge (dfdc) dataset. arXiv preprint arXiv:2006.07397, 2020. 2, 7, 10 · arXiv
  13. Shichao Dong, Jin Wang, Renhe Ji, Jiajun Liang, Haoqiang Fan, and Zheng Ge. Implicit identity leakage: The stumbling block to improving deepfake detection generalization. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3994-4004, 2023. 3
  14. Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Ting Zhang, Weiming Zhang, Nenghai Yu, Dong Chen, Fang Wen, and Baining Guo. Protecting celebrities from deepfake with iden- tity consistency transformer. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9458-9468, 2022. 3
  15. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Transform- ers for image recognition at scale. In International Conference on Learning Representations, 2020. 3, 9
  16. Chao Feng, Ziyang Chen, and Andrew Owens. Self- supervised video forensics by audio-visual anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10491-10503, 2023. 6, 7, 10, 11
  17. Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter. Audio set: An ontology and human-labeled dataset for audio events. In 2017 IEEE international confer- ence on acoustics, speech and signal processing (ICASSP), pages 776-780. IEEE, 2017. 9
  18. Mariana-Iuliana Georgescu, Eduardo Fonseca, Radu Tudor Ionescu, Mario Lucic, Cordelia Schmid, and Anurag Arnab. Audiovisual masked autoencoders. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16144-16154, 2023. 2
  19. Yuan Gong, Andrew Rouditchenko, Alexander H. Liu, David Harwath, Leonid Karlinsky, Hilde Kuehne, and James R. Glass. Contrastive audio-visual masked autoencoder. In ICLR. OpenReview.net, 2023. 2, 4, 5, 6, 8, 9
  20. Andrey Guzhov, Federico Raue, Jörn Hees, and Andreas Dengel. Audioclip: Extending clip to image, text and audio. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 976-980. IEEE, 2022. 1, 2, 4
  21. Alexandros Haliassos, Konstantinos Vougioukas, Stavros Petridis, and Maja Pantic. Lips don't lie: A generalisable and robust approach to face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5039-5049, 2021. 3, 6, 7, 9, 10
  22. Alexandros Haliassos, Rodrigo Mira, Stavros Petridis, and Maja Pantic. Leveraging real talking faces via self- supervision for robust forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14950-14962, 2022. 2, 3, 6, 7, 9, 10, 11
  23. Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15979-15988, 2022. 2, 8
  24. Baojin Huang, Zhongyuan Wang, Jifan Yang, Jiaxin Ai, Qin Zou, Qian Wang, and Dengpan Ye. Implicit identity driven deepfake face swapping detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4490-4499, 2023. 3
  25. Dong Huang and Fernando De La Torre. Facial action transfer with personalized bilinear regression. In Computer Vision- ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part II 12, pages 144-158. Springer, 2012. 10
  26. Po-Yao Huang, Hu Xu, Juncheng Li, Alexei Baevski, Michael Auli, Wojciech Galuba, Florian Metze, and Christoph Feicht- enhofer. Masked autoencoders that listen. Advances in Neural Information Processing Systems, 35:28708-28720, 2022. 4, 5, 6, 8, 9
  27. Yujin Jeong, Wonjeong Ryoo, Seunghyun Lee, Dabin Seo, Wonmin Byeon, Sangpil Kim, and Jinkyu Kim. The power of sound (tpos): Audio reactive video generation with stable diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7822-7832, 2023. 8
  28. Ye Jia, Yu Zhang, Ron Weiss, Quan Wang, Jonathan Shen, Fei Ren, Patrick Nguyen, Ruoming Pang, Ignacio Lopez Moreno, Yonghui Wu, et al. Transfer learning from speaker verification to multispeaker text-to-speech synthesis. Advances in neural information processing systems, 31, 2018. 6, 9
  29. Liming Jiang, Ren Li, Wayne Wu, Chen Qian, and Chen Change Loy. Deeperforensics-1.0: A large-scale dataset for real-world face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition, pages 2889-2898, 2020. 10
  30. Tackhyun Jung, Sangwon Kim, and Keecheon Kim. Deepvi- sion: Deepfakes detection using human eye blinking pattern. IEEE Access, 8:83144-83154, 2020. 3
  31. Bachir Kaddar, Sid Ahmed Fezza, Wassim Hamidouche, Za- hid Akhtar, and Abdenour Hadid. Hcit: Deepfake video detection using a hybrid model of cnn features and vision transformer. In 2021 International Conference on Visual Communications and Image Processing (VCIP), pages 1-5, 2021. 3
  32. Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401-4410, 2019. 10
  33. Hasam Khalid, Shahroz Tariq, Minha Kim, and Simon S Woo. Fakeavceleb: A novel audio-video multimodal deepfake dataset. arXiv preprint arXiv:2108.05080, 2021. 2, 6, 9 · arXiv
  34. Pavel Korshunov and Sébastien Marcel. Deepfakes: a new threat to face recognition? assessment and detection. arXiv preprint arXiv:1812.08685, 2018. 7, 10 · arXiv
  35. Iryna Korshunova, Wenzhe Shi, Joni Dambre, and Lucas Theis. Fast face-swap using convolutional neural networks. In Proceedings of the IEEE international conference on com- puter vision, pages 3677-3685, 2017. 6, 9, 10
  36. Patrick Kwon, Jaeseong You, Gyuhyeon Nam, Sungwoo Park, and Gyeongsu Chae. Kodf: A large-scale korean deepfake detection dataset. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 10744-10753, 2021. 7, 9
  37. Yuezun Li, Ming-Ching Chang, and Siwei Lyu. In ictu oculi: Exposing ai created fake videos by detecting eye blinking. In 2018 IEEE International Workshop on Information Forensics and Security (WIFS), pages 1-7, 2018. 3
  38. Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations, 2016. 9
  39. Ilya Loshchilov and Frank Hutter. Decoupled weight de- cay regularization. In International Conference on Learning Representations, 2018. 9
  40. Kevin Lutz and Robert Bassett. Deepfake detection with inconsistent head poses: Reproducibility and analysis. CoRR, abs/2108.12715, 2021. 3 · arXiv
  41. Shugao Ma, Tomas Simon, Jason Saragih, Dawei Wang, Yuecheng Li, Fernando De la Torre, and Yaser Sheikh. Pixel codec avatars. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 64-73, 2021. 1
  42. Trisha Mittal, Uttaran Bhattacharya, Rohan Chandra, Aniket Bera, and Dinesh Manocha. Emotions don't lie: An audio- visual deepfake detection method using affective cues. In Proceedings of the 28th ACM international conference on multimedia, pages 2823-2832, 2020. 1, 2, 3, 6
  43. Trisha Mittal, Uttaran Bhattacharya, Rohan Chandra, Aniket Bera, and Dinesh Manocha. M3er: Multiplicative multimodal emotion recognition using facial, textual, and speech cues. In Proceedings of the AAAI conference on artificial intelligence, pages 1359-1367, 2020. 1
  44. Pedro Morgado, Ishan Misra, and Nuno Vasconcelos. Ro- bust audio-visual instance discrimination. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12934-12945, 2021. 4
  45. Yuval Nirkin, Yosi Keller, and Tal Hassner. Fsgan: Subject agnostic face swapping and reenactment. In Proceedings of the IEEE/CVF international conference on computer vision, pages 7184-7193, 2019. 6, 9, 10
  46. Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. Towards uni- versal fake image detectors that generalize across generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24480-24489, 2023. 3
  47. Ivan Perov, Daiheng Gao, Nikolay Chervoniy, Kunlin Liu, Sugasa Marangonda, Chris Umé, Mr Dpfks, Carl Shift Facen- heim, Luis RP, Jian Jiang, et al. Deepfacelab: Integrated, flex- ible and extensible face-swapping framework. arXiv preprint arXiv:2005.05535, 2020. 10 · arXiv
  48. Adam Polyak, Lior Wolf, and Yaniv Taigman. Tts skins: Speaker conversion via asr. arXiv preprint arXiv:1904.08983, 2019. 10 · arXiv
  49. KR Prajwal, Rudrabha Mukhopadhyay, Vinay P Namboodiri, and CV Jawahar. A lip sync expert is all you need for speech to lip generation in the wild. In Proceedings of the 28th ACM international conference on multimedia, pages 484-492, 2020. 1, 6, 9, 10
  50. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, pages 8748-8763. PMLR, 2021. 1, 2, 4
  51. Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Chris- tian Riess, Justus Thies, and Matthias Nießner. Faceforen- sics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1-11, 2019. 2, 6, 7, 9
  52. Andrew Rouditchenko, Angie Boggust, David Harwath, Brian Chen, Dhiraj Joshi, Samuel Thomas, Kartik Audhkhasi, Hilde Kuehne, Rameswar Panda, Rogerio Feris, et al. Avlnet: Learn- ing audio-visual language representations from instructional videos. In Annual Conference of the International Speech Communication Association. International Speech Communi- cation Association, 2021. 4
  53. Ludan Ruan, Yiyang Ma, Huan Yang, Huiguo He, Bei Liu, Jianlong Fu, Nicholas Jing Yuan, Qin Jin, and Bain- ing Guo. Mm-diffusion: Learning multi-modal diffusion models for joint audio and video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10219-10228, 2023. 8
  54. Conrad Sanderson and Brian C Lovell. Multi-region proba- bilistic histograms for robust and scalable identity inference. In Advances in biometrics: Third international conference, ICB 2009, alghero, italy, june 2-5, 2009. Proceedings 3, pages 199-208. Springer, 2009. 10
  55. Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. First order motion model for image animation. Advances in neural information processing systems, 32, 2019. 10
  56. Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. Video- mae: Masked autoencoders are data-efficient learners for self-supervised video pre-training. Advances in neural infor- mation processing systems, 35:10078-10093, 2022. 9
  57. Laurens van der Maaten and Geoffrey E. Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9: 2579-2605, 2008. 2
  58. Jun Wang, Yinglu Liu, Yibo Hu, Hailin Shi, and Tao Mei. Facex-zoo: A pytorch toolbox for face recognition. In Pro- ceedings of the 29th ACM International Conference on Multi- media, pages 3779-3782, 2021. 4
  59. Ting-Chun Wang, Arun Mallya, and Ming-Yu Liu. One-shot free-view neural talking-head synthesis for video conferenc- ing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10039-10049, 2021. 1
  60. Zhendong Wang, Jianmin Bao, Wengang Zhou, Weilun Wang, and Houqiang Li. Altfreezing for more general video face forgery detection. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), pages 4129-4138, 2023. 3
  61. Deressa Wodajo and Solomon Atnafu. Deepfake video detec- tion using convolutional vision transformer. arXiv preprint arXiv:2102.11126, 2021. 3, 6 · arXiv
  62. Lior Wolf, Tal Hassner, and Itay Maoz. Face recognition in unconstrained videos with matched background similarity. In CVPR 2011, pages 529-534. IEEE, 2011. 2, 9
  63. Haojie Wu, Pan Hui, and Pengyuan Zhou. Deepfake in the metaverse: An outlook survey. arXiv preprint arXiv:2306.07011, 2023. 1 · arXiv
  64. Wenyuan Yang, Xiaoyu Zhou, Zhikai Chen, Bofei Guo, Zhongjie Ba, Zhihua Xia, Xiaochun Cao, and Kui Ren. Avoid- df: Audio-visual joint learning for detecting deepfake. IEEE Transactions on Information Forensics and Security, 18:2015- 2029, 2023. 2, 3, 6
  65. Xin Yang, Yuezun Li, and Siwei Lyu. Exposing deep fakes using inconsistent head poses. In ICASSP 2019 -2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8261-8265, 2019. 3
  66. Ran Yi, Zipeng Ye, Juyong Zhang, Hujun Bao, and Yong- Jin Liu. Audio-driven talking face video generation with learning-based personalized head pose. arXiv preprint arXiv:2002.10137, 2020. 10 · arXiv
  67. Egor Zakharov, Aliaksandra Shysheya, Egor Burkov, and Victor Lempitsky. Few-shot adversarial learning of realistic neural talking head models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9459- 9468, 2019. 10
  68. Xiaohui Zhao, Yang Yu, Rongrong Ni, and Yao Zhao. Ex- ploring complementarity of global and local spatiotemporal information for fake face video detection. In ICASSP 2022 -2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2884-2888, 2022. 3
  69. Yinglin Zheng, Jianmin Bao, Dong Chen, Ming Zeng, and Fang Wen. Exploring temporal coherence for more general video face forgery detection. In Proceedings of the IEEE/CVF international conference on computer vision, pages 15044- 15054, 2021. 3, 6, 7, 9
  70. Yipin Zhou and Ser-Nam Lim. Joint audio-visual deepfake detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14800-14809, 2021. 3, 7
  71. Yang Zhou, Zhan Xu, Chris Landreth, Evangelos Kalogerakis, Subhransu Maji, and Karan Singh. Visemenet: Audio-driven animator-centric speech animation. ACM Transactions on Graphics (TOG), 37(4):1-10, 2018.
  72. Hao Zhu, Wayne Wu, Wentao Zhu, Liming Jiang, Siwei Tang, Li Zhang, Ziwei Liu, and Chen Change Loy. Celebv-hq: A large-scale video facial attributes dataset. In European conference on computer vision, pages 650-667. Springer, 2022. 2
  73. Wanyi Zhuang, Qi Chu, Zhentao Tan, Qiankun Liu, Haojie Yuan, Changtao Miao, Zixiang Luo, and Nenghai Yu. Uia-vit: Unsupervised inconsistency-aware method based on vision transformer for face forgery detection. In Computer Vision -ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part V, page 391-407, Berlin, Heidelberg, 2022. Springer-Verlag. 3

Authors
Oorloff, T., Koppisetti, S., Bonettini, N., Solanki, D., Colman, B., Yacoob, Y., Shahriyari, A., & Bharaj, G.
Affiliations
  • University of Maryland, College Park (US)
  • Reality Defender Inc.
ORCID
Venue
2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2024-06-16
Pages
27092-27102
ISSN
1063-6919 (ISSN-L)
DOI
10.1109/cvpr52733.2024.02559
OpenAlex
W4402774452
Cited by
85
References
0
Citations by year
2026: 40 · 2025: 41 · 2024: 4
FWCI
17.61
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
DARPA SemaFor Program (HR001120e0124)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
Keywords
deepfake detection, audio-visual feature fusion, cross-modal learning, audio-visual correspondence, self-supervised representation learning, contrastive learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review