The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Emotions Don't Lie: An Audio-Visual Deepfake Detection Method using Affective Cues
Abstract
We present a learning-based method for detecting real and fake deepfake multimedia content. To maximize information for learning, we extract and analyze the similarity between the two audio and visual modalities from within the same video. Additionally, we extract and compare affective cues corresponding to perceived emotion from the two modalities within a video to infer whether the input video is "real" or "fake". We propose a deep learning network, inspired by the Siamese network architecture and the triplet loss. To validate our model, we report the AUC metric on two large-scale deepfake detection datasets, DeepFake-TIMIT Dataset and DFDC. We compare our approach with several SOTA deepfake detection methods and report per-video AUC of 84.4% on the DFDC and 96.6% on the DF-TIMIT datasets, respectively. To the best of our knowledge, ours is the first approach that simultaneously exploits audio and video modalities and also perceived emotions from the two modalities for deepfake detection.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“To overcome these limitations, multimodal approaches have been recently proposed, able to combine information from various domains to enhance the accuracy of the detection process [7,8].”
From A Robust Approach to Multimodal Deepfake Detection (2023) · cites this paper as [8]
In the citing paper
“They evaluate their technique to multiple SOTA deepfake detection approaches, reporting pervideo AUCs of 84.4% on the DFDC dataset and 96.6% on the DF-TIMIT dataset.”
From Deepfake detection using deep learning methods: A systematic and comprehensive review (2024) · cites this paper as Mittal et al. (2020)
In the citing paper
“Their method is the first to use both audio and video modalities and perceived emotions from two modalities to detect deepfakes.”
From Deepfake detection using deep learning methods: A systematic and comprehensive review (2024) · cites this paper as Mittal et al. (2020)
In the citing paper
“The performance of any deepfake detection method is highly dependent on the quantitative and qualitative aspects of the datasets used for development [18, 21, 25, 40, 44, 47, 49, 51, 52, 61, 63, 69, 72, 75].”
From AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset (2024) · cites this paper as [40]
In the citing paper
“Then, compared with several SOTA DFD methods, such as Two-Stream, MesoNet, HeadPose, FWA, VA, Xception, Multi-task, Capsule, and DSP-FWA, they report a per-video AUC of 84.4% on the DFDC and 96.6% on the DF-TIMIT datasets.”
From A Comprehensive Review of DeepFake Detection Using Advanced Machine Learning and Fusion Methods (2023) · cites this paper as [21]
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Multimodal Machine Learning: A Survey and Taxonomy
Baltrusaitis, T., Ahuja, C., & Morency, L.-P. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2798607 · Cited by 5,018
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 5×
“[49] use the XceptionNet, and Zhou et al.”
MethodInfluentialMesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
Bagher Zadeh, A., Liang, P. P., Poria, S., Cambria, E., & Morency, L.-P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1208 · Cited by 1,433 · cited 2×
“There are many other applications of video processing that use and combine multiple modalities for audio-visual speech recognition [28], emotion recognition [40, 63], and language and vision tasks [17, 29].”
BackgroundMethodDeepfake Video Detection Using Recurrent Neural Networks
Guera, D., & Delp, E. J. (2018) · 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2018.8639163 · Cited by 1,285 · cited 2×
“3413570 23, 33, 49] and algorithms [9, 26, 36, 38, 42, 43, 49, 50, 57, 61, 64] for deepfake detection.”
MethodExposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196
Capsule-forensics: Using Capsule Networks to Detect Forged Images and Videos
Nguyen, H. H., Yamagishi, J., & Echizen, I. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682602 · Cited by 951
Facial signs of emotional experience.
Ekman, P., Freisen, W. V., & Ancoli, S. (1980) · Journal of Personality and Social Psychology · doi:10.1037/h0077722 · Cited by 886 · cited 1×
“The CMU-MOSEI dataset describes the perceived emotion space with 6 discrete emotions following the Ekman model [24]: happy, sad, angry, fearful, surprise, and disgust, and a “neutral” emotion to denote the absence of any of these emotions.”
MethodDeep Fakes: A Looming Challenge for Privacy, Democracy, and National Security
Chesney, R., & Citron, D. K. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3213954 · Cited by 880 · cited 1×
“deepfakes [22], that use artificial intelligence to present false media as realistic in highly sophisticated and convincing ways.”
BackgroundExploiting Visual Artifacts to Expose Deepfakes and Face Manipulations
Matern, F., Riess, C., & Stamminger, M. (2019) · 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw.2019.00020 · Cited by 824 · cited 4×
“(5) VA [38] focuses on capturing visual artifacts in the eyes, teeth and facial contours of synthesized faces.”
BackgroundMethodInfluentialExposing DeepFake Videos By Detecting Face Warping Artifacts
Li, Y., & Lyu, S. (2018) · arXiv · doi:10.48550/arxiv.1811.00656 · Cited by 785 · cited 6×
“(4) FWA [36] uses a CNN to expose the face warping artifacts introduced by the resizing and interpolation operations.”
BackgroundMethodInfluentialTwo-Stream Neural Networks for Tampered Face Detection
Zhou, P., Han, X., Morariu, V. I., & Davis, L. S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.229 · Cited by 700 · cited 3×
“3413570 23, 33, 49] and algorithms [9, 26, 36, 38, 42, 43, 49, 50, 57, 61, 64] for deepfake detection.”
MethodRoss, L. A., Saint-Amour, D., Leavitt, V. M., Javitt, D. C., & Foxe, J. J. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhl024 · Cited by 650 · cited 2×
“More specifically, [48, 51, 55, 65] suggest some positive correlation between audio-visual modalities, which have been exploited for multimodal perceived emotion recognition.”
BackgroundDeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection
Jiang, L., Li, R., Wu, W., Qian, C., & Loy, C. C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00296 · Cited by 599
Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos
Nguyen, H. H., Fang, F., Yamagishi, J., & Echizen, I. (2019) · 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS) · doi:10.1109/btas46853.2019.9185974 · Cited by 549 · cited 2×
“Multi-task [34] uses a CNN to simultaneously detect manipulated images and segment manipulated areas as a multi-task learning problem.”
BackgroundMethodDeepFakes: a New Threat to Face Recognition? Assessment and Detection
Korshunov, P., & Marcel, S. (2018) · arXiv · doi:10.48550/arxiv.1812.08685 · Cited by 497 · cited 4×
“We perform experiments on the DF-TIMIT [33] and DFDC [23] datasets, as only these datasets contain modalities for face and speech features (1).”
BackgroundMethodInfluentialpyAudioAnalysis: An Open-Source Python Library for Audio Signal Analysis
Giannakopoulos, T. (2015) · PLOS ONE · doi:10.1371/journal.pone.0144610 · Cited by 454 · cited 2×
“To extract speech features, we use pyAudioAnalysis [25] to extract 13 Mel Frequency Cepstral Coefficients (MFCC) speech features.”
MethodRecurrent convolutional strategies for face manipulation detection in videos
Sabir, E., Cheng, J., Jaiswal, A., AbdAlmageed, W., Masi, I., & Natarajan, P. (2019) · arXiv · doi:10.48550/arxiv.1905.00582 · Cited by 443 · cited 2×
“3413570 23, 33, 49] and algorithms [9, 26, 36, 38, 42, 43, 49, 50, 57, 61, 64] for deepfake detection.”
BackgroundMethodScherer, K. R., Johnstone, & Klasmeyer, G. (2002) · Handbook of Affective Sciences · doi:10.1093/oso/9780195126013.003.0023 · Cited by 426
The Deepfake Detection Challenge (DFDC) Preview Dataset
Dolhansky, B., Howes, R., Pflaum, B., Baram, N., & Ferrer, C. C. (2019) · arXiv · doi:10.48550/arxiv.1910.08854 · Cited by 413 · cited 5×
“DF-TIMIT [33] DFDC [23]”
MethodInfluentialBi-modal emotion recognition from expressive face and body gestures
Gunes, H., & Piccardi, M. (2007) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2006.09.007 · Cited by 382 · cited 2×
“For instance, [12, 27] suggests that when different modalities are modeled and projected into a common space, they should point to similar affective cues.”
BackgroundFace alignment through subspace constrained mean-shifts
Saragih, J. M., Lucey, S., & Cohn, J. F. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459377 · Cited by 368 · cited 1×
“There is vast literature in inference of perceived emotions from a single modality or a combination of multiple modalities like facial expressions [10, 53], speech/audio signals [54], body pose [41], walking styles [16] and physiological features [32].”
BackgroundSeeing to hear better: evidence for early audio-visual interactions in speech identification
SCHWARTZ, J. (2004) · Cognition · doi:10.1016/s0010-0277(04)00054-x · Cited by 279
Seeing to hear better: evidence for early audio-visual interactions in speech identification
Schwartz, J.-L., Berthommier, F., & Savariaux, C. (2004) · Cognition · doi:10.1016/j.cognition.2004.01.006 · Cited by 275 · cited 2×
“More specifically, [48, 51, 55, 65] suggest some positive correlation between audio-visual modalities, which have been exploited for multimodal perceived emotion recognition.”
BackgroundM3ER: Multiplicative Multimodal Emotion Recognition using Facial, Textual, and Speech Cues
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., & Manocha, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i02.5492 · Cited by 253
Self-citationAffective multimodal human-computer interaction
Pantic, M., Sebe, N., Cohn, J. F., & Huang, T. (2005) · Proceedings of the 13th annual ACM international conference on Multimedia · doi:10.1145/1101149.1101299 · Cited by 234 · cited 1×
“Subsequently, such correlation among modalities has been explored extensively to perform multimodal emotion recognition [12, 27, 44].”
BackgroundInducing and Measuring Emotion and Affect: Tips, Tricks, and Secrets
Quigley, K. S., Lindquist, K. A., & Barrett, L. F. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9780511996481.014 · Cited by 163
Beyond Facial Expressions: Learning Human Emotion from Body Gestures
Shan, C., Gong, S., & McOwan, P. W. (2007) · British Machine Vision Conference (BMVC) · doi:10.5244/c.21.43 · Cited by 127 · cited 2×
“[56] state that even if two modalities representing the same emotion vary in terms of appearance, the features detected are similar and should be correlated.”
BackgroundEmotion-Oriented Systems: The Humaine Handbook
Petta, P., Pélachaud, C., & Cowie, R. (2011) · Cognitive technologies · doi:10.1007/978-3-642-15184-2 · Cited by 101
Wang, C., Zhou, Z., Jin, X., Fang, Y., & Lee, M. (2017) · Information Processing & Management · doi:10.1016/j.ipm.2017.02.003 · Cited by 100 · cited 1×
“Affective cues are specific features that convey rich emotional and behavioral information to human observers and help them distinguish between different perceived emotions [59].”
BackgroundFraming image description as a ranking task
HodoshMicah, YoungPeter, & HockenmaierJulia (2013) · doi:10.5555/2566972.2566993 · Cited by 98
Speaker Inconsistency Detection in Tampered Video
Korshunov, P., & Marcel, S. (2018) · 2018 26th European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco.2018.8553270 · Cited by 97 · cited 1×
“[34] analyze the lip-syncing inconsistencies using two channels, the audio and visual of moving lips.”
MethodEsposito, A., Müller, V. C., Hoffmann, R., Vinciarelli, A., & Esposito, A. M. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34584-5 · Cited by 66
DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery DetectionPreprint
Jiang, L., Li, R., Wu, W., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.03024 · Cited by 65 · cited 1×
“0 [31] Jan 2020 50,000 10,000 60,000 Actors DF ✓ –”
Audio‐Textual Emotion Recognition Based on Improved Neural Networks
Cai, L., Hu, Y., Dong, J., & Sitong, Z. (2019) · Mathematical Problems in Engineering · doi:10.1155/2019/2593036 · Cited by 64 · cited 1×
“Prior works [18, 19, 21] using audio or speech signals for various tasks like perceived emotion recognition, and speaker recognition use MFCC features to analyse audio signals.”
BackgroundDynamic modality weighting for multi-stream hmms inaudio-visual speech recognition
Gurban, M., Thiran, J., Thomas Drugman, A., & Dutoit, T. (2008) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1452392.1452442 · Cited by 48 · cited 1×
“There are many other applications of video processing that use and combine multiple modalities for audio-visual speech recognition [28], emotion recognition [40, 63], and language and vision tasks [17, 29].”
BackgroundZhu, L. L., & Beauchamp, M. S. (2017) · Journal of Neuroscience · doi:10.1523/jneurosci.2914-16.2017 · Cited by 47 · cited 2×
“[65] explore the relationship between visual and auditory human modalities.”
BackgroundContrast Enhancement Estimation for Digital Image Forensics
Wen, L., Qi, H., & Lyu, S. (2018) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3183518 · Cited by 42 · cited 1×
“Classical computer vision techniques [15, 60] have sufficed to tackle media manually manipulated by humans.”
BackgroundSanders, D. A., & Goodrich, S. J. (1971) · Journal of Speech and Hearing Research · doi:10.1044/jshr.1401.154 · Cited by 38 · cited 2×
“More specifically, [48, 51, 55, 65] suggest some positive correlation between audio-visual modalities, which have been exploited for multimodal perceived emotion recognition.”
BackgroundMultimedia Forensics: discovering the history of multimedia contents
Battiato, S., Giudice, O., & Paratore, A. (2016) · Proceedings of the 17th International Conference on Computer Systems and Technologies 2016 · doi:10.1145/2983468.2983470 · Cited by 37 · cited 2×
“Media forensics deals with the problem of authenticating the source of digital media such as images, videos, speech, and text in order to identify forgeries and other malicious intents [15].”
BackgroundLearning Unified Deep-Features for Multiple Forensic Tasks
Mayer, O., Bayar, B., & Stamm, M. C. (2018) · ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec) · doi:10.1145/3206004.3206022 · Cited by 37 · cited 1×
“Therefore, there has been a growing interest in developing deep learning-based methods for multimedia forensics [11, 20, 39].”
BackgroundEnhanced movie content similarity based on textual, auditory and visual information
Bougiatiotis, K., & Γιαννακόπουλος, Θ. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.11.050 · Cited by 35 · cited 1×
“Prior works [18, 19, 21] using audio or speech signals for various tasks like perceived emotion recognition, and speaker recognition use MFCC features to analyse audio signals.”
BackgroundWeb Video Verification using Contextual Cues
Papadopoulou, O., Zampoglou, M., Papadopoulos, S., & Kompatsiaris, I. Y. (2017) · doi:10.1145/3078897.3080535 · Cited by 34 · cited 1×
“Other than these modalities, multimodal approaches have also exploited the contextual information in video data to detect fakes [45].”
BackgroundChen, Y., Kang, X., Wang, Z. J., & Zhang, Q. (2018) · ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec) · doi:10.1145/3206004.3206013 · Cited by 32 · cited 1×
“Therefore, there has been a growing interest in developing deep learning-based methods for multimedia forensics [11, 20, 39].”
BackgroundJeon, H., Bang, Y., & Woo, S. S. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00163 · Cited by 31 · cited 1×
“[30] propose FakeTalkerDetect, which is a Siamese-based network to detect the fake videos generated from the neural talking head models.”
BackgroundThe Liar's Walk: Detecting Deception with Gait and GesturePreprint
Randhavane, T., Bhattacharya, U., Kapsaskis, K., Gray, K. J., Bera, A., & Manocha, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.06874 · Cited by 24 · cited 1×
“Ther has also been prior interest in exploring affect for detection deception [47].”
BackgroundSelf-citationVerdoliva, L., & Bestagini, P. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350542 · Cited by 13 · cited 1×
“Classical computer vision techniques are unsuccessful when identifying false media corrupted using deep learning [58].”
BackgroundAkputu, O. K., Seng, K. P., & Lee, Y. (2016) · IGI Global eBooks · doi:10.4018/978-1-5225-0159-6.ch033 · Cited by 5
Bahirat, K., Vaishnav, N., Sukumaran, S., & Prabhakaran, B. (2019) · ACM Multimedia Systems Conference (MMSys) · doi:10.1145/3304109.3325817 · Cited by 3 · cited 1×
“Therefore, there has been a growing interest in developing deep learning-based methods for multimedia forensics [11, 20, 39].”
Background
Research works printed but not linked: 10
- Kingsley Oryina Akputu , Kah Phooi Seng, and Yun Li Lee . 2013 . Facial emotion recognition for intelligent tutoring environment. In IMLCS. 9--13. Kingsley Oryina Akputu, Kah Phooi Seng, and Yun Li Lee. 2013. Facial emotion recognition for intelligent tutoring environment. In IMLCS. 9--13.
- Themis Balomenos , Amaryllis Raouzaiou , Spiros Ioannou , Athanasios Drosopoulos , Kostas Karpouzis , and Stefanos Kollias . 2004 . Emotion analysis in man-machine interaction systems . In International Workshop on Machine Learning for Multimodal Interaction. Springer, 318--328 . Themis Balomenos, Amaryllis Raouzaiou, Spiros Ioannou, Athanasios Drosopoulos, Kostas Karpouzis, and Stefanos Kollias. 2004. Emotion analysis in man-machine interaction systems. In International Workshop on Machine Learning for Multimodal Interaction. Springer, 318--328.
- Tadas Baltruvs aitis, Peter Robinson , and Louis-Philippe Morency . 2016 . Openface: an open source facial behavior analysis toolkit . In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1--10 . Tadas Baltruvs aitis, Peter Robinson, and Louis-Philippe Morency. 2016. Openface: an open source facial behavior analysis toolkit. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1--10.
- Uttaran Bhattacharya , Trisha Mittal , Rohan Chandra , Tanmay Randhavane , Aniket Bera , and Dinesh Manocha . 2019 . Step: Spatial temporal graph convolutional networks for emotion perception from gaits. arXiv preprint arXiv:1910.12906 (2019). Uttaran Bhattacharya, Trisha Mittal, Rohan Chandra, Tanmay Randhavane, Aniket Bera, and Dinesh Manocha. 2019. Step: Spatial temporal graph convolutional networks for emotion perception from gaits. arXiv preprint arXiv:1910.12906 (2019). · arXiv
- Vladimir Chernykh and Pavel Prikhodko . 2017. Emotion recognition from speech with recurrent neural networks. arXiv preprint arXiv:1701.08071 ( 2017 ). Vladimir Chernykh and Pavel Prikhodko. 2017. Emotion recognition from speech with recurrent neural networks. arXiv preprint arXiv:1701.08071 (2017). · arXiv
- R Benjamin Knapp , Jonghwa Kim , and Elisabeth André . 2011. Physiological signals and their use in augmenting emotion recognition for human--machine interaction . In Emotion-oriented systems . Springer , 133--159. R Benjamin Knapp, Jonghwa Kim, and Elisabeth André. 2011. Physiological signals and their use in augmenting emotion recognition for human--machine interaction. In Emotion-oriented systems . Springer, 133--159.
- Gangeshwar Krishnamurthy , Navonil Majumder , Soujanya Poria , and Erik Cambria . 2018. A deep learning approach for multimodal deception detection. arXiv preprint arXiv:1803.00344 ( 2018 ). Gangeshwar Krishnamurthy, Navonil Majumder, Soujanya Poria, and Erik Cambria. 2018. A deep learning approach for multimodal deception detection. arXiv preprint arXiv:1803.00344 (2018). · arXiv
- Yuezun Li , Xin Yang , Pu Sun , Honggang Qi , and Siwei Lyu . 2019 . Celeb-df: A new dataset for deepfake forensics. arXiv preprint arXiv:1909.12962 (2019). Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. 2019. Celeb-df: A new dataset for deepfake forensics. arXiv preprint arXiv:1909.12962 (2019). · arXiv
- Luisa Verdoliva and Paolo Bestagini . 2019 b . Multimedia Forensics. In Proceedings of the 27th ACM International Conference on Multimedia ( Nice, France) (MM '19). Association for Computing Machinery, New York, NY, USA, 2701--2702. https://doi.org/10.1145/3343031.3350542 10.1145/3343031.3350542 Luisa Verdoliva and Paolo Bestagini. 2019 b. Multimedia Forensics. In Proceedings of the 27th ACM International Conference on Multimedia (Nice, France) (MM '19). Association for Computing Machinery, New York, NY, USA, 2701--2702. https://doi.org/10.1145/3343031.3350542 · DOI
- Amir Zadeh , Paul Pu Liang , Navonil Mazumder , Soujanya Poria , Erik Cambria , and Louis-Philippe Morency . 2018 a. Memory fusion network for multi-view sequential learning . In Thirty-Second AAAI Conference on Artificial Intelligence . Amir Zadeh, Paul Pu Liang, Navonil Mazumder, Soujanya Poria, Erik Cambria, and Louis-Philippe Morency. 2018a. Memory fusion network for multi-view sequential learning. In Thirty-Second AAAI Conference on Artificial Intelligence .
Cited by
Deepfake Detection: A Systematic Literature Review
Rana, M. S., Nobi, M. N., Murali, B., & Sung, A. H. (2022) · IEEE Access · doi:10.1109/access.2022.3154404 · Cited by 573
Masood, M., Nawaz, M., Malik, K. M., Javed, A., Irtaza, A., & Malik, H. (2023) · Applied Intelligence · doi:10.1007/s10489-022-03766-z · Cited by 543
Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection
Haliassos, A., Vougioukas, K., Petridis, S., & Pantic, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00500 · Cited by 537
“datasets [46, 38, 30, 16], it is possible to train deep convolutional neural networks (CNNs) to detect fake videos [3, 25, 64, 4, 14, 45, 42, 36].”
BackgroundDeepfake detection using deep learning methods: A systematic and comprehensive review
Heidari, A., Jafari Navimipour, N., Dag, H., & Unal, M. (2024) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1520 · Cited by 349
“In addition, Mittal et al. (2020) demonstrated a learning-based technique for distinguishing between genuine and phony deepfake multimedia content.”
BackgroundExploring Temporal Coherence for More General Video Face Forgery Detection
Zheng, Y., Bao, J., Chen, D., Zeng, M., & Wen, F. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01477 · Cited by 320
“Unlike previous works, they either directly apply 3D convolution network [15, 21] on video or detect a specific kind of incoherence, such as irregular eye blinking [33], lip motions [22] or emotion [37].”
Background
Metadata
- Authors
- Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., & Manocha, D.
- Affiliations
- University of Maryland, College Park (US)
- Venue
- Proceedings of the 28th ACM International Conference on Multimedia
- Publisher
- ACM
- Published
- 2020-10-12
- Pages
- 2823-2832
- OpenAlex
- W3093010840
- Cited by
- 319
- References
- 49
- Citations by year
- 2026: 48 · 2025: 81 · 2024: 48 · 2023: 61 · 2022: 45 · 2021: 34 · 2020: 2
- FWCI
- 10.95
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on Multimedia (ACM MM) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Keywords
- deepfake detection, multimodal fusion, emotional cues, perceived emotion, Siamese network, triplet loss, DFDC
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.