A Robust Approach to Multimodal Deepfake Detection
Open accessThe DOI matches its Crossref record; 53 of the 54 references it cites were found in a registry or an index; 10 of the 23 sources its claims cite were checked for support.
Abstract
The widespread use of deep learning techniques for creating realistic synthetic media, commonly known as deepfakes, poses a significant threat to individuals, organizations, and society. As the malicious use of these data could lead to unpleasant situations, it is becoming crucial to distinguish between authentic and fake media. Nonetheless, though deepfake generation systems can create convincing images and audio, they may struggle to maintain consistency across different data modalities, such as producing a realistic video sequence where both visual frames and speech are fake and consistent one with the other. Moreover, these systems may not accurately reproduce semantic and timely accurate aspects. All these elements can be exploited to perform a robust detection of fake content. In this paper, we propose a novel approach for detecting deepfake video sequences by leveraging data multimodality. Our method extracts audio-visual features from the input video over time and analyzes them using time-aware neural networks. We exploit both the video and audio modalities to leverage the inconsistencies between and within them, enhancing the final detection performance. The peculiarity of the proposed method is that we never train on multimodal deepfake data, but on disjoint monomodal datasets which contain visual-only or audio-only deepfakes. This frees us from leveraging multimodal datasets during training, which is desirable given their lack in the literature. Moreover, at test time, it allows to evaluate the robustness of our proposed detector on unseen multimodal deepfakes. We test different fusion techniques between data modalities and investigate which one leads to more robust predictions by the developed detectors. Our results indicate that a multimodal approach is more effective than a monomodal one, even if trained on disjoint monomodal datasets.
Overview
CoChat Summary
TL;DR
The paper proposes a multimodal deepfake video detector that fuses visual and audio features using time-aware neural networks, trained only on disjoint monomodal deepfake datasets. It reports that Early Fusion is the most robust fusion strategy and that multimodal detection outperforms monomodal baselines on unseen multimodal datasets.
Key findings
- Preliminary monomodal detectors achieved AUC 0.91 and BA0.5 0.83 for visual-only, and AUC 0.96 and BA0.5 0.90 for audio-only.
- Early Fusion was the most effective fusion strategy on average, achieving AUCs ≥ 0.90 for two of three datasets and being best on the remaining dataset; it exceeded other fusion strategies by 7% and 10% on FakeAVceleb and DFDC, respectively, while being competitive on TIMIT.
- The multimodal approach consistently outperformed the monomodal detectors, supporting the benefit of multimodal analysis.
- In mixed-class experiments, the Early Fusion detector achieved best average results when trained with the All Class strategy: average AUC = 0.80 vs. average AUC = 0.76 for Same Class. A fixed threshold t = 0.5 did not considerably change BA values, though FakeAVceleb showed a 5% accuracy improvement when using the best threshold for DEF; Early Fusion on FakeAVceleb was unbalanced toward “Real” (TPR0.5 = 0.954 vs. TNR0.5 = 0.580).
Methods
- Two-stage pipeline: visual features extracted with an EfficientNetB4 ensemble modified per prior work, and audio features extracted as TDNN x-vectors via SpeechBrain; extraction at 10 Hz, with 100 ms non-overlapped audio windows, time window TW = 3.0 s, N = 30 time instants, MV = 1072 visual features, MA = 512 audio features. The detector uses a Transformer-based classifier with a single hidden layer, 8 attention heads, 0.1 dropout, and GELU, evaluated with Late, Mid, and Early Fusion strategies.
- Training data: FaceForensics++ (5000 videos, QP 23 and QP 40 merged) for visual-only training, and ASVspoof 2019 Logical Access (>120,000 audio tracks at 16 kHz) for audio-only training. Testing was cross-dataset on FakeAVCeleb (~20,000 deepfake videos from 500 real VoxCeleb2 videos), DFDC (12,547 samples from folders 45–49), and TIMIT (VidTIMIT + DeepfakeTIMIT + TIMIT-TTS DTW subset with almost 20,000 synthetic speech tracks).
- Training and analysis: 150 epochs with early stopping patience 15, weighted cross-entropy loss, Adam optimization, learning rate e-3, weight decay e-4, and learning-rate reduction on plateau by factor 0.1; class balancing by oversampling the less represented class. Evaluation used ROC curves, AUC, and Balanced Accuracy at threshold t = 0.5 and at the best threshold.
Limitations
- The paper states that multimodal deepfake datasets are lacking, and that training on monomodal datasets means the detectors cannot leverage all intra-modality relationships within the content. It also states the system cannot detect synthetic content that appears realistic in individual modalities but lacks audio-video synchronization, even if simpler detectors trained explicitly for this could spot such inconsistencies.
- The paper notes that training on monomodal datasets could impact performance on multimodal ones, and that poor AUC on DFDC is likely due to distinct characteristics between training and test data.
- Evident from the text: evaluation is limited to three multimodal test corpora (FakeAVCeleb, DFDC, TIMIT) without multimodal training data; fixed threshold t = 0.5 yields unbalanced TPR/TNR on FakeAVCeleb, though best-threshold selection improves accuracy by 5% for DEF. The recursive Late Fusion approach was not considered further due to limited novelty and computational overhead.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures










Verification
Claims the paper makes
6 Valid · 0 Not valid · 6 Not checked · 29 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Moreover, the results of [38] show that an ensemble of audio and visual baselines outperforms monomodal counterparts.”
§ A Robust Approach to Multimodal Deepfake Detection > 2. Deepfake Detection > 2.3. Audio-Visual Deepfake Detection
What the sources say
- Supportedcites [38] Khalid, H., Kim, M., Tariq, S., and Woo, S.S. (2021, January 24). Evaluation of an audio-video multimodal deepfake dataset using unimodal and multimodal detectors. Proceedings of the 1st Workshop on Synthetic Multimedia-Audiovisual Deepfake Generation and Detection, Virtual Event. source
The Abstract (p. 1) says that audio- or video-only unimodal methods do not perform well compared with ensemble-based methods. Section 4.2.2 (p. 5) identifies those ensembles as combinations of one audio and one video classifier, matching the claim. (LLM, conf=0.91)
states a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In the recent years, few multimodal datasets have been proposed, containing both counterfeited video and audio tracks.”
- Also stated in: § 2.3. Audio-Visual Deepfake Detection · plain
“These are DFDC [42], FakeAVCeleb [43], and DeepfakeTIMIT [44] with TIMIT-TTS [6].”
§ A Robust Approach to Multimodal Deepfake Detection > 2. Deepfake Detection > 2.3. Audio-Visual Deepfake Detection
What the sources say
- Partly supportedcites [42] Dolhansky, B., Bitton, J., Pflaum, B., Lu, J., Howes, R., Wang, M., and Ferrer, C.C. (2020). The deepfake detection challenge (DFDC) dataset. arXiv. sourceExperimental — accuracy not yet measured
The Introduction (p. 1) and Dataset Construction (p. 2) identify DFDC as a dataset of videos with manipulated faces, supporting its identification as a counterfeited-video dataset. They describe face swaps but do not establish that the audio tracks are also counterfeited. This source also does not establish the claim about FakeAVCeleb or DeepFakeTIMIT with TIMIT-TTS. (LLM, conf=0.84)
- Partly supportedcites [43] Khalid, H., Tariq, S., Kim, M., and Woo, S.S. (2021, January 6–14). FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset. Proceedings of the Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), Virtual. sourceExperimental — accuracy not yet measured
The Introduction (p. 2) identifies DFDC as a dataset containing deepfake video and synthesized audio, and describes FakeAVCeleb as containing both. That supports the citing sentence’s general point that such datasets were few. But Background and Motivation (p. 3) says Deepfake TIMIT’s audio was real, not fake; our copy does not identify a TIMIT-TTS variant, so the claim’s three-dataset list is not fully supported. (LLM, conf=0.86)
- Partly supportedcites [44] Korshunov, P., and Marcel, S. (2018). Deepfakes: A new threat to face recognition? Assessment and detection. arXiv. sourceExperimental — accuracy not yet measured
Our copy supports scarcity only at a different level: the Introduction (p. 1) presents 'a first publicly available database of videos where faces are swapped' and Related Work (p. 2) states 'no public Deepfake video database where GAN-based approach was applied is available' — i.e., few/none Deepfake VIDEO databases, not multimodal ones with counterfeited audio. The part the source does not support is the audio modality: the Introduction (p. 1) explicitly says 'For the audio, we kept the original audio track of each video, i.e., no manipulation was done to the audio channel', so its own DeepfakeTIMIT dataset is not a counterfeited-video-and-audio dataset, and the paper nowhere discusses the scarcity of such multimodal datasets (DFDC and FakeAVCeleb are not mentioned). (LLM, conf=0.68)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 2.3. Audio-Visual Deepfake Detection · plain
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“To overcome these limitations, multimodal approaches have been recently proposed, able to combine information from various domains to enhance the accuracy of the detection process [7,8].”
§ A Robust Approach to Multimodal Deepfake Detection > 1. Introduction
What the sources say
- Supportedcites [7] Agarwal, S., Hu, L., Ng, E., Darrell, T., Li, H., and Rohrbach, A. (2023, January 2–7). Watch those words: Video falsification detection using word-conditioned facial motion. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA. source
The Related Work section (p. 3) says multimodal techniques use both audio and visual signals to detect deepfakes. The Discussion (p. 8) describes the paper’s own multimodal approach, combining spoken words with facial gestures, and reports that it outperformed other methods in most tested cases. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [8] Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., and Manocha, D. (2020, January 12–16). Emotions don’t lie: An audio-visual deepfake detection method using affective cues. Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA. source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“These results are consistent with those of many cutting-edge detectors reported in the literature [20,34], indicating that the proposed monomodal classifiers are suitable for subsequent multimodal experiments.”
§ A Robust Approach to Multimodal Deepfake Detection > 5. Results > 5.2. Monomodal Results
What the sources say
- Supportedcites [20] Bonettini, N., Cannas, E.D., Mandelli, S., Bondi, L., Bestagini, P., and Tubaro, S. (2021, January 10–15). Video face manipulation detection through ensemble of CNNs. Proceedings of the International Conference on Pattern Recognition (ICPR), Virtual Event. source
The citation supports the comparison with published face-manipulation detectors, not the citing authors’ inference about their later multimodal experiments. The Introduction (p. 2) reports that the proposed CNN ensemble outperformed an FF++ baseline on two datasets, and the Results (p. 6) compare detector performance using AUC and LogLoss. (LLM, conf=0.86)
- Not checked — no full text of the sourcecites [34] Attorresi, L., Salvi, D., Borrelli, C., Bestagini, P., and Tubaro, S. (2022, January 21–25). Combining Automatic Speaker Verification and Prosody Analysis for Synthetic Speech Detection. Proceedings of the International Conference on Pattern Recognition, Montréal, QC, Canada. source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The DEF model achieves the best average results when trained following the All Class strategy (average AUC = 0.80 vs. average AUC = 0.76 of the Same Class).”
§ A Robust Approach to Multimodal Deepfake Detection > 5. Results > 5.3. Multimodal Results
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
18 more claims
Create a free account to see all 41 claims and to verify the 2 references still open.
- It may state two separate claims as one. 15
- The quote only partly states this claim. 1
- Probably not a substantial claim. 2
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Robust Approach to Multimodal Deepfake Detection > 5. Results > 5.3. Multimodal Results
“Early Fusion results on FakeAVceleb are strongly unbalanced towards the “Real” class (i.e., TPR0.5=0.954 vs. TNR0.5=0.580).”
§ A Robust Approach to Multimodal Deepfake Detection > 5. Results > 5.1. Evaluation Metrics
“If the likelihood exceeds the threshold, the sequence is classified as “Fake”, otherwise it is classified as “Real”.”
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072 · cited 1×
“Regarding the visual modality, we exploit the EfficientNetB4 [45] network modified following the implementation proposed in [20], which investigates the ensembling of differently trained CNNs making use of attention layers and siamese training.”
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 2×
“Training Datasets FaceForensics++ [18].”
X-Vectors: Robust DNN Embeddings for Speaker Recognition
Snyder, D., Garcia-Romero, D., Sell, G., Povey, D., & Khudanpur, S. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461375 · Cited by 2,620
VoxCeleb2: Deep Speaker Recognition
Chung, J. S., Nagrani, A., & Zisserman, A. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-1929 · Cited by 2,256
VoxCeleb: A Large-Scale Speaker Identification Dataset
Nagrani, A., Chung, J. S., & Zisserman, A. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-950 · Cited by 2,210 · cited 2×
“In particular, FV was trained on FaceForensics++, while FA was trained on Voxceleb [57] and Voxceleb2 [50] datasets, considering audio data sampled at 16 kHz.”
MesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Deepfake Video Detection Using Recurrent Neural Networks
Guera, D., & Delp, E. J. (2018) · 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2018.8639163 · Cited by 1,285
DARPA TIMIT: acoustic-phonetic continuous speech corpus CD-ROM, NIST speech disc 1-1.1
Garofolo, J. S., Lamel, L. F., Fisher, W. M., Fiscus, J. G., Pallett, D. S., & Dahlgren, N. L. (1993) · doi:10.6028/nist.ir.4930 · Cited by 1,267
Exposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196
Research works printed but not linked: 1
- Koopman, M., Rodriguez, A.M., and Geradts, Z. (2018, January 29–31). Detection of deepfake video manipulation. Proceedings of the 20th Irish Machine Vision and Image Processing Conference (IMVIP), Belfast, UK.
Cited by
Abbas, F., & Taeihagh, A. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124260 · Cited by 184
Deepfake Media Forensics: Status and Future Challenges
Amerini, I., Barni, M., Battiato, S., Bestagini, P., Boato, G., Bruni, V., Caldelli, R., De Natale, F., De Nicola, R., Guarnera, L., et al. (2025) · Journal of Imaging · doi:10.3390/jimaging11030073 · Cited by 113
Self-citationA novel approach for detecting deep fake videos using graph neural network
El-Gayar, M. M., Abouhawwash, M., Askar, S., & Sweidan, S. (2024) · Journal Of Big Data · doi:10.1186/s40537-024-00884-y · Cited by 82
Deepfake video detection methods, approaches, and challenges
Alrashoud, M. (2025) · Alexandria Engineering Journal · doi:10.1016/j.aej.2025.04.007 · Cited by 49
Audio–Visual Synchronization and Lip Movement Analysis for Real-Time Deepfake Detection
Javed, M., Zhang, Z., Dahri, F. H., Laghari, A. A., Krajčík, M., & Almadhor, A. S. (2025) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-025-00911-7 · Cited by 62
Kumar, A., Singh, D., Jain, R., Jain, D. K., Gan, C., & Zhao, X. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102993 · Cited by 58
Video and Audio Deepfake Datasets and Open Issues in Deepfake Technology: Being Ahead of the Curve
Akhtar, Z., Pendyala, T. L., & Athmakuri, V. S. (2024) · Forensic Sciences · doi:10.3390/forensicsci4030021 · Cited by 60
A Review of Deep Learning Techniques for Multimodal Fake News and Harmful Languages Detection
Ayetiran, E. F., & Özgöbek, Ö. (2024) · IEEE Access · doi:10.1109/access.2024.3406258 · Cited by 40
Temporal Feature Prediction in Audio–Visual Deepfake Detection
Gao, Y., Wang, X., Zhang, Y. L., Zeng, P., & Ma, Y. (2024) · Electronics · doi:10.3390/electronics13173433 · Cited by 37
A defensive attention mechanism to detect deepfake content across multiple modalities
Asha, S., Vinod, P., & Menon, V. G. (2024) · Multimedia Systems · doi:10.1007/s00530-023-01248-x · Cited by 16
Wani, T. M., Qadri, S. A. A., Wani, F. A., & Amerini, I. (2024) · Foundations and Trends® in Privacy and Security · doi:10.1561/3300000048 · Cited by 16
A review of deep learning based multimodal forgery detection for video and audio
Tan, D., Yang, Y., Niu, C., Li, S., Yang, D., & Tan, B. (2025) · Discover Applied Sciences · doi:10.1007/s42452-025-07629-3 · Cited by 23
Leveraging Mixture of Experts for Improved Speech Deepfake Detection
Negroni, V., Salvi, D., Mezza, A. I., Bestagini, P., & Tubaro, S. (2025) · ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49660.2025.10890398 · Cited by 14
“As a second baseline, we consider the average ensemble as a simple, input-independent approach for combining scores from multiple experts [20], making it a particularly suitable baseline to compare with the proposed MoE systems.”
MethodSelf-citationDetecting Deepfake Audio Using Spectrogram-Based Machine Learning Approaches
Bohara, R., & Bairwa, A. K. (2025) · IEEE Access · doi:10.1109/access.2025.3602531 · Cited by 21
Deepfake Detection in Video and Audio Clips: A Comprehensive Survey and Analysis
Jbara, W. A., Hussein, N. A.-H. K., & Soud, J. H. (2024) · Mesopotamian Journal of CyberSecurity · doi:10.58496/mjcs/2024/025 · Cited by 21
Deepfake Media Forensics: State of the Art and Challenges Ahead
Amerini, I., Barni, M., Battiato, S., Bestagini, P., Boato, G., Bonaventura, T. S., Bruni, V., Caldelli, R., Natale, F. G. B. D., Nicola, R. D., et al. (2025) · Lecture notes in social networks · doi:10.1007/978-3-031-85386-9_3 · Cited by 46
Self-citationAlrowais, F., Hassan, A. A., ALMUKADI, W. S., Alanazi, M. H., Marzouk, R., & Mahmud, A. (2024) · IEEE Access · doi:10.1109/access.2024.3470128 · Cited by 21
Deepfake Detection Method Based on Spatio-Temporal Information Fusion
Wang, X., Song, W., Hao, C., & Liu, F. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.062922 · Cited by 21
Cyber Security Focused Deepfake Detection System Using Big Data
Kumar, N., & Kundu, A. (2024) · SN Computer Science · doi:10.1007/s42979-024-03105-8 · Cited by 19
Harmonizing Image Forgery Detection & Localization: Fusion of Complementary Approaches
Mareen, H., Neve, L. D., Lambert, P., & Wallendael, G. V. (2023) · Journal of Imaging · doi:10.3390/jimaging10010004 · Cited by 13
Raza, A., Basit, A., Amin, A., Arfeen, Z. A., Masud, M. I., Fayyaz, U., & Jumani, T. A. (2026) · AI · doi:10.3390/ai7020068 · Cited by 22
Water Leak Detection and Classification With Complex-Valued Neural Networks and Sensor Fusion
Leonzio, D. U., Mandelli, S., Bestagini, P., Marcon, M., & Tubaro, S. (2025) · IEEE Access · doi:10.1109/access.2025.3585306 · Cited by 7
Self-citationA Bibliometric Analysis of Deepfakes : Trends, Applications and Challenges
Garg, D., & Gill, R. (2024) · ICST Transactions on Scalable Information Systems · doi:10.4108/eetsis.4883 · Cited by 10
Deepfake Detection Using Multi-Modal Fusion Combined with Attention Mechanism
Shirley, C. P., Jingle, B. J., Abisha, M. B., Venkatesan, R., Yashvanth Ram, R., V, & Elango, E. (2024) · International Conference on Sustainable Expert Systems (ICSES) · doi:10.1109/icses63445.2024.10763221 · Cited by 9
Improving synthetic media generation and detection using generative adversarial networks
Zia, R., Rehman, M., Hussain, A., Nazeer, S., & Anjum, M. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2181 · Cited by 9
Artificial intelligence: Emergence and possible fraudulent use in medical publishing
Jacques, T., Sleiman, R., Diaz, M. I., & Dаrtus, J. (2023) · Orthopaedics & Traumatology Surgery & Research · doi:10.1016/j.otsr.2023.103709 · Cited by 7
Nafea, R. A., Nissirat, L. A., & Al‐Samawi, A. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.1991 · Cited by 8
Lingham, N. S. R., Devakanth, J. J. M. A., Raj, G., Gayathri, K., Janani, R., & Dhanapal, R. (2024) · International Conference on Applied Artificial Intelligence and Computing (ICAAIC) · doi:10.1109/icaaic60222.2024.10575155 · Cited by 4
Sandhya, & Kashyap, A. (2024) · Journal of Forensic Sciences · doi:10.1111/1556-4029.15665 · Cited by 7
Robust deepfake detection using multi-scale feature fusion
Yogarajan, G., Soundhariya, S., & Harini, R. (2025) · Multimedia Tools and Applications · doi:10.1007/s11042-025-20768-4 · Cited by 4
Multimodal Forgery Detection in Videos Using a Tri-Network Model
Chandran, P. P., Sriram, S., & Babu, C. (2024) · International Conference on Advanced Computing and Communication Systems (ICACCS) · doi:10.1109/icaccs60874.2024.10717044 · Cited by 3
Humans Versus Machines: A Deepfake Detection Faceoff
Goh, D. H., Pan, J., & Lee, C. S. (2024) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.1139 · Cited by 6
A Comprehensive Review of Machine Learning Techniques for DeepFake Detection
Kabir, M. H., Noman, A. A., Afiq, A. A., Jaki, H., Hasan, M. N., Ahmad, Islam, M. S., Hasan, H., & Mojumder, I. A. (2025) · Applied Computational Intelligence and Soft Computing · doi:10.1155/acis/8883527 · Cited by 3
Toward comprehensive short utterances manipulations detection in videos
Moufidi, A., Rousseau, D., & Rasti, P. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-20284-x · Cited by 3
Ensemble-Based Biometric Verification: Defending Against Multi-Strategy Deepfake Image Generation
Zen, H., Wagh, R., Wanderley, M., Bicalho, G. C., Park, R. Y., Sun, M., Palacios, R., Carvalho, L. A. D. R., Rinaldo, G., & Gupta, A. D. (2025) · Computers · doi:10.3390/computers14060225 · Cited by 6
All-for-One and One-for-All: Deep Learning-Based Feature Fusion for Synthetic Speech Detection
Mari, D., Salvi, D., Bestagini, P., & Milani, S. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-74627-7_39 · Cited by 5
Self-citationTurós, M., Kenyeres, A. Z., & Szűts, Z. (2024) · Telematics and Informatics Reports · doi:10.1016/j.teler.2024.100160 · Cited by 8
A Review of Deep Learning Techniques for Multimodal Fake News and Harmful Languages DetectionPreprint
Ayetiran, E. F., & Özgöbek, Ö. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4691091 · Cited by 2
Lakhchaura, R., Madan, P., & Vishnoi, A. (2025) · International Conference on Computing, Communication and Automation · doi:10.1109/iccca66364.2025.11325599 · Cited by 3
Jabbar, N., Bhatti, S. M., Rashid, M., Jaffar, A., & Akram, S. (2025) · PLoS ONE · doi:10.1371/journal.pone.0326565 · Cited by 5
Intelligence artificielle : émergence et possible utilisation frauduleuse dans l’édition médicale
Jacques, T., Sleiman, R., Diaz, M. I., & Dаrtus, J. (2023) · Revue de Chirurgie Orthopédique et Traumatologique · doi:10.1016/j.rcot.2023.10.010 · Cited by 1
Deepfakes: A comprehensive survey on techniques, challenges and future directions
Usmani, S., Kumar, S., & Sadhya, D. (2026) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2026.111210 · Cited by 3
S., C., Surendran, R., & Raveena, S. (2025) · International Conference on Augmented Intelligence and Sustainable Systems (ICAISS) · doi:10.1109/icaiss61471.2025.11041754 · Cited by 2
Neuro-Behavioral Fingerprinting: A Multimodal Behavioral Approach to Deepfake Detection
Sheela, K. S., Priyadarshan, A. A., Irfan, J. M., & Tharun, V. V. (2025) · International Conference on Applied Artificial Intelligence and Computing (ICAAIC) · doi:10.1109/icaaic64647.2025.11331135 · Cited by 1
Fitz-Gerald, A., & Padalko, H. (2026) · Frontiers of artificial intelligence, ethics and multidisciplinary applications. · doi:10.1007/978-3-032-05588-0_11 · Cited by 1
Understanding Seniors’ Strategies for Identifying Deepfakes
Tang, Z., Goh, D. H., Lee, C. S., & Yang, Y. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-61947-2_26 · Cited by 4
LADLE-MM: Limited Annotation based Detector with Learned Ensembles for Multimodal MisinformationPreprint
Cardullo, D., Teglia, S., & Amerini, I. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5956871
Enhancing Deep Fake Image Generation and Detection Through Transfer Learning
Bhalgami, R., Bhut, V., Ardeshna, S., Patel, T., & Chokshi, R. (2026) · Lecture notes in networks and systems · doi:10.1007/978-981-95-0701-6_25
Güncel Deepfake Video Algılama Yöntemleri Üzerine Bir Literatür İncelemesi
Aydın, S., & Özen, Z. (2024) · Bilgisayar bilimleri ve mühendisliği dergisi. · doi:10.54525/bbmd.1460699
Deepfake detection with interpretability analysis using optimized algorithm
John, J., Bhave, A. M., Karwatkar, D. G., Gomase, R. P., & Patil, S. (2026) · The Visual Computer · doi:10.1007/s00371-026-04379-0
Multimodal Deepfake Detection: A Scalable AI Pipeline Combining Audio-Visual Analysis
Bhattacharya, D., Dutta, S. M., Mondal, I., Ghosh, D., Das, A. K., & Bhattacharyya, S. (2025) · IEEE India Conference · doi:10.1109/indicon68490.2025.11392906
Deepfake Generation and Detection: A Comprehensive Survey
Sharma, A., Zala, B., Jatan, S. A., Nahar, T., Dubey, V. G., & Parmar, C. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-19300-1_7
Data-Driven Techniques for Speech and Multimodal Deepfake Detection
Salvi, D. (2026) · SpringerBriefs in applied sciences and technology · doi:10.1007/978-3-032-12359-6_11
Self-citationGranulated Ensemble CNN for Multi-object Deepfake Detection and Analysis
Saritha, K., & Tripathy, A. K. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-10940-8_22
Interpretable Multimodal Deep Fake Detection with Cross-Attention Based Fusion and Reasoning
Abinaya, A., Uma, E., Dharsana Dharani, V. G., & Dharanika, S. (2026) · International Conference on Innovations in Cybersecurity and Data Science (ICICDS) · doi:10.1109/icicds70526.2026.11604598
Patra, R. K., Dalui, A., Singh, S., Dwivedi, P., & Nixon, J. S. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-19678-1_21
Constrained Maximum Likelihood Gaussian Score Fusion for Multimodal Deepfake Detection
Khorrami, P., Borgström, B. J., Deng, Y., Dagli, C. K., & Torres‐Carrasquillo, P. A. (2025) · International Workshop on Biometrics and Forensics (IWBF) · doi:10.1109/iwbf63717.2025.11113422
EfficientNet and Vision Transformer Fusion for High-Precision Deepfake Image Identification
Talasila, S., Gurrala, V. K., Matta, V., Musthyala, M., & Balla, S. (2025) · International Conference on Mobile Networks and Wireless Communications (ICMNWC) · doi:10.1109/icmnwc66779.2025.11354178
A Survey on Hybrid GAN-Transformer Models for Image Forgery Detection
Majeed, A. H., Abdul Jabbar, P., & Pushpalatha, K. P. (2026) · International Conference on Computing and Emerging Sciences · doi:10.1145/3797491.3797519
Deepfake detection: A review of audio, visual and multimodal techniques
Kanwal, R., Ashraf, H., Jhanjhi, N. Z., & Mirza, Q. U. A. Z. Z. (2026) · Applied Soft Computing · doi:10.1016/j.asoc.2026.115845
Anjanesh, D., & Sughasiny, M. (2025) · International Conference on Communication, Computer, and Information Technology (IC3IT) · doi:10.1109/ic3it66137.2025.11341241 · Cited by 1
Wtmsa-net: Wavelet transform-based multi-stage attention network for deepfake detection
Chen, Z., Wang, L., Shi, H., Zhang, S., & Liu, H. (2026) · Pattern Analysis and Applications · doi:10.1007/s10044-026-01752-6
Self-citationVysotska, V., Chyrun, L., Лаврут, О., Lavrut, T., & Romavchyk, R. (2026) · Системи управління навігації та зв’язку Збірник наукових праць · doi:10.26906/sunz.2026.2.052
FrEnDTM: Digital Forensic Investigation Using Frequency-Enhanced Attention-Based Deep Learning Model
Kamble, D., & Bartere, M. M. (2026) · SN Computer Science · doi:10.1007/s42979-026-04803-1 · Cited by 1
Diqi, M., Hiswati, M. E., & Setyoningrum, N. G. (2025) · International Journal of Information Technology · doi:10.1007/s41870-025-02988-x
Enhancing Deepfake Detection with Audio Spectrograms and Siamese Networks
Şenavcu, B., Özdemir, S. İ., Çetin, N. C., Gürbüzerol, İ., & Eminağaoğlu, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-11402-0_22
SwinGAN: A Synergistic Approach Combining Swin Transformers and GANs for DeepFake Detection
Kumar, V., Chaudhary, P., Singh, A. K., Srivastava, A., Diswar, P., & Singh, H. (2025) · International Conference on Multidisciplinary Research and Innovations in Engineering (MRIE) · doi:10.1109/mrie66930.2025.11156622
Shah, K., Shah, S., & Ajwalia, M. (2025) · International Conference on IoT Based Control Networks and Intelligent Systems (ICICNIS) · doi:10.1109/icicnis66685.2025.11315573
Hybrid Digital Forensic Framework for Multimodal Deepfake DetectionPreprint
Qureshi, S. M., Saeed, A., Ahmad, F., Ghamdi, M. A. A., & Alsolbi, I. N. (2026) · Research Square · doi:10.21203/rs.3.rs-9312223/v1
FakeTrace: Enhancing Deepfake Detection through Mamba Architecture and Grad-CAM Interpretability
Priyadarsini, S., Mohanty, S., Malik, R. R., & Acharya, &. A. A. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.06.302
Cardullo, D., Teglia, S., & Amerini, I. (2026) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2026.104929 · Cited by 1
MSTM: Multidimensional spatio-temporal analysis for enhanced facial video deepfake detection
Wang, H., He, H., Hong, M., & Sheng, H. (2026) · Signal Processing Image Communication · doi:10.1016/j.image.2026.117704
Prakash, P., & Jaffino, G. (2026) · Engineering Research Express · doi:10.1088/2631-8695/ae6c97
EchoFence: Non-Intrusive Forgery Detection in Video Conferencing via Ultrasonic Sensing
Zhao, L., Shi, L., Liu, J., Xiao, R., & Han, J. (2026) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom59046.2026.11571188
Systematic Review: Real-time Deepfake Detection Integrating Blockchain
Ali, A. B. M. S., Musa, A. S., Sirlantzis, K., & Turner, S. (2025) · International Conference on Future Internet of Things and Cloud (FiCloud) · doi:10.1109/ficloud66139.2025.00073
Deepfake Detection and ToS Risk Analysis Using NLP: A Multimodal Framework
Ülkü, D., & Emek, S. (2025) · International Artificial Intelligence and Data Processing Symposium (IDAP) · doi:10.1109/idap68205.2025.11222288
Compressed-Domain Spatio-Temporal Forensics for AI-Generated Video Detection
Mu, W.-P., Han, X., Cui, H., Xu, Q., Ba, Z.-J., & Sun, T.-F. (2026) · 2026 IEEE 3rd International Conference on Computer Vision and Deep Learning (DLCV) · doi:10.1109/dlcv69906.2026.11635513
HIL-MDF: A Human-in-the-Loop Multimodal Deepfake Detection System
El-Beheiry, H., Asmah, A., Zaher, O., Abdulwagid, M., & Moussa, S. M. (2026) · IEEE International Conference on Smart Sustainable Systems for Computer and Engineering Applications (3SCEA) · doi:10.1109/3scea68071.2026.11603194
Towards Tamper-Resistant Digital Media: An AI and Blockchain Approach for Deepfake Detection
Sakthivel, K., Harine, N., & Chandra Mohan, G. (2025) · International Conference on Electronics, Communication and Aerospace Technology (ICECA) · doi:10.1109/iceca66444.2025.11382545 · Cited by 1
Yeredla, D., Archana, S., Rangu, V., Shirisha, R., & Varun, V. S. (2026) · IEEE Global Symposium on Emerging and Communication Technologies (GSEACT) · doi:10.1109/gseact68539.2026.11620577
Multi-Headed Attention-based Parallel Convolution Neural Network for Deepfake Image Detection
Shukla, H., & Rathod, D. I. (2026) · International Conference on Innovations in Computational Intelligence (ICICI) · doi:10.1109/icici68867.2026.11564880
Ranjan, V., & Singh, D. (2025) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194025500676
Cross-modal deepfake detection: integrating textual and frequency domains
Chen, J., Che, X., Xu, K., Hu, X., & Li, Q. (2026) · Journal of Cloud Computing Advances Systems and Applications · doi:10.1186/s13677-026-00898-2 · Cited by 1
Deepfake Detection Through Multimodal Feature Fusion and Adaptive Security Frameworks
Yousif, A. H. A., Merzah, H. H., & Кудренко, С. О. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-39711-9_4
Şafak, E., & Barışçı, N. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6384978
Multimodal Deepfakes Analysis using Large Language Models (LLM’s)
Zahoor, K., & Bawany, N. Z. (2025) · International Conference on Innovative Computing (ICIC) · doi:10.1109/icic68258.2025.11413000
Deep-Fakes Detection in Media Forensics Using Deep Learning and Fusion Methods
Deshmukh, Y., Haque, M. J., & Bhaladhare, P. (2025) · International Conference for Advancement in Technology (ICONAT) · doi:10.1109/iconat66879.2025.11362482
GAN-Based Detection of Synthetically Generated Satellite Imagery
Anand, V., & Rodriguez, I. (2025) · IEEE MIT Undergraduate Research Technology Conference (URTC) · doi:10.1109/urtc68753.2025.11533044
Kaunang, E. S., Mantoro, T., & Ayu, M. A. (2024) · IEEE International Conference on Computing, Engineering and Design (ICCED) · doi:10.1109/icced64257.2024.10983596
Metadata
- Authors
- Salvi, D., Liu, H., Mandelli, S., Bestagini, P., Zhou, W., Zhang, W., & Tubaro, S.
- Affiliations
- Politecnico di Milano (IT)
- University of Science and Technology of China (CN)
- Corresponding author
- Salvi, Davide
- Bestagini, Paolo
- Venue
- Journal of Imaging
- Publisher
- MDPI AG
- Published
- 2023-06-19
- Volume
- 9
- Issue
- 6
- Pages
- 122
- ISSN
- 2313-433X (ISSN-L)
- PMC
- PMC10299653
- OpenAlex
- W4381140704
- Cited by
- 96
- References
- 54
- Citations by year
- 2026: 34 · 2025: 35 · 2024: 24 · 2023: 3
- FWCI
- 8.68
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Journal of Imaging (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Virtual Community of Pathological Anatomy (University of Castilla La Mancha) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Defense Advanced Research Projects Agency (FA8750-20-2-1004, PE00000001)
- European Union (FA8750-20-2-1004, PE00000001)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- deepfake detection, multimodal fusion, audio-visual consistency, synthetic media, fusion techniques
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.