Combining EfficientNet and Vision Transformers for Video Deepfake Detection
Open accessThe DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
Deepfakes are the result of digital manipulation to forge realistic yet fake imagery. With the astonishing advances in deep generative models, fake images or videos are nowadays obtained using variational autoencoders (VAEs) or Generative Adversarial Networks (GANs). These technologies are becoming more accessible and accurate, resulting in fake videos that are very difficult to be detected. Traditionally, Convolutional Neural Networks (CNNs) have been used to perform video deepfake detection, with the best results obtained using methods based on EfficientNet B7. In this study, we focus on video deep fake detection on faces, given that most methods are becoming extremely accurate in the generation of realistic human faces. Specifically, we combine various types of Vision Transformers with a convolutional EfficientNet B0 used as a feature extractor, obtaining comparable results with some very recent methods that use Vision Transformers. Differently from the state-of-the-art approaches, we use neither distillation nor ensemble methods. Furthermore, we present a straightforward inference procedure based on a simple voting scheme for handling multiple faces in the same video shot. The best model achieved an AUC of 0.951 and an F1 score of 88.0%, very close to the state-of-the-art on the DeepFake Detection Challenge (DFDC).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Additionally, several researches in [57] and [58] have demonstrated that creating patches by using characteristics gathered in an image also increases the quality of the training process.”
From Generalization of Forgery Detection With Meta Deepfake Detection Model (2023) · cites this paper as [57]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Generative adversarial networks
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2020) · Communications of the ACM · doi:10.1145/3422622 · Cited by 18,384
YOLOv3: An Incremental Improvement
Redmon, J., & Farhadi, A. (2018) · arXiv · doi:10.48550/arxiv.1804.02767 · Cited by 17,141
A Multi-Modal Distributed Real-Time IoT System for Urban Traffic Control (Invited Paper)
Khanam, Z., Achari, V. P. S., Boukhennoufa, I., Jindal, A., & Singh, A. K. (2024) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/oasics.ng-res.2024.2 · Cited by 14,040
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465
Analyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00813 · Cited by 6,332
Analyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2019) · arXiv · doi:10.48550/arxiv.1912.04958 · Cited by 6,332
Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks
Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2603342 · Cited by 6,214
Han (2023) · doi:10.1109/tpami.2022.3152247 · Cited by 4,682
Research works printed but not linked: 1
- Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. Adv. Neural Inf. Process. Syst. 28, 91–99 (2015)
Cited by
A Survey on the Detection and Impacts of Deepfakes in Visual, Audio, and Textual Formats
Mubarak, R., Alsboui, T., Alshaikh, O., Inuwa-Dutse, I., Khan, S., & Parkinson, S. (2023) · IEEE Access · doi:10.1109/access.2023.3344653 · Cited by 132
“VAEs are often used for signal analysis tasks in deepfake generation [40], [41].”
MethodA review of deep learning‐based approaches for deepfake content detection
(2024) · doi:10.1111/exsy.13570 · Cited by 116
“The same authors also found that coupling higher-order wavelets and CNN attained an improved and competitive performance compared to a Discrete Cosine Transformer (DCT) approach or working directly on the raw images, where combined architectures show the best performance.”
ResultInfluentialAbbas, F., & Taeihagh, A. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124260 · Cited by 184
“Gener - ally, a face-swapping method comprises three stages (Ding et al., 2022; Coccomini et al., 2022; Sun et al., 2023).”
BackgroundMethodComparison of Deepfake Detection Techniques through Deep Learning
Taeb, M., & Chi, H. (2022) · Journal of Cybersecurity and Privacy · doi:10.3390/jcp2010007 · Cited by 158
“The main difference between deepfake video- and image-detection methods is that video-detection methods can leverage spatial features [14], spatio-temporal anomalies [15] and supervised domain [16] to draw a conclusion on the whole video by aggregating the inferred output both in time and across multiple faces.”
BackgroundDeep convolutional pooling transformer for deepfake detection
Wang, T., Cheng, H., Chow, K. P., & Nie, L. (2023) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/3588574 · Cited by 111
“The three experiments are conducted upon the model with depths [4, 4, 6], [8, 8, 8], and [10, 10, 12] for the three phases and having the total depths of 14, 24, and 32 from shallow to deep, respectively.”
Method
Metadata
- Authors
- Coccomini, D. A., Messina, N., Gennaro, C., & Falchi, F.
- Affiliations
- Istituto di Scienza e Tecnologie dell'Informazione "Alessandro Faedo" (IT)
- Corresponding author
- Messina, Nicola
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer International Publishing
- Published
- 2022
- Pages
- 219-229
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W3179508259
- Cited by
- 296
- References
- 43
- Citations by year
- 2026: 54 · 2025: 85 · 2024: 80 · 2023: 56 · 2022: 21
- FWCI
- 21.94
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Lecture notes in computer science (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- ISTI Open Portal (submittedVersion, other-oa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Keywords
- face manipulation, EfficientNet, vision transformer, CNN-Transformer hybrid, feature extraction, DFDC, generative adversarial networks, variational autoencoder
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Combining EfficientNet and Vision Transformers for Video Deepfake Detection. https://app.cochat.ai/papers/10.1007/978-3-031-06433-3_19
- The paper
- Coccomini, D. A., Messina, N., Gennaro, C., & Falchi, F. (2022). Combining EfficientNet and Vision Transformers for Video Deepfake Detection. Lecture Notes in Computer Science. https://doi.org/10.1007/978-3-031-06433-3_19
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.