CoChat

A Dual-Branch Fusion Model for Deepfake Detection Using Video Frames and Microexpression Features

Open access
Petmezas, Georgios, Vanian, Vazgken, Rufete, Manuel Pastor, Almaloglou, Eleana E. I., Zarpalas, Dimitris · 2025 · Journal of Imaging

The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.

Abstract

Deepfake detection has become a critical issue due to the rise of synthetic media and its potential for misuse. In this paper, we propose a novel approach to deepfake detection by combining video frame analysis with facial microexpression features. The dual-branch fusion model utilizes a 3D ResNet18 for spatiotemporal feature extraction and a transformer model to capture microexpression patterns, which are difficult to replicate in manipulated content. We evaluate the model on the widely used FaceForensics++ (FF++) dataset and demonstrate that our approach outperforms existing state-of-the-art methods, achieving 99.81% accuracy and a perfect ROC-AUC score of 100%. The proposed method highlights the importance of integrating diverse data sources for deepfake detection, addressing some of the current limitations of existing systems.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/jimaging11070231

Cites

26 of 26 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 The detailed microexpression extraction pipeline. It begins with an input facial frame sequence, which is processed through two parallel paths. In the first path, facial landmark detection is performed, where black dots indicate the extracted 2D facial landmarks, including both rigid and non-rigid components. In the second path, the same sequence is used for head pose and gaze estimation. Both streams then feed into a shared module for AU extraction and intensity estimation, culminating in the final AU categorization. Arrows denote the directional flow of data between processing stages.
Figure 1 The detailed microexpression extraction pipeline. It begins with an input facial frame sequence, which is processed through two parallel paths. In the first path, facial landmark detection is performed, where black dots indicate the extracted 2D facial landmarks, including both rigid and non-rigid components. In the second path, the same sequence is used for head pose and gaze estimation. Both streams then feed into a shared module for AU extraction and intensity estimation, culminating in the final AU categorization. Arrows denote the directional flow of data between processing stages.CC BY · PubMed Central
Figure 2 Overview of the proposed methodology.
Figure 2 Overview of the proposed methodology.CC BY · PubMed Central
Figure 3 Detailed architecture of the dual-branch fusion model, illustrating (a) the 3D ResNet18-based frame branch, (b) the Transformer-based microexpression branch, and (c) the late fusion and classification stage.
Figure 3 Detailed architecture of the dual-branch fusion model, illustrating (a) the 3D ResNet18-based frame branch, (b) the Transformer-based microexpression branch, and (c) the late fusion and classification stage.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

  • Attention Is All You Need

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×

    “On the other hand, transformer [25] models, originally introduced for natural language processing, have gained prominence in vision tasks due to their ability to model long-range dependencies and relationships.”
  • Facial action coding system

    Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019

  • FaceForensics++: Learning to Detect Manipulated Facial Images

    Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 1×

    “For our experiments, we utilized two widely recognized deepfake detection benchmarks, namely FF++ [16] and Celeb-DF [17], as well as a recently introduced deepfake dataset named ReenactFaces [18].”
  • Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics

    Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00327 · Cited by 1,846 · cited 2×

    “For our experiments, we utilized two widely recognized deepfake detection benchmarks, namely FF++ [16] and Celeb-DF [17], as well as a recently introduced deepfake dataset named ReenactFaces [18].”
  • Multi-attentional Deepfake Detection

    Zhao, H., Wei, T., Zhou, W., Zhang, W., Chen, D., & Yu, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00222 · Cited by 1,019 · cited 2×

    “…Zhang et al. [5] 95.20% 98.68% Miao et al. [6] 98.71% 99.74% Zhang et al. [7] 94.14% 98.44% Guo et al. [8] 96.01% 98.92% Long et al. [9] 97.64% 99.70% Pang et al. [10] 97.76% 98.81% Yang et al. [11] 97.86% 99.38% Zhao et al. [12] 97.60% 99.29% Alfalasi et al. [15] 97.31% 99.60% Ours 99.81% 100%”
    Influential
  • Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition

    Hara, K., Kataoka, H., & Satoh, Y. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.373 · Cited by 726 · cited 1×

    “The 3D ResNet18 [24] extends the ResNet architecture to video data by incorporating 3D convolutions, enabling it to learn spatiotemporal features from sequences of frames.”
  • Deepfake Detection: A Systematic Literature Review

    Rana, M. S., Nobi, M. N., Murali, B., & Sung, A. H. (2022) · IEEE Access · doi:10.1109/access.2022.3154404 · Cited by 573 · cited 1×

    “The emergence of deepfake technology has created serious concerns about the trust-worthiness of digital media [1].”
  • Cross-dataset learning and person-specific normalisation for automatic Action Unit detection

    Baltrusaitis, T., Mahmoud, M., & Robinson, P. (2015) · 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2015.7284869 · Cited by 412 · cited 1×

    “To leverage this, we followed the methodology by Baltrušaitis et al. [22] to perform a feature-based analysis of facial landmarks and AUs, extracting the relevant microexpression features from the preprocessed video frames.”
  • Deepfake detection using deep learning methods: A systematic and comprehensive review

    Heidari, A., Jafari Navimipour, N., Dag, H., & Unal, M. (2024) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1520 · Cited by 349 · cited 1×

    “In response to the growing threat of deepfakes, researchers have developed automated detection methods that primarily analyze visual and temporal inconsistencies in video frames [2].”
  • Motion Representations for Articulated Animation

    Siarohin, A., Woodford, O. J., Ren, J., Chai, M., & Tulyakov, S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01344 · Cited by 245 · cited 1×

    “We further benchmarked our method on ReenactFaces, a new dataset comprising 10,000 real and 10,000 re-enacted videos created using three state-of-the-art facial re-enactment methods, namely the First-Order Motion Model [19], Thin-Plate Spline Motion [20], and FSRT [21].”

Show all 26 references

Research works printed but not linked: 1

  1. Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., and Natsev, A. (2017). The Kinetics Human Action Video Dataset. arXiv.

Cited by

See all 9

Metadata

Authors
Petmezas, G., Vanian, V., Rufete, M. P., Almaloglou, E. E. I., & Zarpalas, D.
Affiliations
  • Centre for Research and Technology Hellas, 57001 Thessaloniki, Greece
  • Centre for Research and Technology Hellas (GR)
  • Herta Security, 08037 Barcelona, Spain
Corresponding author
Vanian, Vazgken
Venue
Journal of Imaging
Publisher
MDPI AG
Published
2025-07-11
Volume
11
Issue
7
Pages
231
ISSN
2313-433X (ISSN-L)
OpenAlex
W4412196232
Cited by
8
References
26
Citations by year
2026: 8
FWCI
2.81
Type
journal article
Language
en
Open access
Locations
  • Journal of Imaging (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
European Commission (101073928)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Keywords
deepfake detection, facial micro-expressions, video frame analysis, spatiotemporal feature extraction, transformer models, synthetic media, FaceForensics++

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A Dual-Branch Fusion Model for Deepfake Detection Using Video Frames and Microexpression Features. https://app.cochat.ai/papers/10.3390/jimaging11070231
The paper
Petmezas, G., Vanian, V., Rufete, M. P., Almaloglou, E. E. I., & Zarpalas, D. (2025). A Dual-Branch Fusion Model for Deepfake Detection Using Video Frames and Microexpression Features. Journal of Imaging. https://doi.org/10.3390/jimaging11070231

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.