Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 68 of the 77 references it cites were found in a registry or an index.

Using Graph Neural Networks to Improve Generalization Capability of the Models for Deepfake Detection

Open access
She, Huimin, Hu, Yongjian, Liu, Beibei, Li, Jicheng, Li, Chang-Tsun · 2024 · IEEE Transactions on Information Forensics and Security

Abstract

Deepfake detection plays a key role in preventing the misuses of artificial intelligence in video editing. Current deep learning-based deepfake detection methods often perform quite well in intra-dataset testing, but they may lose good performance in cross-dataset testing. In other words, generalization capability is still a crucial problem to be resolved. In this paper, we address deepfake detection by treating an image as non-Euclidean data and representing it as a graph so as to infer the informative connections between image patches/nodes to improve the detector’s generalization capability. Specifically, we propose a graph neural network-based paradigm that casts deepfake detection as a graph binary classification problem. First, we propose a dual-branch network to extract node features from both RGB images and their color difference images (CDIs) via the Transformer-based trainable node encoder module (TNEM). Second, we adopt the adjacency matrix to establish the connections of the nodes and further optimize the graph representation by applying the adaptive threshold to the adjacency matrix. Third, multi-head graph convolutional neural networks are carried out for node feature extraction. RGB node features and CDI node features are concatenated and separately fed into the graph classifier and node classifier for forgery detection and forgery localization. Experimental results demonstrate that our method can overall outperform other state-of-the-art methods on 7 popular benchmark datasets. Notably, our model achieves the highest AUC values of 96.19%, 80.99% and 87.68% on Celeb-DF-V2, DFDC and DFDCP in turn when trained on FF++ (C23). The visualization of node classification results also provides good interpretability of our proposed approach.

Overview

Integrity signals

Registry

Crossref · 10.1109/tifs.2024.3451356

Cites

68 of 68 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

  • Attention Is All You Need

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2×

    “Inspired by [18] and [48], we propose to use the attention matrix M att to further model the correlations among n nodes.”
    BackgroundMethod
  • Exploiting Generative AI to Scale up Intelligent Tutoring Systems

    Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658

  • Image quality assessment: from error visibility to structural similarity

    Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · cited 1×

    “We find their corresponding real video frames and compute the structural dissimilarities (DSSIM) [58] between them to generate manipulated mask M , which are shown in Fig.”
    Background
  • ImageNet Large Scale Visual Recognition Challenge

    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 1×

    “1) Setting: Our Swin Transformer blocks are initialized with the weights pretrained on ImageNet-1K [56].”
    Method
  • Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

    Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 1×

    “Inspired by strong feature extraction capability of Swin Transformer [46], we design Trainable Node Encoder Module with reference to the network structure of Swin Transformer.”
    Method
  • Semi-supervised classification with graph convolutional networks

    Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853 · cited 1×

    “A single GCN layer f GCN can be represented as follows [49].”
    Background
  • Non-local Neural Networks

    Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596 · cited 3×

    “The dot-product similarity in pairwise self-consistency learning can be computed as follows [21], [40]. where x i and x j represent the feature vectors of node v i and v j , respectively. d refers to the feature dimension. σ( · ) is the sigmoid activation function.”
    Background
  • A Comprehensive Survey on Graph Neural Networks

    Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. S. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2978386 · Cited by 11,000 · cited 4×

    “However, if we treat image pixels as non-Euclidean data and regard each pixel as a node, we can use a graph rather than a matrix to represent an image [15], [16].”
    BackgroundMethodInfluential
  • FaceForensics++: Learning to Detect Manipulated Facial Images

    Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 5×

    “Pioneering deep learning-based methods (e.g., [1], [2], [29]) directly borrow popular neural networks from computer vision for deepfake detection.”
    BackgroundMethodInfluential
  • MesoNet: a Compact Facial Video Forgery Detection Network

    Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872 · cited 2×

    “Pioneering deep learning-based methods (e.g., [1], [2], [29]) directly borrow popular neural networks from computer vision for deepfake detection.”
    Method

Show all 77 references

Research works printed but not linked: 1

  1. Dufour, 2019, Contributing Data to Deepfake Detection Research

Cited by

See all 24

Metadata

Authors
She, H., Hu, Y., Liu, B., Li, J., & Li, C.-T.
Affiliations
  • South China University of Technology (CN)
  • Deakin University (AU)
Venue
IEEE Transactions on Information Forensics and Security
Published
2024
Volume
19
Pages
8414-8427
ISSN
  • 1556-6013 (ISSN-L)
  • 1556-6021
OpenAlex
W4401980784
Cited by
24
References
77
Citations by year
2026: 12 · 2025: 11 · 2024: 1
FWCI
3.35
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Information Forensics and Security (publishedVersion)
  • Figshare (submittedVersion, cc-by)
Funders
Science and Technology Foundation of Guangzhou Huangpu Development District (2022GH15)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
Keywords
deepfake detection, graph neural networks, cross-dataset generalization, image forgery detection, tamper localization, node classification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Using Graph Neural Networks to Improve Generalization Capability of the Models for Deepfake Detection. https://app.cochat.ai/papers/10.1109/tifs.2024.3451356
The paper
She, H., Hu, Y., Liu, B., Li, J., & Li, C.-T. (2024). Using Graph Neural Networks to Improve Generalization Capability of the Models for Deepfake Detection. IEEE Transactions on Information Forensics and Security. https://doi.org/10.1109/tifs.2024.3451356

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.