Masked Relation Learning for DeepFake Detection
The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
DeepFake detection aims to differentiate falsified faces from real ones. Most approaches formulate it as a binary classification problem by solely mining the local artifacts and inconsistencies of face forgery, which neglect the relation across local regions. Although several recent works explore local relation learning for DeepFake detection, they overlook the propagation of relational information and lead to limited performance gains. To address these issues, this paper provides a new perspective by formulating DeepFake detection as a graph classification problem, in which each facial region corresponds to a vertex. But relational information with large redundancy hinders the expressiveness of graphs. Inspired by the success of masked modeling, we propose Masked Relation Learning which decreases the redundancy to learn informative relational features. Specifically, a spatiotemporal attention module is exploited to learn the attention features of multiple facial regions. A relation learning module masks partial correlations between regions to reduce redundancy and then propagates the relational information across regions to capture the irregularity from a global view of the graph. We empirically discover that a moderate masking rate (e.g., 50%) brings the best performance gain. Experiments verify the effectiveness of Masked Relation Learning and demonstrate that our approach outperforms the state of the art by 2% AUC on the cross-dataset DeepFake video detection. Code will be available athttps://github.com/zimyang/MaskRelation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Self-citationBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
The graph neural network model
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2005605 · Cited by 9,582
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Masked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollar, P., & Girshick, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01553 · Cited by 8,489
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01228-7 · Cited by 6,909
Residual Attention Network for Image Classification
Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., Zhang, H., Wang, X., & Tang, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.683 · Cited by 4,073
Research works printed but not linked: 55
- 10.5555/2969033.2969125 · doi:10.5555/2969033.2969125
- Kingma, Auto-encoding variational Bayes, 1
- 2016, Fakeapp
- 10.1109/ICCV.2019.00728 · DOI · Paper page
- Agarwal, Protecting world leaders against deep fakes, 38
- Dosovitskiy, An image is worth 16×16 words: Transformers for image recognition at scale, 1
- 10.1109/ICASSP43922.2022.9746888 · DOI
- 10.1109/CVPR52688.2022.00925 · DOI · Paper page
- 10.1609/aaai.v36i1.19954 · DOI · Paper page
- 10.1007/978-3-031-20065-6_23 · DOI
- 10.1007/978-3-031-19781-9_7 · DOI
- 10.1145/3394171.3414034 · DOI · Paper page
- 10.1016/j.patcog.2021.107950 · DOI
- 10.1016/j.sigpro.2021.108051 · DOI
- 10.1016/j.knosys.2022.109114 · DOI
- 10.1145/3534678.3539321 · DOI · Paper page
- 10.24963/ijcai.2019/565 · DOI
- 10.1016/j.patrec.2021.04.021 · DOI
- Jin, 2020, Self-supervised learning on graphs: Deep insights and new direction, arXiv:2006.10141 · arXiv
- 10.1145/3437880.3460400 · DOI · Paper page
- 10.1109/ICCVW.2019.00213 · DOI
- 10.1145/3197517.3201350 · DOI
- 10.1109/CVPR46437.2021.00361 · DOI · Paper page
- 10.1007/978-3-031-19775-8_35 · DOI
- 10.1109/TIFS.2020.3013186 · DOI
- 10.1609/aaai.v34i07.6857 · DOI
- 10.1109/CVPR52688.2022.00304 · DOI
- 10.1109/CVPR52688.2022.01515 · DOI
- Kipf, Semi-supervised classification with graph convolutional networks, 1
- 10.1038/s41467-021-23415-2 · DOI
- 10.1016/j.patcog.2022.108581 · DOI
- 10.1109/ICASSP.2018.8462291 · DOI · Paper page
- 10.1145/1390156.1390160 · DOI
- 10.1109/ICCV.2019.00356 · DOI · Paper page
- 10.1109/TITS.2019.2935152 · DOI · Paper page
- 10.48550/ARXIV.1706.03762 · DOI · Paper page
- 10.1109/ICCV.2019.00557 · DOI
- Wettig, 2022, Should you mask 15% in masked language modeling?, arXiv:2202.08005 · arXiv
- Feichtenhofer, 2022, Masked autoencoders as spatiotemporal learners, arXiv:2205.09113 · arXiv
- 10.1109/ICTAI50040.2020.00154 · DOI
- 10.1145/3394486.3403140 · DOI · Paper page
- 10.1109/CVPR.2018.00675 · DOI · Paper page
- 10.1007/978-3-030-58565-5_10 · DOI · Paper page
- Krizhevsky, ImageNet classification with deep convolutional neural networks, 1106
- Guan, Delving into sequential patches for deepfake detection, 15
- 10.1109/CVPR52688.2022.01171 · DOI
- 10.1109/ASRU46091.2019.9003738 · DOI
- 10.1109/ICCV48922.2021.01211 · DOI
- Srivastava, 2014, Dropout: A simple way to prevent neural networks from overfitting, J. Mach. Learn. Res., 15, 1929
- 10.1109/CVPR.2016.262 · DOI · Paper page
- 10.1145/3306346.3323035 · DOI · Paper page
- 10.1109/LSP.2016.2603342 · DOI · Paper page
- Kingma, Adam: A method for stochastic optimization, 1
- Battaglia, 2018, Relational inductive biases, deep learning, and graph networks, arXiv:1806.01261 · arXiv
- 10.1109/TIP.2020.2994412 · DOI
Cited by
FAMM: Facial Muscle Motions for Detecting Compressed Deepfake Videos Over Social Networks
Liao, X., Wang, Y., Wang, T., Hu, J., & Wu, X. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3278310 · Cited by 169
Deepfake detection using convolutional vision transformers and convolutional neural networks
Soudy, A. H., Sayed, O., Tag-Elser, H., Ragab, R., Mohsen, S., Mostafa, T., Abohany, A. A., & Slim, S. O. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10181-7 · Cited by 129
“Yang et al. [33] presented a novel view by developing deepfake identification as a graph classification problem in which each face area is associated with a vertex.”
Deepfake Attacks: Generation, Detection, Datasets, Challenges, and Research Directions
Naitali, A., Ridouani, M., Salahdine, F., & Kaabouch, N. (2023) · Computers · doi:10.3390/computers12100216 · Cited by 117
Beyond the Prior Forgery Knowledge: Mining Critical Clues for General Face Forgery Detection
Luo, A., Kong, C., Huang, J., Hu, Y., Kang, X., & Kot, A. C. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3332218 · Cited by 115
A Comprehensive Review of DeepFake Detection Using Advanced Machine Learning and Fusion Methods
Gupta, G., Raja, K., Gupta, M., Jan, T., Whiteside, S. T., & Prasad, M. (2023) · Electronics · doi:10.3390/electronics13010095 · Cited by 99
Metadata
- Authors
- Yang, Z., Liang, J., Xu, Y., Zhang, X.-Y., & He, R.
- Venue
- IEEE Transactions on Information Forensics and Security
- Published
- 2023
- Volume
- 18
- Pages
- 1696-1708
- ISSN
- 1556-6013 (ISSN-L)
- 1556-6021
- OpenAlex
- W4322576470
- Cited by
- 143
- References
- 40
- FWCI
- 13.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Funders
- National Natural Science Foundation of China (U2003111, U21B2045, 62276256)
- Beijing Nova Program (Z211100002121108)
- Chinese Association for Artificial Intelligence (CAAI)-Huawei MindSpore Open Fund
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Masked Relation Learning for DeepFake Detection. https://app.cochat.ai/papers/10.1109/tifs.2023.3249566
- The paper
- Yang, Z., Liang, J., Xu, Y., Zhang, X.-Y., & He, R. (2023). Masked Relation Learning for DeepFake Detection. IEEE Transactions on Information Forensics and Security. https://doi.org/10.1109/tifs.2023.3249566
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.