MSTA-Net: Forgery Detection by Generating Manipulation Trace Based on Multi-Scale Self-Texture Attention
The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Lots of Deepfake videos are circulating on the Internet, which not only damages the personal rights of the forged individual, but also pollutes the web environment. What’s worse, it may trigger public opinion and endanger national security. Therefore, it is urgent to fight deep forgery. Most of the current forgery detection algorithms are based on convolutional neural networks to learn the feature differences between forged and real frames from big data. In this paper, from the perspective of image generation, we simulate the forgery process based on image generation and explore possible trace of forgery. We propose a multi-scale self-texture attention Generative Network(MSTA-Net) to track the potential texture trace in image generation process and eliminate the interference of deep forgery post-processing. Firstly, a generator with encoder-decoder is to disassemble images and performed trace generation, then we merge the generated trace image and the original map, which is input into the classifier with Resnet as the backbone. Secondly, the self-texture attention mechanism(STA) is proposed as the skip connection between the encoder and the decoder, which significantly enhances the texture characteristics in the image disassembly process and assists the generation of texture trace. Finally, we propose a loss function called Prob-tuple loss restricted by classification probability to amend the generation of forgery trace directly. To verify the performance of the MSTA-Net, we design different experiments to verify the feasibility and advancement of the method. Experimental results show that the proposed method performs well on deep forged databases represented by FaceForensics++, Celeb-DF, Deeperforensics and DFDC, and some results are reaching the state-of-the-art.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267 · cited 1×
“To improve the intra-class compactness and the separability between real and fake samples, we also use multi-scale triplet loss [42] for adjusting the sample distribution according to the triplets’ characteristic.”
MethodWang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01228-7 · Cited by 6,909
Analyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00813 · Cited by 6,332
Analyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2019) · arXiv · doi:10.48550/arxiv.1912.04958 · Cited by 6,332
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 6×
“[4] achieved significantly improved performance on compressed images, which was essential for”
BackgroundMethodInfluentialMesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics
Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00327 · Cited by 1,846 · cited 2×
“For example, although FF ++ [4] and Celeb-DF [7] are both face tampering databases, they have overlap space because of the contour and color features common to faces, but there are also many non-overlapping parts.”
BackgroundMethodSelf-citation
Research works printed but not linked: 31
- 10.1109/TCSVT.2021.3089724 · DOI
- 10.1109/TCSVT.2019.2945057 · DOI
- 10.1109/TCSVT.2019.2918852 · DOI
- Zhao, 2021, A transferable anti-forensic attack on forensic CNNs using a generative adversarial network, arXiv:2101.09568 · arXiv
- 10.1109/TCSVT.2019.2892608 · DOI
- Goodfellow, Generative adversarial nets, 2672
- Feng, 2020, Learning generalized spoof cues for face anti-spoofing, arXiv:2005.03922 · arXiv
- 10.1109/TCSVT.2021.3061153 · DOI
- 10.1109/TCSVT.2020.3014889 · DOI
- Dolhansky, 2020, The DeepFake detection challenge dataset, CoRR, abs/2006.07397 · arXiv
- 2018, Faceswap-GAN
- 2018, Faceswap: Deepfakes Software for All
- Li, 2019, FaceShifter: Towards high fidelity and occlusion aware face swapping, arXiv:1912.13457 · arXiv
- 10.1109/TCSVT.2021.3068294 · DOI
- 10.1145/3306346.3323035 · DOI · Paper page
- Karras, 2021, Alias-free generative adversarial networks, arXiv:2106.12423 · arXiv
- 10.1109/CVPR46437.2021.00991 · DOI · Paper page
- 10.1109/ICCV48922.2021.00209 · DOI · Paper page
- Agarwal, Protecting world leaders against deep fakes, 1
- 10.1109/WACV.2016.7477553 · DOI · Paper page
- Güera, We need no pixels: Video manipulation detection using stream descriptors, 1
- 10.1145/3297280.3297410 · DOI · Paper page
- 10.1109/CVPR.2018.00116 · DOI · Paper page
- 10.1109/ICCV.2019.00765 · DOI · Paper page
- 10.1109/TCSVT.2021.3074259 · DOI · Paper page
- 10.1109/TPAMI.2019.2928540 · DOI · Paper page
- Vaswani, 2017, Attention is all you need, CoRR, abs/1706.03762 · arXiv
- 10.1109/TCSVT.2018.2884203 · DOI
- 10.1016/j.future.2021.06.043 · DOI · Paper page
- 10.1109/WIFS49906.2020.9360901 · DOI · Paper page
- 10.1007/s10479-011-0841-3 · DOI · Paper page
Cited by
Masood, M., Nawaz, M., Malik, K. M., Javed, A., Irtaza, A., & Malik, H. (2023) · Applied Intelligence · doi:10.1007/s10489-022-03766-z · Cited by 543
Liu, X., Liu, Y., Chen, J., & Liu, X. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3189545 · Cited by 417
FAMM: Facial Muscle Motions for Detecting Compressed Deepfake Videos Over Social Networks
Liao, X., Wang, Y., Wang, T., Hu, J., & Wu, X. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3278310 · Cited by 169
“A multi-scale self-texturing network [17] was presented to track potential texture traces during image generation to detect Deepfake videos from the perspective of image generation, simulating the forgery process based on image generation.”
BackgroundA Survey on the Detection and Impacts of Deepfakes in Visual, Audio, and Textual Formats
Mubarak, R., Alsboui, T., Alshaikh, O., Inuwa-Dutse, I., Khan, S., & Parkinson, S. (2023) · IEEE Access · doi:10.1109/access.2023.3344653 · Cited by 132
Deepfake Attacks: Generation, Detection, Datasets, Challenges, and Research Directions
Naitali, A., Ridouani, M., Salahdine, F., & Kaabouch, N. (2023) · Computers · doi:10.3390/computers12100216 · Cited by 117
Metadata
- Authors
- Yang, J., Xiao, S., Li, A., Lu, W., Gao, X., & Li, Y.
- Venue
- IEEE Transactions on Circuits and Systems for Video Technology
- Published
- 2022-07
- Volume
- 32
- Issue
- 7
- Pages
- 4854-4866
- ISSN
- 1051-8215 (ISSN-L)
- 1558-2205
- OpenAlex
- W4205974231
- Cited by
- 115
- References
- 23
- FWCI
- 7.67
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Funders
- National Natural Science Foundation of China (61871283)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: MSTA-Net: Forgery Detection by Generating Manipulation Trace Based on Multi-Scale Self-Texture Attention. https://app.cochat.ai/papers/10.1109/tcsvt.2021.3133859
- The paper
- Yang, J., Xiao, S., Li, A., Lu, W., Gao, X., & Li, Y. (2022). MSTA-Net: Forgery Detection by Generating Manipulation Trace Based on Multi-Scale Self-Texture Attention. IEEE Transactions on Circuits and Systems for Video Technology. https://doi.org/10.1109/tcsvt.2021.3133859
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.