CoChat
Sign Up
Cited by 32Digital Media Forensic DetectionGenerative Adversarial Networks and Image SynthesisImage and Video Quality Assessment

MFF-Net: Deepfake Detection Network Based on Multi-Feature Fusion

Open access
Zhao, Lei, Zhang, Mingcheng, Ding, Hongwei, Cui, Xiaohui·2021·Entropy
DOI 10.3390/e23121692PMC PMC8700337
Open this paper in CoChat — free accountOpen PDF

Abstract

Significant progress has been made in generating counterfeit images and videos. Forged videos generated by deepfaking have been widely spread and have caused severe societal impacts, which stir up public concern about automatic deepfake detection technology. Recently, many deepfake detection methods based on forged features have been proposed. Among the popular forged features, textural features are widely used. However, most of the current texture-based detection methods extract textures directly from RGB images, ignoring the mature spectral analysis methods. Therefore, this research proposes a deepfake detection network fusing RGB features and textural information extracted by neural networks and signal processing methods, namely, MFF-Net. Specifically, it consists of four key components: (1) a feature extraction module to further extract textural and frequency information using the Gabor convolution and residual attention blocks; (2) a texture enhancement module to zoom into the subtle textural features in shallow layers; (3) an attention module to force the classifier to focus on the forged part; (4) two instances of feature fusion to firstly fuse textural features from the shallow RGB branch and feature extraction module and then to fuse the textural features and semantic information. Moreover, we further introduce a new diversity loss to force the feature extraction module to learn features of different scales and directions. The experimental results show that MFF-Net has excellent generalization and has achieved state-of-the-art performance on various deepfake datasets.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/e23121692

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The network can capture a thorough grasp of the video content owing to the multi-feature fusion strategy, improving the detection accuracy and robustness against various DeepFake manipulation approaches.

    “The network can capture a thorough grasp of the video content owing to the multi-feature fusion strategy, improving the detection accuracy and robustness against various DeepFake manipulation approaches.”

    From A Comprehensive Review of DeepFake Detection Using Advanced Machine Learning and Fusion Methods (2023) · cites this paper as [63]

    The Method (§4.2) describes fusion of RGB, textural, and frequency features from input face images, and the Experiments (§5.2) report improved detection across the tested face-manipulation methods. But the paper does not establish a thorough grasp of video content: its network takes face images as input. Its robustness tests (§5.5) concern image disturbances, while the manipulation-method tests train and test separately on each method, so robustness to varied or unseen manipulation approaches is overstated. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • The experimental findings in this study showed that when compared to single-feature-based approaches and other cutting-edge DeepFake detection models, MTFF-Net performed better.

    “The experimental findings in this study showed that when compared to single-feature-based approaches and other cutting-edge DeepFake detection models, MTFF-Net performed better.”

    From A Comprehensive Review of DeepFake Detection Using Advanced Machine Learning and Fusion Methods (2023) · cites this paper as [63]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 36

  1. [1]Goodfellow I. Pouget-Abadie J. Mirza M. Xu B. Warde-Farley D. Ozair S. Courville A. Bengio Y. Generative adversarial networks Commun. ACM 2020 63 139 144 10.1145/3422622 · DOI · Paper page
  2. [2]Karras T. Aila T. Laine S. Lehtinen J. Progressive Growing of GANs for Improved Quality, Stability, and Variation Proceedings of the International Conference on Learning Representations Vancouver, BC, Canada 30 April 30–3 May 2018
  3. [3]Karras T. Laine S. Aila T. A style-based generator architecture for generative adversarial networks Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Long Beach, CA, USA 15–20 June 2019 4401 4410
  4. [8]Rossler A. Cozzolino D. Verdoliva L. Riess C. Thies J. Nießner M. Faceforensics++: Learning to detect manipulated facial images Proceedings of the IEEE/CVF International Conference on Computer Vision Seoul, Korea 27–28 October 2019 1 11
  5. [9]Thies J. Zollhöfer M. Nießner M. Deferred neural rendering: Image synthesis using neural textures Acm Trans. Graph. (TOG) 2019 38 1 12 10.1145/3306346.3323035 · DOI
  6. [10]Afchar D. Nozick V. Yamagishi J. Echizen I. Mesonet: A compact facial video forgery detection network Proceedings of the 2018 IEEE International Workshop on Information Forensics and Security (WIFS) Hong Kong, China 11–13 December 2018 1 7
  7. [11]Li L. Bao J. Zhang T. Yang H. Chen D. Wen F. Guo B. Face X-ray for more general face forgery detection Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Seattle, WA, USA 16–18 June 2020 5001 5010
  8. [12]Matern F. Riess C. Stamminger M. Exploiting visual artifacts to expose deepfakes and face manipulations Proceedings of the 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW) Waikoloa Village, HI, USA 7–11 January 2019 83 92
  9. [14]Nguyen H.H. Fang F. Yamagishi J. Echizen I. Multi-task learning for detecting and segmenting manipulated facial images and videos Proceedings of the 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS) Tampa, FL, USA 23–26 September 2019 1 8
  10. [15]Frank J. Eisenhofer T. Schönherr L. Fischer A. Kolossa D. Holz T. Leveraging frequency analysis for deep fake image recognition Proceedings of the International Conference on Machine Learning PMLR, Montréal, QC, Canada 13–18 July 2020 3247 3258
  11. [18]Durall R. Keuper M. Pfreundt F.J. Keuper J. Unmasking deepfakes with simple features arXiv 2019 1911.00686
  12. [19]Qian Y. Yin G. Sheng L. Chen Z. Shao J. Thinking in frequency: Face forgery detection by mining frequency-aware clues Proceedings of the European Conference on Computer Vision Glasgow, UK 23–28 August 2020 86 103
  13. [22]Arjovsky M. Chintala S. Bottou L. Wasserstein generative adversarial networks Proceedings of the International conference on machine learning PMLR, Sydney, Australia 6–11 August 2017 214 223
  14. [23]Berthelot D. Schumm T. Metz L. Began: Boundary equilibrium generative adversarial networks arXiv 2017 1703.10717
  15. [24]Kodali N. Abernethy J. Hays J. Kira Z. On convergence and stability of gans arXiv 2017 1705.07215
  16. [25]Karras T. Laine S. Aittala M. Hellsten J. Lehtinen J. Aila T. Analyzing and improving the image quality of stylegan Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Seattle, WA, USA 13–19 June 2020 8110 8119
  17. [26]Miyato T. Kataoka T. Koyama M. Yoshida Y. Spectral Normalization for Generative Adversarial Networks Proceedings of the International Conference on Learning Representations Vancouver, BC, Canada 30 April–3 May 2018
  18. [27]Li C.L. Chang W.C. Cheng Y. Yang Y. Póczos B. MMD GAN: Towards deeper understanding of moment matching network Proceedings of the 31st International Conference on Neural Information Processing Systems Long Beach, CA, USA 4–9 December 2017 2200 2210
  19. [28]Yang X. Li Y. Lyu S. Exposing deep fakes using inconsistent head poses Proceedings of the ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) Brighton, UK 12–17 May 2019 8261 8265
  20. [29]Agarwal S. Farid H. Gu Y. He M. Nagano K. Li H. Protecting World Leaders Against Deep Fakes Proceedings of the CVPR Workshops Long Beach, CA, USA 16 June 2019 38 45
  21. [30]Carvalho T. Faria F.A. Pedrini H. Torres R.d.S. Rocha A. Illuminant-based transformed spaces for image forensics IEEE Trans. Inf. Forensics Secur. 2015 11 720 733 10.1109/TIFS.2015.2506548 · DOI
  22. [31]Durall R. Keuper M. Keuper J. Watch your up-convolution: Cnn based generative deep neural networks are failing to reproduce spectral distributions Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Seattle, WA, USA 13–19 June 2020 7890 7899
  23. [32]Huang Y. Juefei-Xu F. Wang R. Guo Q. Ma L. Xie X. Li J. Miao W. Liu Y. Pu G. Fakepolisher: Making deepfakes more detection-evasive by shallow reconstruction Proceedings of the 28th ACM International Conference on Multimedia Seattle, WA, USA 12–16 October 2020 1217 1226
  24. [34]Odena A. Dumoulin V. Olah C. Deconvolution and checkerboard artifacts Distill 2016 1 e3 10.23915/distill.00003 · DOI
  25. [35]Zhao H. Zhou W. Chen D. Wei T. Zhang W. Yu N. Multi-attentional deepfake detection Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Nashville, TN, USA 19–25 June 2021 2185 2194
  26. [36]Chollet F. Xception: Deep learning with depthwise separable convolutions Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Honolulu, HI, USA 21–26 July 2017 1251 1258
  27. [37]Woo S. Park J. Lee J.Y. Kweon I.S. Cbam: Convolutional block attention module Proceedings of the European conference on computer vision (ECCV) Munich, Germany 8–14 September 2018 3 19
  28. [38]Liu Z. Luo P. Wang X. Tang X. Deep learning face attributes in the wild Proceedings of the IEEE international conference on computer vision Santiago, Chile 7–13 December 2015 3730 3738
  29. [40]Cozzolino D. Poggi G. Verdoliva L. Recasting residual-based local descriptors as convolutional neural networks: An application to image forgery detection Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security Philadelphia, PA, USA 20–21 June 2017 159 164
  30. [41]Bayar B. Stamm M.C. A deep learning approach to universal image manipulation detection using a new convolutional layer Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security Vigo, Spain 20–22 June 2016 5 10
  31. [42]Rahmouni N. Nozick V. Yamagishi J. Echizen I. Distinguishing computer graphics from natural images using convolution neural networks Proceedings of the 2017 IEEE Workshop on Information Forensics and Security (WIFS) Rennes, France 4–7 December 2017 1 6
  32. [43]Gunawan T.S. Hanafiah S.A.M. Kartiwi M. Ismail N. Za’bah N.F. Nordin A.N. Development of photo forensics algorithm by detecting photoshop manipulation using error level analysis Indones. J. Electr. Eng. Comput. Sci. 2017 7 131 137 10.11591/ijeecs.v7.i1.pp131-137 · DOI
  33. [44]Chen M. Sedighi V. Boroumand M. Fridrich J. JPEG-phase-aware convolutional neural network for steganalysis of JPEG images Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security Philadelphia, PA, USA 20–22 June 2017 75 84
  34. [47]Thies J. Zollhofer M. Stamminger M. Theobalt C. Nießner M. Face2face: Real-time face capture and reenactment of rgb videos Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Las Vegas, NV, USA 27–30 June 2016 2387 2395
  35. [49]Trinh L. Tsang M. Rambhatla S. Liu Y. Interpretable and trustworthy deepfake detection via dynamic prototypes Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision Waikola, HI, USA 5–9 January 2021 1973 1983
  36. [50]Chen C. Li O. Tao D. Barnett A. Rudin C. Su J.K. This looks like that: Deep learning for interpretable image recognition Adv. Neural Inf. Process. Syst. 2019 32 8930 8941

Cited by

Sorted by citations

Authors
Zhao, L., Zhang, M., Ding, H., & Cui, X.
Affiliations
  • Wuhan University (CN)
  • Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University, Wuhan 430072, China
Corresponding author
Cui, Xiaohui
ORCID
Venue
Entropy
Publisher
MDPI AG
Published
2021-12-17
Volume
23
Issue
12
Pages
1692
ISSN
1099-4300 (ISSN-L)
DOI
10.3390/e23121692
PMC
PMC8700337
OpenAlex
W4200551480
Cited by
32
References
10
Citations by year
2026: 3 · 2025: 6 · 2024: 7 · 2023: 11 · 2022: 5
FWCI
2.07
Type
journal article
Language
en
Open access
Licence
Locations
  • Entropy (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
Keywords
deepfake detection, multi-feature fusion, textural properties, spectral analysis, frequency characteristics, attention mechanism, feature fusion, diversity loss, video forgery

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review