CoChat
Sign Up
Cited by 7Digital Media Forensic DetectionMisinformation and Its ImpactsGenerative Adversarial Networks and Image Synthesis

The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.

Deepfake Detection Based on Incompatibility Between Multiple Modes

Zhang, Yuxin, Zhan, Jinyu, Jiang, Wei, Fan, Zhufeng·2021·2021 International Conference on Intelligent Technology and Embedded Systems (ICITES)
DOI 10.1109/icites53477.2021.9637096
Open this paper in CoChat — free account

Abstract

We propose a multi-modal detection for deepfake videos, called the Incompatibility Between Multiple Modes (IBMM) detection. The detection algorithm can detect whether the video is real or fake, and may be embedded in the monitoring equipment in the future. The model adopts EfficientNet and simple 3D-CNN, and it identifies deepfake videos through three modes. In the facial motion mode and lip motion mode, we use the EfficientNet for feature learning. This network uses a series of fixed scaling coefficients to scale the dimensions of the network uniformly and achieves good results in learning image features. In the audio mode, we adopt 3D-CNN network to train the hot coding diagram of audio data. Besides, for a single mode, we use the cross-entropy loss to calculate the irrationality of the mode. For different modes, the contrastive loss is used to calculate the incongruity between the modes, such as incompatibility between lips and voice. Experimental results show that, compared with other existing fake detection methods, the method presented in this paper has higher accuracy (95.87%) on DFDC datasets. And compared with the existing methods, the accuracy increases by 5.21%.

Overview

Integrity signals

Registry

Crossref · 10.1109/icites53477.2021.9637096

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 6 references

Research works printed but not linked: 23

  1. sabour, 2017, Dynamic routing between capsules[J], ArXiv Preprint
  2. 10.1109/ICCV.2019.00009 · DOI · Paper page
  3. 10.1145/2929464.2929475 · DOI
  4. chung, 2016, Out of time: automated lip sync in the wild[C], Asian Conference on Computer Vision, 251
  5. 0, AutoGAN
  6. 2018, Fake Videos by Detecting Eye Blinking[C], 2018 IEEE International Workshop on Information Forensics and Security (WIFS), 23
  7. dolhansky, 2019, The deepfake detection challenge (dfdc) preview dataset[J], ArXiv Preprint
  8. li, 2018, Exposing deepfake videos by detecting face warping artifacts[J], ArXiv Preprint
  9. 10.3115/v1/D14-1179 · DOI · Paper page
  10. 10.1109/WACVW.2019.00020 · DOI · Paper page
  11. 10.1109/18.850674 · DOI · Paper page
  12. durall, 2020, Unmasking DeepFakes with Simple Features
  13. li, 2018, Exposing DeepFake Videos By Detecting Face Warping Artifacts, CVPR Workshops
  14. 10.1007/s11042-015-2847-3 · DOI
  15. li, 0, Detection of deep network generated images using disparities in color components. arXiv 2018[J], ArXiv Preprint
  16. 10.1016/j.jnlest.2020.100031 · DOI
  17. li, 2018, Exposing deepfake videos by detecting face warping artifacts[J], ArXiv Preprint
  18. 10.1016/j.jnlest.2020.100027 · DOI
  19. sanderson, 2002, The vidtimit database, Technical Report IDIAP
  20. gupta, 2021, Not made for each other-Audio-Visual Dissonance-based deepfake detection and localization[J]
  21. 10.1109/BTAS46853.2019.9185974 · DOI · Paper page
  22. han, 2017, Two-stream neural networks for tampered face detection[C], Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 19
  23. 10.1145/3183518 · DOI

Cited by

Sorted by citations

Metadata

Authors
Zhang, Y., Zhan, J., Jiang, W., & Fan, Z.
Affiliations
University of Electronic Science and Technology of China (CN)
ORCID
Venue
2021 International Conference on Intelligent Technology and Embedded Systems (ICITES)
Publisher
IEEE
Published
2021-10-31
Pages
1-7
DOI
10.1109/icites53477.2021.9637096
OpenAlex
W4200164272
Cited by
7
References
6
Citations by year
2026: 2 · 2024: 3 · 2023: 2
FWCI
0.41
Type
conference paper
Language
en
Open access
closed
Licence
Locations
International Conference on Intelligent Technology and Embedded Systems (ICITES) (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Keywords
deepfake detection, multimodal detection, video authenticity, facial motion, lip movement, audio features, contrastive loss, EfficientNet, 3D convolutional neural networks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review