CoChat

Media Forensics Considerations on DeepFake Detection with Hand-Crafted Features

Open access
Siegel, Dennis, Kraetzer, Christian, Seidlitz, Stefan, Dittmann, Jana · 2021 · Journal of Imaging
Cited by 50

The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.

Abstract

DeepFake detection is a novel task for media forensics and is currently receiving a lot of research attention due to the threat these targeted video manipulations propose to the trust placed in video footage. The current trend in DeepFake detection is the application of neural networks to learn feature spaces that allow them to be distinguished from unmanipulated videos. In this paper, we discuss, with features hand-crafted by domain experts, an alternative to this trend. The main advantage that hand-crafted features have over learned features is their interpretability and the consequences this might have for plausibility validation for decisions made. Here, we discuss three sets of hand-crafted features and three different fusion strategies to implement DeepFake detection. Our tests on three pre-existing reference databases show detection performances that are under comparable test conditions (peak AUC > 0.95) to those of state-of-the-art methods using learned features. Furthermore, our approach shows a similar, if not better, generalization behavior than neural network-based methods in tests performed with different training and test sets. In addition to these pattern recognition considerations, first steps of a projection onto a data-centric examination approach for forensics process modeling are taken to increase the maturity of the present investigation.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/jimaging7070108

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 33

  1. Chesney, 2019, Deepfakes and the new disinformation war: The coming age of post-truth geopolitics, Foreign Aff., 98, 147
  2. Palmer, G.L. (2001). A Road Map for Digital Forensics Research—Report from the First Digital Forensics Research Workshop (DFRWS) (Technical Report DTR-T001-01 Final), Air Force Research Laboratory, Rome Research Site. Technical Report.
  3. Champod, C., and Vuille, J. (2011). Scientific Evidence in Europe—Admissibility, Evaluation and Equality of Arms. Int. Comment. Evid., 9. · DOI
  4. Krätzer, C. (2013). Statistical Pattern Recognition for Audio-forensics—Empirical Investigations on the Application Scenarios Audio Steganalysis and Microphone Forensics. [Ph.D. Thesis, Otto-von-Guericke-University].
  5. Kiltz, S. (2020). Data-Centric Examination Approach (DCEA) for a Qualitative Determination of Error, Loss and Uncertainty in Digital and Digitised Forensics. [Ph.D. Thesis, Otto-von-Guericke-University].
  6. Geradts, Z.J.M.H., Franke, K.Y., and Veenman, C.J. (2009). Multimedia forensics is not computer forensics. Computational Forensics, Springer. · doi:10.1007/978-3-642-03521-0
  7. Bondi, L., Cannas, E.D., Bestagini, P., and Tubaro, S. (2020). Training Strategies and Data Augmentations in CNN-based DeepFake Video Detection. arXiv. · DOI · Paper page
  8. Sabir, E., Cheng, J., Jaiswal, A., AbdAlmageed, W., Masi, I., and Natarajan, P. (2019). Recurrent Convolutional Strategies for Face Manipulation Detection in Videos. arXiv.
  9. Korshunov, P., and Marcel, S. (2018). DeepFakes: A New Threat to Face Recognition? Assessment and Detection. arXiv.
  10. Li, Y., and Lyu, S. (2018). Exposing DeepFake Videos By Detecting Face Warping Artifacts. arXiv.
  11. McCloskey, S., and Albright, M. (2018). Detecting GAN-generated Imagery using Color Cues. arXiv. · DOI · Paper page
  12. Agarwal, S., Farid, H., Gu, Y., He, M., Nagano, K., and Li, H. (2019, January 16–20). Protecting World Leaders Against Deep Fakes. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Long Beach, CA, USA.
  13. Lin, 2020, Comparison of handcrafted features and convolutional neural networks for liver MR image adequacy assessment, Sci. Rep., 10, 20336, 10.1038/s41598-020-77264-y · DOI
  14. Yin, H., Tino, P., Corchado, E., Byrne, W., and Yao, X. (2007). Filter Methods for Feature Selection—A Comparative Study. Intelligent Data Engineering and Automated Learning—IDEAL 2007, Springer. · DOI
  15. Law, 2004, Simultaneous feature selection and clustering using mixture models, IEEE Trans. Pattern Anal. Mach. Intell., 26, 1154, 10.1109/TPAMI.2004.71 · DOI · Paper page
  16. Kiltz, S., Dittmann, J., and Vielhauer, C. (2015, January 18–20). Supporting Forensic Design—A Course Profile to Teach Forensics. Proceedings of the 2015 Ninth International Conference on IT Security Incident Management and IT Forensics, Magdeburg, Germany. · doi:10.1109/imf.2015.16
  17. Altschaffel, R. (2020). Computer Forensics in Cyber-Physical Systems: Applying Existing Forensic Knowledge and Procedures from Classical IT to Automation and Automotive. [Ph.D. Thesis, Otto-von-Guericke-University].
  18. Kiltz, S., Hoppe, T., and Dittmann, J. (2009, January 5–7). A New Forensic Model and Its Application to the Collection, Extraction and Long Term Storage of Screen Content off a Memory Dump. Proceedings of the 16th International Conference on Digital Signal Processing, DSP’09, Santorini, Greece. · doi:10.1109/icdsp.2009.5201189
  19. Sagonas, 2016, 300 Faces In-The-Wild Challenge, Image Vis. Comput., 47, 3, 10.1016/j.imavis.2016.01.002 · DOI · Paper page
  20. King, 2009, Dlib-ml: A Machine Learning Toolkit, J. Mach. Learn. Res., 10, 1755
  21. Makrushin, A., Neubert, T., and Dittmann, J. (March, January 27). Automatic generation and detection of visually faultless facial morphs. Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications—Volume 6: VISAPP, (VISIGRAPP 2017), INSTICC, Porto, Portugal. · DOI · Paper page
  22. Kraetzer, C., Makrushin, A., Neubert, T., Hildebrandt, M., and Dittmann, J. (2017, January 20–22). Modeling Attacks on Photo-ID Documents and Applying Media Forensics for the Detection of Facial Morphing. Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security, IH&MMSec ’17, Philadelphia, PA, USA. · DOI
  23. Hall, 2009, The WEKA data mining software: An update, SIGKDD Explor., 11, 10, 10.1145/1656274.1656278 · DOI · Paper page
  24. Quinlan, J.R. (1993). C4.5: Programs for Machine Learning, Morgan Kaufmann Publishers Inc.
  25. Sanderson, 2009, Multi-Region Probabilistic Histograms for Robust and Scalable Identity Inference, LNCS, 5558, 199
  26. Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., and Nießner, M. (2018). FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces. arXiv.
  27. Agarwal, S., Farid, H., Fried, O., and Agrawala, M. (2020, January 14–19). Detecting Deep-Fake Videos From Phoneme-Viseme Mismatches. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Seattle, WA, USA. · DOI · Paper page
  28. Bradski, 2000, The OpenCV Library, Dobb J. Softw. Tools, 120, 122
  29. Ross, A.A., Nandakumar, K., and Jain, A.K. (2006). Levels of Fusion in Biometrics. Handbook of Multibiometrics, Springer.
  30. Kuncheva, L.I. (2004). Combining Pattern Classifiers: Methods and Algorithms, Wiley. · DOI · Paper page
  31. Zhang, W., Zhao, C., and Li, Y. (2020). A Novel Counterfeit Feature Extraction Technique for Exposing Face-Swap Images Based on Deep Learning and Error Level Analysis. Entropy, 22. · DOI · Paper page
  32. Zhang, W., and Zhao, C. (2020). Exposing Face-Swap Images Based on Deep Learning and ELA Detection. Proceedings, 46. · DOI
  33. Krawetz, N. (August, January 28). A Picture ’ s Worth…Digital Image Analysis and Forensics. Proceedings of the Black Hat Briefings 2007, Las Vegas, NV, USA.

Cited by

See all 50

Metadata

Authors
Siegel, D., Kraetzer, C., Seidlitz, S., & Dittmann, J.
Venue
Journal of Imaging
Publisher
MDPI AG
Published
2021-07-01
Volume
7
Issue
7
Pages
108
ISSN
2313-433X (ISSN-L)
OpenAlex
W3175824036
Cited by
50
References
15
FWCI
3.09
Type
journal article
Language
en
Open access
gold
Funders
Bundesministerium für Bildung und Forschung (13N15736)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.