CoChat

Detecting handcrafted facial image manipulations and GAN-generated facial images using Shallow-FakeFaceNet

Lee, Sangyup, Tariq, Shahroz, Shin, Youjin, Woo, Simon S. · 2021 · Applied Soft Computing
Cited by 93

The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.

Abstract

The rapid progress of sophisticated image editing tools has made it easier to manipulate original face images and create fake media content by putting one’s face to another. In addition to image editing tools, creating natural-looking fake human faces can be easily achieved by Generative Adversarial Networks (GANs). However, malicious use of these new media generation technologies can lead to severe problems, such as the development of fake pornography, defamation, or fraud. In this paper, we introduce a novel Handcrafted Facial Manipulation (HFM) image dataset and soft computing neural network models (Shallow-FakeFaceNets) with an efficient facial manipulation detection pipeline. Our neural network classifier model, Shallow-FakeFaceNet (SFFN), shows the ability to focus on the manipulated facial landmarks to detect fake images. The detection pipeline only relies on detecting fake facial images based on RGB information, not leveraging any metadata, which can be easily manipulated. Our results show that our method achieves the best performance of 72.52% in Area Under the Receiver Operating Characteristic (AUROC), gaining 3.99% F1-score and 2.91% AUROC on detecting handcrafted fake facial images, and 93.99% on detecting small GAN-generated fake images, gaining 1.98% F1-score and 10.44% AUROC compared to the best performing state-of-the-art classifier. This study is targeted for developing an automated defense mechanism to combat fake images used in different online services and applications, leveraging our state-of-the-art hand-crafted fake facial dataset (HFM) and the neural network classifier Shallow-FakeFaceNet (SFFN). In addition, our work presents various experimental results that can help guide better applied soft computing research in the future to effectively combat and detect human and GAN-generated fake face images.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.asoc.2021.107256

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 15 references

Research works printed but not linked: 21

  1. W. Xiong, J. Yu, Z. Lin, J. Yang, X. Lu, C. Barnes, J. Luo, Foreground-aware image inpainting, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 5840–5848. · DOI · Paper page
  2. K. He, X. Zhang, S. Ren, J. Sun, Delving deep into rectifiers: Surpassing human-level performance on imagenet classification, in: Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1026–1034. · DOI · Paper page
  3. Tariq, 2018, Detecting both machine and human created fake face images in the wild, 81, 10.1145/3267357.3267367 · DOI · Paper page
  4. Tariq, 2019, GAN Is a friend or foe? A framework to detect various fake face images, 1296, 10.1145/3297280.3297410 · DOI · Paper page
  5. M. Huh, A. Liu, A. Owens, A.A. Efros, Fighting fake news: Image splice detection via learned self-consistency, in: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 101–117. · DOI · Paper page
  6. Z. Liu, P. Luo, X. Wang, X. Tang, Deep learning face attributes in the wild, in: Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 3730–3738. · DOI · Paper page
  7. Yang, 2015, Estimating JPEG compression history of bitmaps based on factor histogram, Digit. Signal Process., 41, 90, 10.1016/j.dsp.2015.03.014 · DOI
  8. Farid, 2009, Exposing digital forgeries from JPEG ghosts, IEEE Trans. Inf. Forensics Secur., 4, 154, 10.1109/TIFS.2008.2012215 · DOI · Paper page
  9. Krawetz, 2007, A picture’s worth, Hacker Factor Solut., 6, 2
  10. P. Zhou, X. Han, V.I. Morariu, L.S. Davis, Learning rich features for image manipulation detection, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 1053–1061. · DOI · Paper page
  11. J.-Y. Zhu, T. Park, P. Isola, A.A. Efros, Unpaired image-to-image translation using cycle-consistent adversarial networks, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2223–2232. · DOI · Paper page
  12. Zerdoumi, 2018, Image pattern recognition in big data: taxonomy and open challenges: survey, Multimedia Tools Appl., 77, 10091, 10.1007/s11042-017-5045-7 · DOI
  13. F. Chollet, Xception: Deep learning with depthwise separable convolutions, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1251–1258. · DOI · Paper page
  14. Lin, 2009, Fast, automatic and fine-grained tampered JPEG image detection via DCT coefficient analysis, Pattern Recognit., 42, 2492, 10.1016/j.patcog.2009.03.019 · DOI · Paper page
  15. Schlegl, 2019, F-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks, Med. Image Anal., 54, 30, 10.1016/j.media.2019.01.010 · DOI · Paper page
  16. H. Jeon, Y. Bang, S.S. Woo, Faketalkerdetect: Effective and practical realistic neural talking head detection with a highly unbalanced dataset, in: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2015.
  17. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, Rethinking the inception architecture for computer vision, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2818–2826. · DOI · Paper page
  18. Eaton-Rosen, 2018, Improving data augmentation for medical image segmentation, Open Rev. Med. Imag. Deep Learn.
  19. S.-Y. Wang, O. Wang, A. Owens, R. Zhang, A.A. Efros, Detecting photoshopped faces by scripting photoshop, in: Proceedings of the IEEE International Conference on Computer Vision, 2019, pp. 10072–10081. · DOI · Paper page
  20. R.R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, D. Batra, Grad-cam: Visual explanations from deep networks via gradient-based localization, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 618–626. · DOI · Paper page
  21. C. Han, K. Murao, T. Noguchi, Y. Kawata, F. Uchiyama, L. Rundo, H. Nakayama, S. Satoh, Learning more with less: conditional PGGAN-based data augmentation for brain metastases detection using highly-rough annotation on MR images, in: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019, pp. 119–127. · DOI

Cited by

See all 93

Metadata

Authors
Lee, S., Tariq, S., Shin, Y., & Woo, S. S.
Venue
Applied Soft Computing
Publisher
Elsevier BV
Published
2021-07
Volume
105
Pages
107256
ISSN
1568-4946 (ISSN-L)
OpenAlex
W3135133133
Cited by
93
References
15
FWCI
6.17
Type
journal article
Language
en
Open access
closed
Funders
  • National Research Foundation of Korea (2020R1C1C1006004)
  • Institute for Information and Communications Technology Promotion (2019-0-01343, 2019-0-01579, 2021-0-00017)
  • Ministry of Science and ICT, South Korea

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Detecting handcrafted facial image manipulations and GAN-generated facial images using Shallow-FakeFaceNet. https://app.cochat.ai/papers/10.1016/j.asoc.2021.107256
The paper
Lee, S., Tariq, S., Shin, Y., & Woo, S. S. (2021). Detecting handcrafted facial image manipulations and GAN-generated facial images using Shallow-FakeFaceNet. Applied Soft Computing. https://doi.org/10.1016/j.asoc.2021.107256

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.