CoChat
Sign Up
PreprintCited by 18Digital Media Forensic DetectionGenerative Adversarial Networks and Image SynthesisAdvanced Steganography and Watermarking Techniques

The DOI matches its DataCite record.

Detecting CNN-Generated Facial Images in Real-World Scenarios

Open access
Hulzebosch, Nils, Ibrahimi, Sarah, Worring, Marcel·2020·arXiv
DOI 10.48550/arxiv.2005.05632arXiv 2005.05632
Open this paper in CoChat — free account

Abstract

Artificial, CNN-generated images are now of such high quality that humans have trouble distinguishing them from real images. Several algorithmic detection methods have been proposed, but these appear to generalize poorly to data from unknown sources, making them infeasible for real-world scenarios. In this work, we present a framework for evaluating detection methods under real-world conditions, consisting of cross-model, cross-data, and post-processing evaluation, and we evaluate state-of-the-art detection methods using the proposed framework. Furthermore, we examine the usefulness of commonly used image pre-processing methods. Lastly, we evaluate human performance on detecting CNN-generated images, along with factors that influence this performance, by conducting an online survey. Our results suggest that CNN-based detection methods are not yet robust enough to be used in real-world scenarios.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2005.05632

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 39

  1. [2][2] Michael Albright and Scott McCloskey. Source generator attribution via inversion. In CVPR Workshop on Media Forensics , pages 96-103, 2019. 3 -
  2. [3][3] Belhassen Bayar and Matthew C Stamm. A deep learning approach to universal image manipulation detection using a new convolutional layer. In ACM IHMS , 2016. 2 -
  3. [4][4] Bolin Chen, Haodong Li, and Weiqi Luo. Image processing operations identification via convolutional neural network. arXiv preprint arXiv:1709.02908 , 2017. 4 - · arXiv
  4. [5][5] Jiansheng Chen, Xiangui Kang, Ye Liu, and Z Jane Wang. Median filtering forensics based on convolutional neural networks. IEEE Signal Processing Letters , 22(11):1849-1853, 2015. 3 -
  5. [6][6] Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In CVPR , 2018. 2, 4 -
  6. [7][7] Franc ¸ois Chollet. Xception: Deep learning with depthwise separable convolutions. In CVPR , 2017. 2, 3, 4 -
  7. [12][12] Davide Cozzolino, Giovanni Poggi, and Luisa Verdoliva. Recasting residual-based local descriptors as convolutional neural networks: an application to image forgery detection. In ACM IHMS , 2017. 2, 3, 4 -
  8. [13][13] Davide Cozzolino, Justus Thies, Andreas R¨ ossler, Christian Riess, Matthias Nießner, and Luisa Verdoliva. Forensictransfer: Weakly-supervised domain adaptation for forgery detection. arXiv preprint arXiv:1812.02510 , 2018. 1, 2, 3, 4 - · arXiv
  9. [14][14] Jessica Fridrich and Jan Kodovsky. Rich models for steganalysis of digital images. TIFS , 7(3):868-882, 2012. 3, 4 -
  10. [16][16] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NeurIPS , 2014. 1, 2 -
  11. [17][17] Satoshi Iizuka, Edgar Simo-Serra, and Hiroshi Ishikawa. Globally and locally consistent image completion. ACM Transactions on Graphics (ToG) , 36(4):1-14, 2017. 2 -
  12. [18][18] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In CVPR , 2017. 2 -
  13. [19][19] Xiaohan Jin, Ye Qi, and Shangxuan Wu. Cyclegan face-off. arXiv preprint arXiv:1712.03451 , 2017. 2 - · arXiv
  14. [20][20] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In ICLR , 2018. 2, 4 -
  15. [21][21] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In CVPR , 2019. 1, 2, 4 -
  16. [22][22] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. arXiv preprint arXiv:1912.04958 , 2019. 1, 2 - · arXiv
  17. [23][23] Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. In NeurIPS , 2018. 2, 4 -
  18. [24][24] Haodong Li, Bin Li, Shunquan Tan, and Jiwu Huang. Detection of deep network generated images using disparities in color components. arXiv preprint arXiv:1808.07276 , 2018. 2, 3, 4 - · arXiv
  19. [25][25] Haodong Li, Weiqi Luo, Xiaoqing Qiu, and Jiwu Huang. Identification of various image operations using residualbased features. IEEE Transactions on Circuits and Systems for Video Technology , 28(1):31-45, 2018. 3 -
  20. [27][27] Zhengzhe Liu, Xiaojuan Qi, Jiaya Jia, and Philip Torr. Global texture enhancement for fake face detection in the wild. In CVPR , 2020. 3 -
  21. [31][31] Scott McCloskey and Michael Albright. Detecting gangenerated imagery using color cues. arXiv preprint arXiv:1812.08247 , 2018. 2 - · arXiv
  22. [32][32] Huaxiao Mo, Bolin Chen, and Weiqi Luo. Fake faces identification via convolutional neural network. In ACM IHMS , 2018. 3 -
  23. [33][33] Lakshmanan Nataraj, Tajuddin Manhar Mohammed, BS Manjunath, Shivkumar Chandrasekaran, Arjuna Flenner, Jawadul H Bappy, and Amit K Roy-Chowdhury. Detecting gan generated fake images using co-occurrence matrices. Electronic Imaging , 2019(5):532-1, 2019. 3, 4 -
  24. [34][34] Sophie J Nightingale, Kimberley A Wade, and Derrick G Watson. Can people identify original and manipulated photos of real-world scenes? Cognitive research: principles and implications , 2(1):30, 2017. 3 -
  25. [35][35] Taesung Park, Ming-Yu Liu, Ting-Chun Wang, and Jun-Yan Zhu. Semantic image synthesis with spatially-adaptive normalization. In CVPR , 2019. 2 -
  26. [37][37] Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In CVPR , 2016. 2 -
  27. [38][38] Tom´ aˇ s Pevny, Patrick Bas, and Jessica Fridrich. Steganalysis by subtractive pixel adjacency matrix. IEEE Transactions on information Forensics and Security , 5(2):215-224, 2010. 3 -
  28. [42][42] Andreas R¨ ossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. Faceforensics++: Learning to detect manipulated facial images. In ICCV , 2019. 3, 4 -
  29. [43][43] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. IJCV , 115(3):211-252, 2015. 3 -
  30. [45][45] Kenneth Sullivan, Upamanyu Madhow, Shivkumar Chandrasekaran, and BS Manjunath. Steganalysis for markov cover data with applications to images. IEEE Transactions on Information Forensics and Security , 1(2):275-287, 2006. 3 -
  31. [47][47] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In CVPR , 2015. 4 -
  32. [48][48] Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. Cnn-generated images are surprisingly easy to spot... for now. In CVPR , 2020. 3 -
  33. [49][49] Xin Yang, Yuezun Li, Honggang Qi, and Siwei Lyu. Exposing gan-synthesized faces using landmark locations. In ACM IHMS , 2019. 2 -
  34. [50][50] Raymond A Yeh, Chen Chen, Teck Yian Lim, Alexander G Schwing, Mark Hasegawa-Johnson, and Minh N Do. Semantic image inpainting with deep generative models. In CVPR , 2017. 2 -
  35. [51][51] Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. Generative image inpainting with contextual attention. In CVPR , 2018. 2 -
  36. [52][52] Ning Yu, Larry S Davis, and Mario Fritz. Attributing fake images to gans: Learning and analyzing gan fingerprints. In CVPR , 2019. 3 -
  37. [53][53] Lilei Zheng, Ying Zhang, and Vrizlynn LL Thing. A survey on image tampering and its detection in real-world photos. Journal of Visual Communication and Image Representation , 58:380-399, 2019. 3 -
  38. [54][54] Sharon Zhou, Mitchell Gordon, Ranjay Krishna, Austin Narcomey, Li F Fei-Fei, and Michael Bernstein. Hype: A benchmark for human eye perceptual evaluation of generative models. In NeurIPS , 2019. 3, 4 -
  39. [55][55] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycleconsistent adversarial networks. In ICCV , 2017. 2

Cited by

Sorted by citations

See all 18

Metadata

Authors
Hulzebosch, N., Ibrahimi, S., & Worring, M.
Affiliations
University of Amsterdam (NL)
ORCID
Worring, Marcel
Venue
arXiv
Publisher
arXiv
Published
2020
Pages
2729-2738
DOI
10.48550/arxiv.2005.05632
arXiv
2005.05632
OpenAlex
W3025515273
Cited by
18
References
0
Citations by year
2026: 2 · 2024: 3 · 2023: 2 · 2022: 5 · 2021: 4 · 2020: 2
FWCI
1.00
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (publishedVersion)
  • UvA-DARE (University of Amsterdam) (publishedVersion, other-oa)
  • arXiv (Cornell University) (submittedVersion)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
Keywords
deepfakes, generative adversarial networks, image forgery detection, cross-model evaluation, post-processing, image preprocessing, online surveys

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review