CoChat
Sign Up
PreprintCited by 390Digital Media Forensic DetectionGenerative Adversarial Networks and Image SynthesisAdvanced Image Processing Techniques

Leveraging Frequency Analysis for Deep Fake Image Recognition

Open access
Frank, Joel, Eisenhofer, Thorsten, Schönherr, Lea, Fischer, Asja, Kolossa, Dorothea, Holz, Thorsten·2020·arXiv
DOI 10.48550/arxiv.2003.08685arXiv 2003.08685
Open this paper in CoChat — free accountOpen PDF

Abstract

Deep neural networks can generate images that are astonishingly realistic, so much so that it is often hard for humans to distinguish them from actual photos. These achievements have been largely made possible by Generative Adversarial Networks (GANs). While deep fake images have been thoroughly investigated in the image domain - a classical approach from the area of image forensics - an analysis in the frequency domain has been missing so far. In this paper, we address this shortcoming and our results reveal that in frequency space, GAN-generated images exhibit severe artifacts that can be easily identified. We perform a comprehensive analysis, showing that these artifacts are consistent across different neural network architectures, data sets, and resolutions. In a further investigation, we demonstrate that these artifacts are caused by upsampling operations found in all current GAN architectures, indicating a structural and fundamental problem in the way images are generated via GANs. Based on this analysis, we demonstrate how the frequency representation can be used to identify deep fake images in an automated way, surpassing state-of-the-art methods.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2003.08685

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

Cited by 390

Research works printed but not linked: 49

  1. Arjovsky, M., Chintala, S., and Bottou, L. Wasserstein gan. In International Conference on Machine Learning (ICML) , 2017.
  2. Azulay, A. and Weiss, Y. Why do deep convolutional networks generalize so poorly to small image transformations? arXiv preprint arXiv:1805.12177 , 2018. · arXiv
  3. Bappy, J. H., Roy-Chowdhury, A. K., Bunk, J., Nataraj, L., and Manjunath, B. Exploiting spatial structure for localizing manipulated image regions. In IEEE International Conference on Computer Vision (ICCV) , 2017.
  4. Bayar, B. and Stamm, M. C. A deep learning approach to universal image manipulation detection using a new convolutional layer. In ACM Workshop on Information Hiding and Multimedia Security , 2016.
  5. Bellemare, M. G., Danihelka, I., Dabney, W., Mohamed, S., Lakshminarayanan, B., Hoyer, S., and Munos, R. The cramer distance as a solution to biased wasserstein gradients. arXiv preprint arXiv:1705.10743 , 2017. · arXiv
  6. Bestagini, P., Milani, S., Tagliasacchi, M., and Tubaro, S. Local tampering detection in video sequences. In IEEE International Workshop on Multimedia Signal Processing (MMSP) , 2013.
  7. Bi´ nkowski, M., Sutherland, D. J., Arbel, M., and Gretton, A. Demystifying mmd gans. In International Conference on Learning Representations (ICLR) , 2018.
  8. Brock, A., Donahue, J., and Simonyan, K. Large scale GAN training for high fidelity natural image synthesis. In International Conference on Learning Representations (ICLR) , 2019.
  9. Carlini, N. and Farid, H. Evading deepfake-image detectors with white-and black-box attacks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops , 2020.
  10. Cozzolino, D., Poggi, G., and Verdoliva, L. Recasting residual-based local descriptors as convolutional neural networks: an application to image forgery detection. In ACM Workshop on Information Hiding and Multimedia Security , 2017.
  11. Durall, R., Keuper, M., and Keuper, J. Watch your upconvolution: Cnn based generative deep neural networks are failing to reproduce spectral distributions. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2020.
  12. Field, D. J. Relations between the statistics of natural images and the response properties of cortical cells. Journal of the Optical Society of America. A, Optics and image science , 1987.
  13. Field, D. J. Wavelets, vision and the statistics of natural scenes. Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences , 1999.
  14. Fridrich, J. Digital image forensics. IEEE Signal Processing Magazine , 2009.
  15. Fried, O., Tewari, A., Zollh¨ ofer, M., Finkelstein, A., Shechtman, E., Goldman, D. B., Genova, K., Jin, Z., Theobalt, C., and Agrawala, M. Text-based editing of talking-head video. ACM Transactions on Graphics (TOG) , 2019.
  16. Gonzalez, R. C. and Woods, R. E. Digital Image Processing . Pearson, 1992.
  17. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial nets. In Advances in Neural Information Processing Systems (NeurIPS) , 2014.
  18. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A. C. Improved training of wasserstein gans. In Advances in Neural Information Processing Systems (NeurIPS) , 2017.
  19. Hao, K. The biggest threat of deepfakes isnt the deepfakes themselves. MIT Technology Review , 2019.
  20. Karras, T., Aila, T., Laine, S., and Lehtinen, J. Progressive growing of GANs for improved quality, stability, and variation. In International Conference on Learning Representations (ICLR) , 2018.
  21. Karras, T., Laine, S., and Aila, T. A style-based generator architecture for generative adversarial networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2019.
  22. Khosla, A., Jayadevaprakash, N., Yao, B., and Fei-Fei, L. Novel dataset for fine-grained image categorization. In First Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2011.
  23. Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR) , 2015.
  24. Kumar, K., Kumar, R., de Boissiere, T., Gestin, L., Teoh, W. Z., Sotelo, J., de Br´ ebisson, A., Bengio, Y., and Courville, A. C. Melgan: Generative adversarial networks for conditional waveform synthesis. In Advances in Neural Information Processing Systems (NeurIPS) , 2019.
  25. Liu, Z., Luo, P., Wang, X., and Tang, X. Deep learning face attributes in the wild. In IEEE International Conference on Computer Vision (ICCV) , December 2015.
  26. Luk´ aˇ s, J., Fridrich, J., and Goljan, M. Digital camera identification from sensor pattern noise. IEEE Transactions on Information Forensics and Security , 2006.
  27. Lyu, S. Natural image statistics in digital image forensics. In Digital Image Forensics , pp. 239-256. Springer, 2013.
  28. Marra, F., Gragnaniello, D., Cozzolino, D., and Verdoliva, L. Detection of gan-generated fake images over social networks. In IEEE Conference on Multimedia Information Processing and Retrieval (MIPR) , 2018.
  29. Marra, F., Gragnaniello, D., Verdoliva, L., and Poggi, G. Do gans leave artificial fingerprints? In IEEE Conference on Multimedia Information Processing and Retrieval (MIPR) , 2019.
  30. McCloskey, S. and Albright, M. Detecting gangenerated imagery using color cues. arXiv preprint arXiv:1812.08247 , 2018. · arXiv
  31. Mirza, M. and Osindero, S. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 , 2014. · arXiv
  32. Miyato, T., Kataoka, T., Koyama, M., and Yoshida, Y. Spectral normalization for generative adversarial networks. International Conference on Learning Representations (ICLR) , 2018.
  33. Mo, H., Chen, B., and Luo, W. Fake faces identification via convolutional neural network. In ACM Workshop on Information Hiding and Multimedia Security , 2018.
  34. Petzka, H., Fischer, A., and Lukovnicov, D. On the regularization of wasserstein gans. In International Conference on Learning Representations (ICLR) , 2018.
  35. Radford, A., Metz, L., and Chintala, S. Unsupervised representation learning with deep convolutional generative adversarial networks. In International Conference on Learning Representations (ICLR) , 2016.
  36. Razavi, A., van den Oord, A., and Vinyals, O. Generating diverse high-fidelity images with vq-vae-2. In Advances in Neural Information Processing Systems (NeurIPS) , 2019.
  37. Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., and Chen, X. Improved techniques for training gans. In Advances in Neural Information Processing Systems (NeurIPS) , 2016.
  38. Sirovich, L. and Kirby, M. Low-dimensional procedure for the characterization of human faces. Journal of the Optical Society of America. A, Optics and image science , 1987.
  39. Song, L., Wu, W., Qian, C., He, R., and Loy, C. C. Everybody's talkin': Let me talk as you want. arXiv preprint arXiv:2001.05201 , 2020. · arXiv
  40. Tariq, S., Lee, S., Kim, H., Shin, Y., and Woo, S. S. Gan is a friend or foe? a framework to detect various fake face images. In ACM/SIGAPP Symposium on Applied Computing , 2019.
  41. Valle, R., Cai, W., and Doshi, A. Tequilagan: How to easily identify gan samples. arXiv preprint arXiv:1807.04919 , 2018. · arXiv
  42. van den Oord, A., Vinyals, O., et al. Neural discrete representation learning. In Advances in Neural Information Processing Systems (NeurIPS) , 2017.
  43. Wang, S.-Y., Wang, O., Zhang, R., Owens, A., and Efros, A. A. Cnn-generated images are surprisingly easy to spot... for now. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2020.
  44. Wen-Hsiung Chen, Smith, C., and Fralick, S. A fast computational algorithm for the discrete cosine transform. IEEE Transactions on Communications , 1977.
  45. Yu, F., Zhang, Y., Song, S., Seff, A., and Xiao, J. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365 , 2015. · arXiv
  46. Yu, N., Davis, L. S., and Fritz, M. Attributing fake images to gans: Learning and analyzing gan fingerprints. In IEEE International Conference on Computer Vision (ICCV) , 2019a.
  47. Zhang, R. Making convolutional networks shift-invariant again. In International Conference on Machine Learning (ICML) , 2019.
  48. Zhou, P., Han, X., Morariu, V. I., and Davis, L. S. Learning rich features for image manipulation detection. In IEEE International Conference on Computer Vision (ICCV) , 2018.
  49. Zhu, J.-Y., Park, T., Isola, P., and Efros, A. A. Unpaired image-to-image translation using cycle-consistent adversarial networks. In IEEE International Conference on Computer Vision (ICCV) , 2017.

Authors
Frank, J., Eisenhofer, T., Schönherr, L., Fischer, A., Kolossa, D., & Holz, T.
Affiliations
  • Ruhr University Bochum (DE)
  • Helmholtz Center for Information Security (DE)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
Volume
1
Pages
3247-3258
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2003.08685
arXiv
2003.08685
OpenAlex
W3034946971
Cited by
390
References
0
Citations by year
2026: 1 · 2025: 63 · 2024: 114 · 2023: 95 · 2022: 53 · 2021: 55 · 2020: 9
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • Figshare (submittedVersion, cc-by)
  • arXiv (Cornell University)
  • CISPA Helmholtz Center (cc-by)
  • International Conference on Machine Learning
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image Processing Techniques
Keywords
deepfake detection, generative adversarial networks, frequency domain analysis, image forensics, spectral representation, image forgery detection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review