CoChat
Sign Up
PreprintCited by 145Digital Media Forensic DetectionAdvanced Steganography and Watermarking TechniquesGenerative Adversarial Networks and Image Synthesis

The DOI matches its DataCite record.

GenImage: A Million-Scale Benchmark for Detecting AI-Generated Image

Open access
Zhu, Mingjian, Chen, Hanting, Yan, Qiangyu, Huang, Xudong, Lin, Guanyu, Li, Wei, Tu, Zhijun, Hu, Hailin, Hu, Jie, Wang, Yunhe·2023·arXiv
DOI 10.48550/arxiv.2306.08571arXiv 2306.08571
Open this paper in CoChat — free accountOpen PDF

Abstract

The extraordinary ability of generative models to generate photographic images has intensified concerns about the spread of disinformation, thereby leading to the demand for detectors capable of distinguishing between AI-generated fake images and real images. However, the lack of large datasets containing images from the most advanced image generators poses an obstacle to the development of such detectors. In this paper, we introduce the GenImage dataset, which has the following advantages: 1) Plenty of Images, including over one million pairs of AI-generated fake images and collected real images. 2) Rich Image Content, encompassing a broad range of image classes. 3) State-of-the-art Generators, synthesizing images with advanced diffusion models and GANs. The aforementioned advantages allow the detectors trained on GenImage to undergo a thorough evaluation and demonstrate strong applicability to diverse images. We conduct a comprehensive analysis of the dataset and propose two tasks for evaluating the detection method in resembling real-world scenarios. The cross-generator image classification task measures the performance of a detector trained on one generator when tested on the others. The degraded image classification task assesses the capability of the detectors in handling degraded images such as low-resolution, blurred, and compressed images. With the GenImage dataset, researchers can effectively expedite the development and evaluation of superior AI-generated image detectors in comparison to prevailing methodologies.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2306.08571

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Diffusion-generated images exhibit fewer artifacts than GAN-generated images, while GAN-generated images display a noticeable checkerboard pattern of artifacts.

    “Particularly, diffusion-generated images exhibit fewer artifacts, while GAN-generated images display a noticeable checkerboard pattern of artifacts.”

    From DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection (2023) · cites this paper as [54]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 27

  1. Lu, Z.; Huang, D.; Bai, L.; Liu, X.; Qu, J.; Ouyang, W. Seeing is not always believing: A Quantitative Study on Human Perception of AI-Generated Images. arXiv preprint arXiv:2304.13023 2023, · arXiv
  2. Yang, X.; Li, Y.; Lyu, S. Exposing deep fakes using inconsistent head poses. ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2019; pp 8261-8265.
  3. Wang, R.; Juefei-Xu, F.; Ma, L.; Xie, X.; Huang, Y.; Wang, J.; Liu, Y. Fakespotter: A simple yet robust baseline for spotting ai-synthesized fake faces. arXiv preprint arXiv:1909.06122 2019, · arXiv
  4. Dang, H.; Liu, F.; Stehouwer, J.; Liu, X.; Jain, A. K. On the detection of digital face manipulation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern recognition. 2020; pp 5781- 5790.
  5. Gandhi, A.; Jain, S. Adversarial perturbations fool deepfake detectors. 2020 international joint conference on neural networks (IJCNN). 2020; pp 1-8.
  6. He, Y.; Gan, B.; Chen, S.; Zhou, Y.; Yin, G.; Song, L.; Sheng, L.; Shao, J.; Liu, Z. Forgerynet: A versatile benchmark for comprehensive forgery analysis. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021; pp 4360-4369.
  7. Wang, Y.; Huang, Z.; Hong, X. Benchmarking Deepart Detection. arXiv preprint arXiv:2302.14475 2023, · arXiv
  8. Wang, S.-Y.; Wang, O.; Zhang, R.; Owens, A.; Efros, A. A. CNN-generated images are surprisingly easy to spot... for now. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020; pp 8695-8704.
  9. Verdoliva, L.; Cozzolino, D.; Nagano, K. 2022 IEEE Image and Video Processing Cup Synthetic Image Detection.
  10. Sha, Z.; Li, Z.; Yu, N.; Zhang, Y. DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Diffusion Models. arXiv preprint arXiv:2210.06998 2022, · arXiv
  11. Bird, J. J.; Lotfi, A. CIFAKE: Image Classification and Explainable Identification of AI-Generated Synthetic Images. arXiv preprint arXiv:2303.14126 2023, · arXiv
  12. Karras, T.; Aila, T.; Laine, S.; Lehtinen, J. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196 2017, · arXiv
  13. Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; Fei-Fei, L. Imagenet: A large-scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition. 2009; pp 248-255.
  14. Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; Ommer, B. High-Resolution Image Synthesis With Latent Diffusion Models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022; pp 10684-10695.
  15. Brock, A.; Donahue, J.; Simonyan, K. Large scale GAN training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096 2018, · arXiv
  16. Nichol, A.; Dhariwal, P.; Ramesh, A.; Shyam, P.; Mishkin, P.; McGrew, B.; Sutskever, I.; Chen, M. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741 2021, · arXiv
  17. Gu, S.; Chen, D.; Bao, J.; Wen, F.; Zhang, B.; Chen, D.; Yuan, L.; Guo, B. Vector quantized diffusion model for text-to-image synthesis. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022; pp 10696-10706.
  18. Dhariwal, P.; Nichol, A. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems 2021, 34, 8780-8794.
  19. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016; pp 770-778.
  20. Touvron, H.; Cord, M.; Douze, M.; Massa, F.; Sablayrolles, A.; Jégou, H. Training data-efficient image transformers & distillation through attention. International conference on machine learning. 2021; pp 10347-10357.
  21. Liu, Z.; Lin, Y.; Cao, Y.; Hu, H.; Wei, Y.; Zhang, Z.; Lin, S.; Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF international conference on computer vision. 2021; pp 10012-10022.
  22. Qian, Y.; Yin, G.; Sheng, L.; Chen, Z.; Shao, J. Thinking in frequency: Face forgery detection by mining frequency-aware clues. Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XII. 2020; pp 86-103.
  23. Liu, Z.; Qi, X.; Torr, P. H. Global texture enhancement for fake face detection in the wild. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020; pp 8060-8069.
  24. Zhang, X.; Karaman, S.; Chang, S.-F. Detecting and simulating artifacts in gan fake images. 2019 IEEE international workshop on information forensics and security (WIFS). 2019; pp 1-6.
  25. Huang, G. B.; Jain, V.; Learned-Miller, E. Unsupervised Joint Alignment of Complex Images. ICCV. 2007.
  26. Schuhmann, C.; Beaumont, R.; Vencu, R.; Gordon, C.; Wightman, R.; Cherti, M.; Coombes, T.; Katta, A.; Mullis, C.; Wortsman, M., et al. Laion-5b: An open large-scale dataset for training next generation image-text models. arXiv preprint arXiv:2210.08402 2022, · arXiv
  27. Wang, Z. J.; Montoya, E.; Munechika, D.; Yang, H.; Hoover, B.; Chau, D. H. DiffusionDB: A Large-Scale Prompt Gallery Dataset for Text-to-Image Generative Models. arXiv:2210.14896 [cs] 2022, · arXiv

Cited by

Sorted by citations

Metadata

Authors
Zhu, M., Chen, H., Yan, Q., Huang, X., Lin, G., Li, W., Tu, Z., Hu, H., Hu, J., & Wang, Y.
ORCID
Wang, Yunhe
Venue
arXiv
Publisher
arXiv
Published
2023
Pages
77771-77782
ISSN
1049-5258 (ISSN-L)
DOI
10.48550/arxiv.2306.08571
arXiv
2306.08571
OpenAlex
W7133207051
Cited by
145
References
0
Citations by year
2026: 78 · 2025: 41 · 2024: 25 · 2023: 1
FWCI
14.74
Type
conference paper
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Keywords
AI-generated image detection, image forgery detection, benchmark, large-scale datasets, generative adversarial networks, deepfakes

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review