On The Detection of Synthetic Images Generated by Diffusion Models
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Over the past decade, there has been tremendous progress in creating synthetic media, mainly thanks to the development of powerful methods based on generative adversarial networks (GAN). Very recently, methods based on diffusion models (DM) have been gaining the spotlight. In addition to providing an impressive level of photorealism, they enable the creation of text-based visual content, opening up new and exciting opportunities in many different application fields, from arts to video games. On the other hand, this property is an additional asset in the hands of malicious users, who can generate and distribute fake media perfectly adapted to their attacks, posing new challenges to the media forensic community. With this work, we seek to understand how difficult it is to distinguish synthetic images generated by diffusion models from pristine ones and whether current state-of-the-art detectors are suitable for the task. To this end, first we expose the forensics traces left by diffusion models, then study how current detectors, developed for GAN-generated images, perform on these new synthetic images, especially in challenging social-network scenarios involving image compression and resizing. Datasets and code are available at https:github.com/grip-unina/DMimageDetection.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Generalization capability is a critical concern as detectors trained on specific datasets often fail to maintain performance when exposed to unseen data distributions, novel manipulation techniques, or different generator architectures [17-20].”
From Deepfake detection across image, video, and audio: a comprehensive survey with empirical evaluation of generalization and robustness (2026) · cites this paper as [19]
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662206 · Cited by 8,983 · cited 1×
“…the GAN fingerprint F , and a random noise component W i , so the fingerprint is estimated by simply averaging a large number of image residuals In this work we use the same procedure and rely on the denoising filter proposed in [28], which proved successful for camera fingerprint extraction [29].”
MethodAnalyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00813 · Cited by 6,332
Analyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2019) · arXiv · doi:10.48550/arxiv.1912.04958 · Cited by 6,332
Progressive Growing of GANs for Improved Quality, Stability, and Variation
Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017) · arXiv · doi:10.48550/arxiv.1710.10196 · Cited by 4,727
Large Scale GAN Training for High Fidelity Natural Image Synthesis
Brock, A., Donahue, J., & Simonyan, K. (2018) · arXiv · doi:10.48550/arxiv.1809.11096 · Cited by 3,717
Diffusion Models Beat GANs on Image Synthesis
Dhariwal, P., & Nichol, A. (2021) · arXiv · doi:10.48550/arxiv.2105.05233 · Cited by 3,362
Taming Transformers for High-Resolution Image Synthesis
Esser, P., Rombach, R., & Ommer, B. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01268 · Cited by 2,514 · cited 5×
“Fig.2 shows the . amplitude of these spectra for several architectures of interest, both GANs [20, 21, 22] or VQ-GANs [23, 24] (top row), and DMs [5, 25, 4, 3] (bottom row).”
BackgroundMethodInfluentialTaming Transformers for High-Resolution Image Synthesis
Esser, P., Rombach, R., & Ommer, B. (2020) · arXiv · doi:10.48550/arxiv.2012.09841 · Cited by 2,514
Hierarchical Text-Conditional Image Generation with CLIP Latents
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06125 · Cited by 2,511
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., & Chen, M. (2021) · arXiv · doi:10.48550/arxiv.2112.10741 · Cited by 1,357
Digital Camera Identification From Sensor Pattern Noise
Luka, J., Fridrich, J., & Goljan, M. (2006) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2006.873602 · Cited by 1,328
CNN-Generated Images Are Surprisingly Easy to Spot… for Now
Wang, S.-Y., Wang, O., Zhang, R., Owens, A., & Efros, A. A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00872 · Cited by 1,210
Efficient Geometry-aware 3D Generative Adversarial Networks
Chan, E. R., Lin, C. Z., Chan, M. A., Nagano, K., Pan, B., de Mello, S., Gallo, O., Guibas, L., Tremblay, J., Khamis, S., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01565 · Cited by 1,093
Self-citationUCID: an uncompressed color image database:
Schaefer, G., & Stich, M. (2003) · SPIE Proceedings · doi:10.1117/12.525375 · Cited by 1,071
Media Forensics and DeepFakes: An Overview
Verdoliva, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.3002101 · Cited by 979 · cited 1×
“These traces are different from those typical of modern digital devices enabling fake image detection [6].”
BackgroundSelf-citationAlias-Free Generative Adversarial Networks
Karras, T., Aittala, M., Laine, S., Härkönen, E., Hellsten, J., Lehtinen, J., & Aila, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.12423 · Cited by 840
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2021) · arXiv · doi:10.48550/arxiv.2112.10752 · Cited by 737
Detecting and Simulating Artifacts in GAN Fake Images
Zhang, X., Karaman, S., & Chang, S.-F. (2019) · 2019 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs47025.2019.9035107 · Cited by 567 · cited 3×
“We compare the following detectors: Spec [9], based on frequency analysis, PatchForensics [18], that relies on local patch analysis, Wang2020 [15], a ResNet50 with blurring and compression augmentation, and Grag2021 [16], same backbone as before but avoiding down-sampling in the first layer and intense augmentation.”
BackgroundMethodNoiseprint: A CNN-Based Camera Model Fingerprint
Cozzolino, D., & Verdoliva, L. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2019.2916364 · Cited by 499 · cited 1×
“In this work we use the same procedure, using the denoising filter proposed in [28], which proved already successful for camera fingerprint extraction [29].”
MethodSelf-citationAttributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints
Yu, N., Davis, L., & Fritz, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00765 · Cited by 437
Leveraging Frequency Analysis for Deep Fake Image Recognition
Frank, J., Eisenhofer, T., Schönherr, L., Fischer, A., Kolossa, D., & Holz, T. (2020) · arXiv · doi:10.48550/arxiv.2003.08685 · Cited by 390 · cited 1×
“The presence and distinctiveness of such traces have been proven by extracting a sort of spatial-domain artificial fingerprints [7, 8], but also through frequency-domain analyses showing that the upsampling operation performed in most GAN architectures give rise to clear spectral peaks [9, 10].”
BackgroundDo GANs leave artificial fingerprints?
Marra, F., Gragnaniello, D., Verdoliva, L., & Poggi, G. (2019) · 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr.2019.00103 · Cited by 361
Self-citationWhat Makes Fake Images Detectable? Understanding Properties that Generalize
Chai, L., Bau, D., Lim, S.-N., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58574-7_7 · Cited by 357
Are GAN Generated Images Easy to Detect? A Critical Analysis of the State-Of-The-Art
Gragnaniello, D., Cozzolino, D., Marra, F., Poggi, G., & Verdoliva, L. (2021) · 2021 IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme51207.2021.9428429 · Cited by 212
Self-citationDE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Models
Sha, Z., Li, Z., Yu, N., & Zhang, Y. (2023) · Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3576915.3616588 · Cited by 175
Detecting Gan-Generated Images by Orthogonal Training of Multiple CNNs
Mandelli, S., Bonettini, N., Bestagini, P., & Tubaro, S. (2022) · Proceedings - International Conference on Image Processing · doi:10.1109/icip46576.2022.9897310 · Cited by 115
Fusing Global and Local Features for Generalized AI-Synthesized Image Detection
Ju, Y., Jia, S., Ke, L., Xue, H., Nagano, K., & Lyu, S. (2022) · 2022 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip46576.2022.9897820 · Cited by 94
Self-citationLighting (In)consistency of Paint by TextPreprint
Farid, H. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.13744 · Cited by 55
Perspective (In)consistency of Paint by TextPreprint
Farid, H. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.14617 · Cited by 48
Research works printed but not linked: 2
- Corvi, 2022, IEEE Video and Image Processing Cup
- Platt, 1999, Probabilistic outputs for support vector machines and comparison to regularized likelihood methods, Advances in Large Margin Classiers
Cited by
Multi-Concept Customization of Text-to-Image Diffusion
Kumari, N., Zhang, B., Zhang, R., Shechtman, E., & Zhu, J.-Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00192 · Cited by 674
Zero-shot Image-to-Image Translation
Parmar, G., Kumar Singh, K., Zhang, R., Li, Y., Lu, J., & Zhu, J.-Y. (2023) · Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Proceedings · doi:10.1145/3588432.3591513 · Cited by 361
DIRE for Diffusion-Generated Image Detection
Wang, Z., Bao, J., Zhou, W., Wang, W., Hu, H., Chen, H., & Li, H. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02051 · Cited by 354
Raising the Bar of AI-generated Image Detection with CLIP
Cozzolino, D., Poggi, G., Corvi, R., Nießner, M., & Verdoliva, L. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00439 · Cited by 310
Self-citationCIFAKE: Image Classification and Explainable Identification of AI-Generated Synthetic Images
Bird, J. J., & Lotfi, A. (2024) · IEEE Access · doi:10.1109/access.2024.3356122 · Cited by 265
Metadata
- Authors
- Corvi, R., Cozzolino, D., Zingarini, G., Poggi, G., Nagano, K., & Verdoliva, L.
- Affiliations
- University of Naples Federico II (IT)
- Nvidia (United Kingdom) (GB)
- Venue
- ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Publisher
- IEEE
- Published
- 2023-06-04
- Pages
- 1-5
- ISSN
- 1520-6149 (ISSN-L)
- OpenAlex
- W4375869427
- Cited by
- 304
- References
- 29
- Citations by year
- 2026: 75 · 2025: 114 · 2024: 93 · 2023: 22
- FWCI
- 32.47
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Acoustics Speech and Signal Processing (publishedVersion)
- Funders
- Ministry of Education
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- synthetic image detection, diffusion models, deepfakes, multimedia forensics, generative adversarial networks, fake media, image compression, image scaling, social networks, text-to-image generation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.