Fighting Deepfake by Exposing the Convolutional Traces on Images
Open accessThe DOI matches its Crossref record; 37 of the 38 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Advances in Artificial Intelligence and Image Processing are changing the way people interacts with digital images and video. Widespread mobile apps like FACEAPP make use of the most advanced Generative Adversarial Networks (GAN) to produce extreme transformations on human face photos such gender swap, aging, etc. The results are utterly realistic and extremely easy to be exploited even for non-experienced users. This kind of media object took the name of Deepfake and raised a new challenge in the multimedia forensics field: the Deepfake detection challenge. Indeed, discriminating a Deepfake from a real image could be a difficult task even for human eyes but recent works are trying to apply the same technology used for generating images for discriminating them with preliminary good results but with many limitations: employed Convolutional Neural Networks are not so robust, demonstrate to be specific to the context and tend to extract semantics from images. In this paper, a new approach aimed to extract a Deepfake fingerprint from images is proposed. The method is based on the Expectation-Maximization algorithm trained to detect and extract a fingerprint that represents the Convolutional Traces (CT) left by GANs during image generation. The CT demonstrates to have high discriminative power achieving better results than state-of-the-art in the Deepfake detection task also proving to be robust to different attacks. Achieving an overall classification accuracy of over 98%, considering Deepfakes from 10 different GAN architectures not only involved in images of faces, the CT demonstrates to be reliable and without any dependence on image semantic. Finally, tests carried out on Deepfakes generated by FACEAPP achieving 93% of accuracy in the fake detection task, demonstrated the effectiveness of the proposed technique on a real-case scenario.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Current state-of-the-art (SOTA) methods have proven to be very effective in detecting deepfake content generated by GAN architectures [93,94,95] and Diffusion Models (DMs) [96,97], demonstrating the ability to specialize not only in recognizing the architecture but also in identifying the specific pattern within the creation process.”
From Deepfake Media Forensics: Status and Future Challenges (2025) · cites this paper as [94]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The CFFN en- compasses several dense units with each unit including different numbers of dense blocks [61] to improve the representative capability for the input images.”
From Deep learning for deepfakes creation and detection: A survey (2022) · cites this paper as [61]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2008) · Journal of Machine Learning Research · Cited by 33,051
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714 · cited 3×
“The stage changes when surprising results of Deepfake images were obtained by Style Generative Adversarial Network (STYLEGAN) [13].”
MethodAnalyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00813 · Cited by 6,332
Analyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2019) · arXiv · doi:10.48550/arxiv.1912.04958 · Cited by 6,332
Progressive Growing of GANs for Improved Quality, Stability, and Variation
Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017) · arXiv · doi:10.48550/arxiv.1710.10196 · Cited by 4,727
StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., & Choo, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00916 · Cited by 4,163
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314
The expectation-maximization algorithm
Moon, T. K. (1996) · IEEE Signal Processing Magazine · doi:10.1109/79.543975 · Cited by 3,181
Semantic Image Synthesis With Spatially-Adaptive Normalization
Park, T., Liu, M.-Y., Wang, T.-C., & Zhu, J.-Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00244 · Cited by 2,974
Generative Adversarial Text to Image Synthesis
Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., & Lee, H. (2016) · arXiv · doi:10.48550/arxiv.1605.05396 · Cited by 2,494
Globally and locally consistent image completion
Iizuka, S., Simo-Serra, E., & Ishikawa, H. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073659 · Cited by 2,203 · cited 1×
“, image to image translation [6], image super resolution [7], image completion [8], and textto-image generation [9].”
BackgroundFace2Face: Real-Time Face Capture and Reenactment of RGB Videos
Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., & Niessner, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.262 · Cited by 1,747
Deferred neural rendering: image synthesis using neural textures
Thies, J., Zollhöfer, M., & Nießner, M. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323035 · Cited by 1,356
CNN-Generated Images Are Surprisingly Easy to Spot… for Now
Wang, S.-Y., Wang, O., Zhang, R., Owens, A., & Efros, A. A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00872 · Cited by 1,210
Media Forensics and DeepFakes: An Overview
Verdoliva, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.3002101 · Cited by 979
AttGAN: Facial attribute editing by only changing what you want
He, Z., Zuo, W., Kan, M., Shan, S., & Chen, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2916751 · Cited by 833
Demo of Face2Face: real-time face capture and reenactment of RGB videos
Thies, J., Zollhöfer, M., Stamminger, M., Theobalt, C., & Nießner, M. (2016) · ACM SIGGRAPH 2016 Emerging Technologies · doi:10.1145/2929464.2929475 · Cited by 589
Detecting and Simulating Artifacts in GAN Fake Images
Zhang, X., Karaman, S., & Chang, S.-F. (2019) · 2019 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs47025.2019.9035107 · Cited by 567 · cited 3×
“[17] (the authors of AutoGAN) and Wang et al.”
BackgroundInfluentialAn Overview on Image Forensics
Piva, A. (2013) · ISRN Signal Processing · doi:10.1155/2013/496701 · Cited by 426
DeepFake Detection by Analyzing Convolutional Traces
Guarnera, L., Giudice, O., & Battiato, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00341 · Cited by 374 · cited 2×
“[5] proposed a new analytical solution to extract an unique”
BackgroundMethodSelf-citationFakeSpotter: A Simple yet Robust Baseline for Spotting AI-Synthesized Fake Faces
Wang, R., Juefei-Xu, F., Ma, L., Xie, X., Huang, Y., Wang, J., & Liu, Y. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/476 · Cited by 250
Image-To-Image Translation via Group-Wise Deep Whitening-And-Coloring Transformation
Cho, W., Choi, S., Park, D. K., Shin, I., & Choo, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01089 · Cited by 154 · cited 10×
“[12] proposed the ‘‘group-wise deep whitening-and coloring method’’ (GDWCT) for a better styling capacity, obtaining a great improvement in the image translation and style transfer task in terms of computational efficiency and quality of generated images.”
BackgroundMethodInfluentialDiverse Image Synthesis From Semantic Layouts via Conditional IMLE
Li, K., Zhang, T., & Malik, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00432 · Cited by 91 · cited 7×
“[26] K. Li, T. Zhang, and J. Malik, ‘‘Diverse image synthesis from semantic layouts via conditional IMLE,’’ in Proc.”
MethodInfluentialA Classification Engine for Image Ballistics of Social Data
Giudice, O., Paratore, A., Moltisanti, M., & Battiato, S. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-68548-9_57 · Cited by 82
Self-citationDeferred Neural Rendering: Image Synthesis using Neural Textures
Thies, J., Zollhöfer, M., & Nießner, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.12356 · Cited by 46
FakeSpotter: A Simple Baseline for Spotting AI-Synthesized Fake Faces
Wang, R., Ma, L., Juefei-Xu, F., Xie, X., Wang, J., & Liu, Y. (2019) · Cited by 39
Preliminary Forensics Analysis of DeepFake Images
Guarnera, L., Giudice, O., Nastasi, C., & Battiato, S. (2020) · 2020 AEIT International Annual Conference (AEIT) · doi:10.23919/aeit50178.2020.9241108 · Cited by 38
Self-citationMultimedia Forensics: discovering the history of multimedia contents
Battiato, S., Giudice, O., & Paratore, A. (2016) · Proceedings of the 17th International Conference on Computer Systems and Technologies 2016 · doi:10.1145/2983468.2983470 · Cited by 37
Self-citationFakeSpotter: A Simple yet Robust Baseline for Spotting AI-Synthesized Fake FacesPreprint
Wang, R., Juefei-Xu, F., Ma, L., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06122 · Cited by 32
Detecting GANs and Retouching based Digital Alterations via DAD-HCNN
Jain, A., Majumdar, P., Singh, R., & Vatsa, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00344 · Cited by 29
Detecting CNN-Generated Facial Images in Real-World Scenarios
Hulzebosch, N., Ibrahimi, S., & Worring, M. (2020) · arXiv · doi:10.48550/arxiv.2005.05632 · Cited by 18
(2020) · doi:10.1109/cvprw50498.2020.00329 · Cited by 18
1-D DCT Domain Analysis for JPEG Double Compression Detection
Giudice, O., Guarnera, F., Paratore, A., & Battiato, S. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-30645-8_65 · Cited by 17
Self-citation
Cited by
Deepfake Detection: A Systematic Literature Review
Rana, M. S., Nobi, M. N., Murali, B., & Sung, A. H. (2022) · IEEE Access · doi:10.1109/access.2022.3154404 · Cited by 573
Deep learning for deepfakes creation and detection: A survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103525 · Cited by 464
“Using convolutional traces on GAN-based images [162] KNN, SVM, and LDA Training the expectation-maximization algorithm [163] to detect and extract discriminative features via a fingerprint that represents the convolutional traces left by GANs during image generation.”
MethodDeep Learning for Deepfakes Creation and Detection: A Survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2019) · arXiv · doi:10.48550/arxiv.1909.11573 · Cited by 464
Deepfake detection using deep learning methods: A systematic and comprehensive review
Heidari, A., Jafari Navimipour, N., Dag, H., & Unal, M. (2024) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1520 · Cited by 349
“Also, Guarnera et al. ( 2020) presented the completion of a previous study on deepfake image analysis.”
BackgroundUnmasking Deepfakes Advancements, Challenges, and Ethical Considerations
Kosarkar, U. P., & Sakarkar, G. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-99-8661-3_19 · Cited by 216
Metadata
- Authors
- Guarnera, L., Giudice, O., & Battiato, S.
- Affiliations
- University of Catania (IT)
- Venue
- IEEE Access
- Published
- 2020
- Volume
- 8
- Pages
- 165085-165098
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3048007723
- Cited by
- 149
- References
- 35
- Citations by year
- 2026: 19 · 2025: 32 · 2024: 47 · 2023: 23 · 2022: 19 · 2021: 9
- FWCI
- 4.96
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- iCTLab s.r.l. - Spin-off of University of Catania
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- deepfake detection, generative adversarial networks, expectation-maximization algorithm, image forensics, robustness to attacks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.