CoChat
Sign Up
PreprintCited by 6,332Generative Adversarial Networks and Image SynthesisDigital Media Forensic DetectionAdvanced Neural Network Applications

Analyzing and Improving the Image Quality of StyleGAN

Open access
Karras, Tero, Laine, Samuli, Aittala, Miika, Hellsten, Janne, Lehtinen, Jaakko, Aila, Timo·2019·arXiv
DOI 10.48550/arxiv.1912.04958arXiv 1912.04958
Open this paper in CoChat — free accountOpen PDF

Abstract

The style-based GAN architecture (StyleGAN) yields state-of-the-art results in data-driven unconditional generative image modeling. We expose and analyze several of its characteristic artifacts, and propose changes in both model architecture and training methods to address them. In particular, we redesign the generator normalization, revisit progressive growing, and regularize the generator to encourage good conditioning in the mapping from latent codes to images. In addition to improving image quality, this path length regularizer yields the additional benefit that the generator becomes significantly easier to invert. This makes it possible to reliably attribute a generated image to a particular network. We furthermore visualize how well the generator utilizes its output resolution, and identify a capacity problem, motivating us to train larger models for additional quality improvements. Overall, our improved model redefines the state of the art in unconditional image modeling, both in terms of existing distribution quality metrics as well as perceived image quality.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1912.04958

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The authors' models closely reach the performance of state-of-the-art methods such as StyleGAN2 and ADM while significantly reducing the required compute resources.

    “We closely reach the performance of state of the art methods as StyleGAN2 [42] and ADM [15] while significantly reducing the required compute resources.”

    From High-Resolution Image Synthesis with Latent Diffusion Models (2022) · cites this paper as [42]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 48

  1. Rameen Abdal, Yipeng Qin, and Peter Wonka. Im- age2StyleGAN: How to embed images into the StyleGAN latent space? In ICCV, 2019. 7
  2. Michael Albright and Scott McCloskey. Source generator attribution via inversion. In CVPR Workshops, 2019. 8
  3. Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, 2006. 17
  4. Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. CoRR, abs/1809.11096, 2018. · arXiv
  5. Yann N. Dauphin, Harm de Vries, and Yoshua Bengio. Equi- librated adaptive learning rates for non-convex optimization. CoRR, abs/1502.04390, 2015. 5 · arXiv
  6. Emily L. Denton, Soumith Chintala, Arthur Szlam, and Robert Fergus. Deep generative image models using a Laplacian pyramid of adversarial networks. CoRR, abs/1506.05751, 2015. · arXiv
  7. Vincent Dumoulin, Ethan Perez, Nathan Schucher, Flo- rian Strub, Harm de Vries, Aaron Courville, and Yoshua Bengio. Feature-wise transformations. Distill, 2018. https://distill.pub/2018/feature-wise-transformations. 1
  8. Vincent Dumoulin, Jonathon Shlens, and Manjunath Kud- lur. A learned representation for artistic style. CoRR, abs/1610.07629, 2016. 1 · arXiv
  9. Aviv Gabbay and Yedid Hoshen. Style generator in- version for image enhancement and animation. CoRR, abs/1906.11880, 2019. 7 · arXiv
  10. Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. ImageNet-trained CNNs are biased towards texture; increas- ing shape bias improves accuracy and robustness. CoRR, abs/1811.12231, 2018. 1, 4 · arXiv
  11. Golnaz Ghiasi, Honglak Lee, Manjunath Kudlur, Vincent Dumoulin, and Jonathon Shlens. Exploring the structure of a real-time, arbitrary neural artistic stylization network. CoRR, abs/1705.06830, 2017. 1 · arXiv
  12. Xavier Glorot and Yoshua Bengio. Understanding the diffi- culty of training deep feedforward neural networks. In Pro- ceedings of the Thirteenth International Conference on Arti- ficial Intelligence and Statistics, pages 249-256, 2010. 3
  13. G.H. Golub and C.F. Van Loan. Matrix Computations. Johns Hopkins Studies in the Mathematical Sciences. Johns Hop- kins University Press, 2013. 17
  14. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. In NIPS, 2014. 1, 5, 11
  15. Ishaan Gulrajani, Faruk Ahmed, Martín Arjovsky, Vincent Dumoulin, and Aaron C. Courville. Improved training of Wasserstein GANs. CoRR, abs/1704.00028, 2017. 6 · arXiv
  16. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CoRR, abs/1512.03385, 2015. 6 · arXiv
  17. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level perfor- mance on ImageNet classification. CoRR, abs/1502.01852, 2015. 3 · arXiv
  18. Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local Nash equi- librium. In Proc. NIPS, pages 6626-6637, 2017. 1
  19. Xun Huang and Serge J. Belongie. Arbitrary style trans- fer in real-time with adaptive instance normalization. CoRR, abs/1703.06868, 2017. 1 · arXiv
  20. Animesh Karnewar and Oliver Wang. MSG-GAN: multi- scale gradients for generative adversarial networks. In Proc. CVPR, 2020. 6
  21. Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of GANs for improved quality, stability, and variation. CoRR, abs/1710.10196, 2017. 1, 5, 11 · arXiv
  22. Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proc. CVPR, 2018. 1, 2, 4, 5, 11, 13, 16, 20
  23. Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015. 11, 19
  24. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. ImageNet classification with deep convolutional neural net- works. In NIPS, pages 1097-1105. 2012. 11
  25. Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall met- ric for assessing generative models. In Proc. NeurIPS, 2019. 1, 2, 4
  26. Barbara Landau, Linda B. Smith, and Susan S. Jones. The importance of shape in early lexical learning. Cognitive De- velopment, 3(3), 1988. 4
  27. Haodong Li, Han Chen, Bin Li, and Shunquan Tan. Can forensic detectors identify GAN generated images? In Proc. Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 2018. 7
  28. Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. Which training methods for GANs do actually converge? CoRR, abs/1801.04406, 2018. 5, 11 · arXiv
  29. Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative ad- versarial networks. CoRR, abs/1802.05957, 2018. 1, 5, 6, 20 · arXiv
  30. Augustus Odena, Jacob Buckman, Catherine Olsson, Tom B. Brown, Christopher Olah, Colin Raffel, and Ian Goodfellow. Is generator conditioning causally related to GAN perfor- mance? CoRR, abs/1802.08768, 2018. 5, 18 · arXiv
  31. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- Net: Convolutional networks for biomedical image segmen- tation. In Proc. Medical Image Computing and Computer- Assisted Intervention (MICCAI), pages 234-241, 2015. 6
  32. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Fei-Fei Li. ImageNet large scale visual recognition chal- lenge. In Proc. CVPR, 2015. 4
  33. Mehdi S. M. Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly. Assessing generative models via precision and recall. CoRR, abs/1806.00035, 2018. 1 · arXiv
  34. Tim Salimans and Diederik P. Kingma. Weight normaliza- tion: A simple reparameterization to accelerate training of deep neural networks. CoRR, abs/1602.07868, 2016. 3 · arXiv
  35. Yujun Shen, Jinjin Gu, Xiaoou Tang, and Bolei Zhou. Inter- preting the latent space of GANs for semantic face editing. CoRR, abs/1907.10786, 2019. 1 · arXiv
  36. Karen Simonyan and Andrew Zisserman. Very deep convo- lutional networks for large-scale image recognition. CoRR, abs/1409.1556, 2014. 1, 4 · arXiv
  37. Run Wang, Lei Ma, Felix Juefei-Xu, Xiaofei Xie, Jian Wang, and Yang Liu. FakeSpotter: A simple baseline for spotting AI-synthesized fake faces. CoRR, abs/1909.06122, 2019. 7 · arXiv
  38. Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A. Efros. CNN-generated images are surprisingly easy to spot... for now. CoRR, abs/1912.11035, 2019. 7 · arXiv
  39. Lance Williams. Pyramidal parametrics. SIGGRAPH Com- put. Graph., 17(3):1-11, 1983. 6
  40. Sitao Xiang and Hao Li. On the effects of batch and weight normalization in generative adversarial networks. CoRR, abs/1704.03971, 2017. 3 · arXiv
  41. Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianx- iong Xiao. LSUN: Construction of a large-scale image dataset using deep learning with humans in the loop. CoRR, abs/1506.03365, 2015. 11 · arXiv
  42. Ning Yu, Larry Davis, and Mario Fritz. Attributing fake im- ages to GANs: Analyzing fingerprints in generated images. CoRR, abs/1811.08180, 2018. 7 · arXiv
  43. Han Zhang, Ian Goodfellow, Dimitris Metaxas, and Augus- tus Odena. Self-attention generative adversarial networks. CoRR, abs/1805.08318, 2018. 5 · arXiv
  44. Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaolei Huang, Xiaogang Wang, and Dimitris N. Metaxas. Stack- GAN: text to photo-realistic image synthesis with stacked generative adversarial networks. In ICCV, 2017. 6
  45. Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiao- gang Wang, Xiaolei Huang, and Dimitris N. Metaxas. Stack- GAN++: realistic image synthesis with stacked generative adversarial networks. CoRR, abs/1710.10916, 2017. 6 · arXiv
  46. Richard Zhang. Making convolutional networks shift- invariant again. In Proc. ICML, 2019. 5, 11
  47. Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proc. CVPR, 2018. 4, 8, 19
  48. Xu Zhang, Svebor Karaman, and Shih-Fu Chang. Detect- ing and simulating artifacts in GAN fake images. CoRR, abs/1907.06515, 2019. · arXiv

Cited by

Sorted by citations

Authors
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T.
Affiliations
  • Nvidia (United Kingdom) (GB)
  • Aalto University (FI)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2019
Pages
8107-8116
ISSN
1063-6919 (ISSN-L)
DOI
10.48550/arxiv.1912.04958
arXiv
1912.04958
OpenAlex
W2995327724
Cited by
6,332
References
0
Citations by year
2026: 463 · 2025: 1070 · 2024: 1389 · 2023: 1362 · 2022: 1009 · 2021: 792 · 2020: 243 · 2019: 4
FWCI
262.37
Type
conference paper
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • TIB Data Manager
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
StyleGAN, image quality, unconditional image generation, model capacity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review