CoChat
Sign Up
PreprintCited by 3,438Generative Adversarial Networks and Image SynthesisAdvanced Neural Network ApplicationsAdversarial Robustness in Machine Learning

Spectral Normalization for Generative Adversarial Networks

Open access
Miyato, Takeru, Kataoka, Toshiki, Koyama, Masanori, Yoshida, Yuichi·2018·arXiv
DOI 10.48550/arxiv.1802.05957arXiv 1802.05957
Open this paper in CoChat — free accountOpen PDF

Abstract

One of the challenges in the study of generative adversarial networks is the instability of its training. In this paper, we propose a novel weight normalization technique called spectral normalization to stabilize the training of the discriminator. Our new normalization technique is computationally light and easy to incorporate into existing implementations. We tested the efficacy of spectral normalization on CIFAR10, STL-10, and ILSVRC2012 dataset, and we experimentally confirmed that spectrally normalized GANs (SN-GANs) is capable of generating images of better or equal quality relative to the previous training stabilization techniques.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1802.05957

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The capsule network method yields the best performance compared to its competing methods in all of the four datasets.

    “The proposed method yields the best performance compared to its competing methods in all of these datasets.”

    From Deep learning for deepfakes creation and detection: A survey (2022) · cites this paper as [123]

    The Experiments section (p. 5) identifies the method as spectral normalization for GANs and names CIFAR-10, STL-10, and ImageNet—not a capsule network evaluated on four datasets. Results on CIFAR-10 and STL-10 (p. 7) say its best scores exceeded almost all other methods, which is narrower than best performance against every competitor. (LLM, conf=0.88)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

References 24 · Cited by 3,438

Research works printed but not linked: 40

  1. Martin Arjovsky and L´ eon Bottou. Towards principled methods for training generative adversarial networks. In ICLR , 2017.
  2. Martin Arjovsky, Soumith Chintala, and L´ eon Bottou. Wasserstein generative adversarial networks. In ICML , pp. 214-223, 2017.
  3. Devansh Arpit, Yingbo Zhou, Bhargava U Kota, and Venu Govindaraju. Normalization propagation: A parametric technique for removing internal covariate shift in deep networks. In ICML , pp. 1168-1176, 2016.
  4. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450 , 2016. · arXiv
  5. Andrew Brock, Theodore Lim, James M Ritchie, and Nick Weston. Neural photo editing with introspective adversarial networks. arXiv preprint arXiv:1609.07093 , 2016. · arXiv
  6. Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In AISTATS , pp. 215-223, 2011.
  7. Harm de Vries, Florian Strub, J´ er´ emie Mary, Hugo Larochelle, Olivier Pietquin, and Aaron C Courville. Modulating early visual processing by language. In NIPS , pp. 6576-6586, 2017.
  8. DC Dowson and BV Landau. The fr´ echet distance between multivariate normal distributions. Journal of Multivariate Analysis , 12(3):450-455, 1982.
  9. Vincent Dumoulin, Jonathon Shlens, and Manjunath Kudlur. A learned representation for artistic style. In ICLR , 2017.
  10. Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse rectifier neural networks. In AISTATS , pp. 315-323, 2011.
  11. Gene H Golub and Henk A Van der Vorst. Eigenvalue computation in the 20th century. Journal of Computational and Applied Mathematics , 123(1):35-65, 2000.
  12. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NIPS , pp. 2672-2680, 2014.
  13. Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, G¨ unter Klambauer, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a nash equilibrium. arXiv preprint arXiv:1706.08500 , 2017. · arXiv
  14. Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML , pp. 448-456, 2015.
  15. Kevin Jarrett, Koray Kavukcuoglu, Marc'Aurelio Ranzato, and Yann LeCun. What is the best multi-stage architecture for object recognition? In ICCV , pp. 2146-2153, 2009.
  16. Kui Jia, Dacheng Tao, Shenghua Gao, and Xiangmin Xu. Improving training of deep neural networks via singular value bounding. In CVPR , 2017.
  17. Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR , 2015.
  18. Jiwei Li, Will Monroe, Tianlin Shi, Alan Ritter, and Dan Jurafsky. Adversarial learning for neural dialogue generation. In EMNLP , pp. 2147-2159, 2017.
  19. Jae Hyun Lim and Jong Chul Ye. Geometric GAN. arXiv preprint arXiv:1705.02894 , 2017. · arXiv
  20. Andrew L Maas, Awni Y Hannun, and Andrew Y Ng. Rectifier nonlinearities improve neural network acoustic models. In ICML Workshop on Deep Learning for Audio, Speech and Language Processing , 2013.
  21. Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 , 2014. · arXiv
  22. Takeru Miyato and Masanori Koyama. cGANs with projection discriminator. In ICLR , 2018.
  23. Shakir Mohamed and Balaji Lakshminarayanan. Learning in implicit generative models. NIPS Workshop on Adversarial Training , 2017.
  24. Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. In ICML , pp. 807-814, 2010.
  25. Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-GAN: Training generative neural samplers using variational divergence minimization. In NIPS , pp. 271-279, 2016.
  26. Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxiliary classifier GANs. In ICML , pp. 2642-2651, 2017.
  27. Guo-Jun Qi. Loss-sensitive generative adversarial networks on lipschitz densities. arXiv preprint arXiv:1701.06264 , 2017. · arXiv
  28. Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. In ICLR , 2016.
  29. Masaki Saito, Eiichi Matsumoto, and Shunta Saito. Temporal generative adversarial nets with singular value clipping. In ICCV , 2017.
  30. Tim Salimans and Diederik P Kingma. Weight normalization: A simple reparameterization to accelerate training of deep neural networks. In NIPS , pp. 901-909, 2016.
  31. Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training GANs. In NIPS , pp. 2226-2234, 2016.
  32. Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In CVPR , pp. 1-9, 2015.
  33. Seiya Tokui, Kenta Oono, Shohei Hido, and Justin Clayton. Chainer: a next-generation open source framework for deep learning. In Proceedings of workshop on machine learning systems (LearningSys) in the twentyninth annual conference on neural information processing systems (NIPS) , 2015.
  34. Antonio Torralba, Rob Fergus, and William T Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence , 30 (11):1958-1970, 2008.
  35. Dustin Tran, Rajesh Ranganath, and David M Blei. Deep and hierarchical implicit models. arXiv preprint arXiv:1702.08896 , 2017. · arXiv
  36. Masatoshi Uehara, Issei Sato, Masahiro Suzuki, Kotaro Nakayama, and Yutaka Matsuo. Generative adversarial nets from a density ratio estimation perspective. NIPS Workshop on Adversarial Training , 2016.
  37. David Warde-Farley and Yoshua Bengio. Improving generative adversarial networks with denoising feature matching. In ICLR , 2017.
  38. Sitao Xiang and Hao Li. On the effect of batch normalization and weight normalization in generative adversarial networks. arXiv preprint arXiv:1704.03971 , 2017. · arXiv
  39. Jianwei Yang, Anitha Kannan, Dhruv Batra, and Devi Parikh. LR-GAN: Layered recursive generative adversarial networks for image generation. ICLR , 2017.
  40. Yuichi Yoshida and Takeru Miyato. Spectral norm regularization for improving the generalizability of deep learning. arXiv preprint arXiv:1705.10941 , 2017. · arXiv

Authors
Miyato, T., Kataoka, T., Koyama, M., & Yoshida, Y.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2018
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1802.05957
arXiv
1802.05957
OpenAlex
W2785678896
Cited by
3,438
References
24
Citations by year
2026: 60 · 2025: 181 · 2024: 348 · 2023: 421 · 2022: 405 · 2021: 805 · 2020: 675 · 2019: 428 · 2018: 111 · 2017: 4
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
spectral normalization, generative adversarial networks, weight normalization, training instability, image generation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review