CoChat
Sign Up
PreprintCited by 1,170Generative Adversarial Networks and Image SynthesisDigital Media Forensic DetectionAdvanced Neural Network Applications

Training Generative Adversarial Networks with Limited Data

Open access
Karras, Tero, Aittala, Miika, Hellsten, Janne, Laine, Samuli, Lehtinen, Jaakko, Aila, Timo·2020·arXiv
DOI 10.48550/arxiv.2006.06676arXiv 2006.06676
Open this paper in CoChat — free accountOpen PDF

Abstract

Training generative adversarial networks (GAN) using too little data typically leads to discriminator overfitting, causing training to diverge. We propose an adaptive discriminator augmentation mechanism that significantly stabilizes training in limited data regimes. The approach does not require changes to loss functions or network architectures, and is applicable both when training from scratch and when fine-tuning an existing GAN on another dataset. We demonstrate, on several datasets, that good results are now possible using only a few thousand training images, often matching StyleGAN2 results with an order of magnitude fewer images. We expect this to open up new application domains for GANs. We also find that the widely used CIFAR-10 is, in fact, a limited data benchmark, and improve the record FID from 5.59 to 2.42.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2006.06676

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Many computer science datasets relevant to art focus on Western schools of art (e.g., WikiArt and MetFaces).

    “since many of the computer science datasets relevant to art focus on Western schools of art (i.e. WikiArt and MetFaces) [25, 47].”

    From Design Guidelines for Prompt Engineering Text-to-Image Generative Models (2022) · cites this paper as [25]

    Section 4.3 (p. 8) identifies MetFaces as the authors’ dataset of faces drawn from the Metropolitan Museum of Art collection, supporting its use as an example of an art-related computer science dataset. The paper does not establish that MetFaces focuses on Western schools of art, discuss WikiArt, or assess whether many such datasets have that focus. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Part of this paper’s reference list could not be split into single entries.

Flagged first, then by citations

Research works printed but not linked: 44

  1. [2][2] M. Arjovsky and L. Bottou. Towards principled methods for training generative adversarial networks. In Proc. ICLR, 2017.
  2. [3][3] M. Bińkowski, D. J. Sutherland, M. Arbel, and A. Gretton. Demystifying MMD GANs. In Proc. ICLR, 2018.
  3. [4][4] A. Bora, E. Price, and A. Dimakis. AmbientGAN: Generative models from lossy measurements. In Proc. ICLR, 2018.
  4. [7][7] Y. Choi, Y. Uh, J. Yoo, and J.-W. Ha. StarGAN v2: Diverse image synthesis for multiple domains. In Proc. CVPR, 2020.
  5. [8][8] E. D. Cubuk, B. Zoph, D. Mané, V. Vasudevan, and Q. V. Le. AutoAugment: Learning augmentation policies from data. In Proc. CVPR, 2019.
  6. [9][9] E. D. Cubuk, B. Zoph, J. Shlens, and Q. V. Le. RandAugment: Practical automated data augmentation with a reduced search space. CoRR, abs/1909.13719, 2019. · arXiv
  7. [11][11] T. De Vries and G. Taylor. Improved regularization of convolutional neural networks with cutout. CoRR, abs/1708.04552, 2017. · arXiv
  8. [13][13] X. Gong, S. Chang, Y. Jiang, and Z. Wang. AutoGAN: Neural architecture search for generative adversarial networks. In Proc. ICCV, 2019. 10
  9. [14][14] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial networks. In Proc. NIPS, 2014.
  10. [16][16] S. Gurumurthy, R. K. Sarvadevabhatla, and V. B. Radhakrishnan. DeLiGAN: Generative adversarial networks for diverse and limited data. In Proc. CVPR, 2017.
  11. [17][17] T. He, Z. Zhang, H. Zhang, Z. Zhang, J. Xie, and M. Li. Bag of tricks for image classification with convolutional neural networks. In Proc. CVPR, 2019.
  12. [19][19] T. Karras, T. Aila, S. Laine, and J. Lehtinen. Progressive growing of GANs for improved quality, stability, and variation. In Proc. ICLR, 2018.
  13. [20][20] T. Karras, S. Laine, and T. Aila. A style-based generator architecture for generative adversarial networks. In Proc. CVPR, 2018.
  14. [21][21] T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila. Analyzing and improving the image quality of StyleGAN. In Proc. CVPR, 2020.
  15. [22][22] I. Kavalerov, W. Czaja, and R. Chellappa. cGANs with multi-hinge loss. CoRR, abs/1912.04216, 2019. · arXiv
  16. [23][23] M. Kettunen, E. Härkönen, and J. Lehtinen. E-LPIPS: robust perceptual image similarity via random transformation ensembles. CoRR, abs/1906.03973, 2019. · arXiv
  17. [24][24] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In Proc. ICLR, 2015.
  18. [26][26] T. Kynkäänniemi, T. Karras, S. Laine, J. Lehtinen, and T. Aila. Improved precision and recall metric for assessing generative models. In Proc. NeurIPS, 2019.
  19. [27][27] S. Laine and T. Aila. Temporal ensembling for semi-supervised learning. In Proc. ICLR, 2017.
  20. [28][28] X. Mao, Q. Li, H. Xie, R. Y. K. Lau, and Z. Wang. Least squares generative adversarial networks. In Proc. ICCV, 2017.
  21. [29][29] M. Marchesi. Megapixel size image creation using generative adversarial networks. CoRR, abs/1706.00082, 2017. · arXiv
  22. [30][30] L. Mescheder, A. Geiger, and S. Nowozin. Which training methods for GANs do actually converge? In Proc. ICML, 2018.
  23. [31][31] T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida. Spectral normalization for generative adversarial networks. In Proc. ICLR, 2018.
  24. [32][32] T. Miyato and M. Koyama. cGANs with projection discriminator. In Proc. ICLR, 2018.
  25. [33][33] S. Mo, M. Cho, and J. Shin. Freeze the discriminator: a simple baseline for fine-tuning GANs. CoRR, abs/2002.10964, 2020. · arXiv
  26. [34][34] A. Noguchi and T. Harada. Image generation from small datasets via batch statistics adaptation. In Proc. ICCV, 2019.
  27. [35][35] M. Sajjadi, M. Javanmardi, and T. Tasdizen. Regularization with stochastic transformations and perturba- tions for deep semi-supervised learning. In Proc. NIPS, 2016.
  28. [36][36] M. S. M. Sajjadi, O. Bachem, M. Lucic, O. Bousquet, and S. Gelly. Assessing generative models via precision and recall. In Proc. NIPS, 2018.
  29. [37][37] T. Salimans, I. J. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. Improved techniques for training GANs. In Proc. NIPS, 2016.
  30. [38][38] E. Schönfeld, B. Schiele, and A. Khoreva. A U-net based discriminator for generative adversarial networks. CoRR, abs/2002.12655, 2020. · arXiv
  31. [39][39] O. Sendik, D. Lischinski, and D. Cohen-Or. Unsupervised multi-modal styled content generation. CoRR, abs/2001.03640, 2020. · arXiv
  32. [40][40] C. Shorten and T. M. Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of Big Data, 6, 2019.
  33. [41][41] C. Sønderby, J. Caballero, L. Theis, W. Shi, and F. Huszár. Amortised MAP inference for image super- resolution. In Proc. ICLR, 2017.
  34. [42][42] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15:1929–1958, 2014.
  35. [43][43] N.-T. Tran, V.-H. Tran, N.-B. Nguyen, T.-K. Nguyen, and N.-M. Cheung. On data augmentation for GAN training. CoRR, abs/2006.05338, 2020. · arXiv
  36. [44][44] Y. Wang, A. Gonzalez-Garcia, D. Berga, L. Herranz, F. S. Khan, and J. van de Weijer. MineGAN: Effective knowledge transfer from GANs to target domains with few images. In Proc. CVPR, 2020.
  37. [45][45] Y. Wang, C. Wu, L. Herranz, J. van de Weijer, A. Gonzalez-Garcia, and B. Raducanu. Transferring GANs: Generating images from limited data. In Proc. ECCV, 2018.
  38. [46][46] J. Wishart and M. S. Bartlett. The distribution of second order moment statistics in a normal system. Mathematical Proceedings of the Cambridge Philosophical Society, 28(4):455–459, 1932. 11
  39. [47][47] X. Yi, E. Walia, and P. S. Babyn. Generative adversarial network in medical imaging: A review. Medical Image Analysis, 58, 2019.
  40. [48][48] D. Zhang and A. Khoreva. PA-GAN: Improving GAN training by progressive augmentation. In Proc. NeurIPS, 2019.
  41. [50][50] H. Zhang, Z. Zhang, A. Odena, and H. Lee. Consistency regularization for generative adversarial networks. In Proc. ICLR, 2019.
  42. [51][51] S. Zhao, Z. Liu, J. Lin, J.-Y. Zhu, and S. Han. Differentiable augmentation for data-efficient GAN training. CoRR, abs/2006.10738, 2020. · arXiv
  43. [52][52] Y. Zhao, C. Li, P. Yu, J. Gao, and C. Chen. Feature quantization improves GAN training. CoRR, abs/2004.02088, 2020. · arXiv
  44. [53][53] Z. Zhao, S. Singh, H. Lee, Z. Zhang, A. Odena, and H. Zhang. Improved consistency regularization for GANs. CoRR, abs/2002.04724, 2020. · arXiv

Cited by

Sorted by citations

Authors
Karras, T., Aittala, M., Hellsten, J., Laine, S., Lehtinen, J., & Aila, T.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2006.06676
arXiv
2006.06676
OpenAlex
W3034720584
Cited by
1,170
References
5
Citations by year
2026: 9 · 2025: 141 · 2024: 320 · 2023: 368 · 2022: 201 · 2021: 121 · 2020: 9 · 2019: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
generative adversarial networks, limited training data, training stability, CIFAR-10, image generation, Fréchet Inception Distance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review