The DOI matches its Crossref record; 83 of the 83 references it cites were found in a registry or an index.
Generative Adversarial Networks (GANs): Challenges, Solutions, and Future Directions
Abstract
Generative Adversarial Networks (GANs) is a novel class of deep generative models that has recently gained significant attention. GANs learn complex and high-dimensional distributions implicitly over images, audio, and data. However, there exist major challenges in training of GANs, i.e., mode collapse, non-convergence, and instability, due to inappropriate design of network architectre, use of objective function, and selection of optimization algorithm. Recently, to address these challenges, several solutions for better design and optimization of GANs have been investigated based on techniques of re-engineered network architectures, new objective functions, and alternative optimization algorithms. To the best of our knowledge, there is no existing survey that has particularly focused on the broad and systematic developments of these solutions. In this study, we perform a comprehensive survey of the advancements in GANs design and optimization solutions proposed to handle GANs challenges. We first identify key research issues within each design and optimization technique and then propose a new taxonomy to structure solutions by key research issues. In accordance with the taxonomy, we provide a detailed discussion on different GANs variants proposed within each solution and their relationships. Finally, based on the insights gained, we present promising research directions in this rapidly growing field.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714
(2016) · Cited by 11,679
Conditional Generative Adversarial Nets
Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365
Improved Techniques for Training GANs
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016) · arXiv · doi:10.48550/arxiv.1606.03498 · Cited by 6,965
Improved training of Wasserstein GANs
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. (2017) · arXiv · doi:10.48550/arxiv.1704.00028 · Cited by 6,800
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
(2018) · doi:10.1109/cvpr40276.2018 · Cited by 4,737
The Earth Mover's Distance as a Metric for Image Retrieval
Rubner, Y., Tomasi, C., & Guibas, L. J. (2000) · International Journal of Computer Vision · doi:10.1023/a:1026543900054 · Cited by 4,568
2013 IEEE International Conference on Computer Vision
(2013) · doi:10.1109/iccv19984.2013 · Cited by 4,183
5th International Conference on Learning Representations (ICLR 17)
Havrylov, S., & Titov, I. (2017) · Cited by 3,503
Self-Attention Generative Adversarial Networks
Zhang, H., Goodfellow, I., Metaxas, D., & Odena, A. (2018) · arXiv · doi:10.48550/arxiv.1805.08318 · Cited by 3,353
Numerical Methods for Ordinary Differential Equations
Butcher, J. C. (2016) · Wiley eBooks · doi:10.1002/9781119121534 · Cited by 2,986
Natural Gradient Works Efficiently in Learning
Amari, S.-I. (1998) · Neural Computation · doi:10.1162/089976698300017746 · Cited by 2,681
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
Akhtar, N., & Mian, A. (2018) · IEEE Access · doi:10.1109/access.2018.2807385 · Cited by 2,021
Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
Bengio, Y., Léonard, N., & Courville, A. (2013) · arXiv · doi:10.48550/arxiv.1308.3432 · Cited by 1,910
Salakhutdinov, R., & Hinton, G. E. (2009) · International Conference on Artificial Intelligence and Statistics · Cited by 1,890
2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
(2001) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR 2001 CVPR-01 · doi:10.1109/cvpr.2001.990916 · Cited by 1,876
Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
Kim, T., Cha, M., Kim, H., Lee, J. K., & Kim, J. (2017) · arXiv · doi:10.48550/arxiv.1703.05192 · Cited by 1,716
2007 IEEE Conference on Computer Vision and Pattern Recognition
(2007) · doi:10.1109/cvpr12085.2007 · Cited by 1,688
Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., & Frey, B. (2015) · arXiv · doi:10.48550/arxiv.1511.05644 · Cited by 1,579
Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
Denton, E., Chintala, S., Szlam, A. D., & Fergus, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.05751 · Cited by 1,541
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
Wu, J., Zhang, C., Xue, T., Freeman, W. T., & Tenenbaum, J. B. (2016) · arXiv · doi:10.48550/arxiv.1610.07584 · Cited by 1,393 · cited 1×
“Generation of realistic images has wide range of practical applications, such as anime character generation [159]–[164], image synthesis [165]–[168], super resolution [10], [124], [169]–[177], image editing and blending [178], [179], inpainting [125], [180]–[182], interactive image generation [183], [184], human pose…”
BackgroundA Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
Gui, J., Sun, Z., Wen, Y., Tao, D., & Ye, J. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3130191 · Cited by 1,231
f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
Nowozin, S., Cseke, B., & Tomioka, R. (2016) · arXiv · doi:10.48550/arxiv.1606.00709 · Cited by 1,183
InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.03657 · Cited by 1,156
Pros and cons of GAN evaluation measures
Borji, A. (2019) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2018.10.009 · Cited by 963
Energy-based Generative Adversarial Network
Zhao, J., Mathieu, M., & LeCun, Y. (2016) · arXiv · doi:10.48550/arxiv.1609.03126 · Cited by 912 · cited 14×
“Zhao, et al. [85] presented an Energy-based GANs (EBGAN) in which an autoencoder is used for D and implemented an energy function on D to assign low energies to regions near the data manifold and higher energies to other regions (see Figure 22).”
BackgroundMethodInfluentialWeight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks
Salimans, T., & Kingma, D. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.07868 · Cited by 883
A Tutorial on Energy-Based Learning
LeCun, Y., Chopra, S., Hadsell, R. T., Ranzato, A., & Huang, F. J. (2006) · Cited by 842
Generative adversarial networks: introduction and outlook
Wang, K., Gou, C., Duan, Y., Lin, Y., Zheng, X., & Wang, F.-Y. (2017) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2017.7510583 · Cited by 806
Recent Progress on Generative Adversarial Networks (GANs): A Survey
Pan, Z., Yu, W., Yi, X., Khan, A., Yuan, F., & Zheng, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2905015 · Cited by 701
2015 IEEE Winter Conference on Applications of Computer Vision
(2015) · doi:10.1109/wacv33479.2015 · Cited by 672
Deep Convolutional Inverse Graphics Network
Kulkarni, T. D., Whitney, W., Kohli, P., & Tenenbaum, J. B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.03167 · Cited by 653
Donahue, J., Krähenbühl, P., & Darrell, T. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.09782 · Cited by 616
Quantum Generative Adversarial Learning
Lloyd, S., & Weedbrook, C. (2018) · Physical Review Letters · doi:10.1103/physrevlett.121.040502 · Cited by 576
Integral Probability Metrics and Their Generating Classes of Functions
Müller, A. (1997) · Advances in Applied Probability · doi:10.1017/s000186780002807x · Cited by 548
The International Joint Conference on Artificial Intelligence (IJCAI)
(1972) · Artificial Intelligence · doi:10.1016/0004-3702(72)90052-5 · Cited by 480
Kernel Measures of Conditional Dependence
Fukumizu, K., Gretton, A., Sun, X., & Schölkopf, B. (2007) · Cited by 435
Stacked Generative Adversarial Networks
Huang, X., Li, Y., Poursaeed, O., Hopcroft, J., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.202 · Cited by 434
Gradient descent GAN optimization is locally stablePreprint
Nagarajan, V., & Kolter, J. (2017) · Neural Information Processing Systems · doi:10.48550/arxiv.1706.04156 · Cited by 383
Unrolled Generative Adversarial Networks
Metz, L., Poole, B., Pfau, D., & Sohl‐Dickstein, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02163 · Cited by 382
FairGAN: Fairness-aware Generative Adversarial Networks
Xu, D., Yuan, S., Zhang, L., & Wu, X. (2018) · 2018 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata.2018.8622525 · Cited by 379
How Generative Adversarial Networks and Their Variants Work: An Overview
Hong, Y., Hwang, U., Yoo, J., & Yoon, S. (2020) · ACM Computing Surveys · doi:10.1145/3301282 · Cited by 373
Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities
Qi, G.-J. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01265-2 · Cited by 341
Stabilizing Training of Generative Adversarial Networks through Regularization
Roth, K., Lucchi, A., Nowozin, S., & Hofmann, T. (2017) · arXiv (Cornell University) · doi:10.3929/ethz-b-000223162 · Cited by 321
Dual Discriminator Generative Adversarial NetsPreprint
Nguyen, T. D., Le, T., Vu, H. T., & Phung, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.03831 · Cited by 281
Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
Wang, Z., She, Q., & Ward, T. E. (2022) · ACM Computing Surveys · doi:10.1145/3439723 · Cited by 267
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
Gui, J., Sun, Z., Wen, Y., Tao, D., & Ye, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.06937 · Cited by 253
On integral probability metrics, ϕ-divergences and binary classificationPreprint
Sriperumbudur, B. K., Fukumizu, K., Gretton, A., Schölkopf, B., & Lanckriet, G. R. G. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0901.2698 · Cited by 224
Steganographic generative adversarial networks
Volkhonskiy, D., Nazarov, I., & Burnaev, E. (2020) · Twelfth International Conference on Machine Vision (ICMV 2019) · doi:10.1117/12.2559429 · Cited by 224
Variational Approaches for Auto-Encoding Generative Adversarial Networks
Rosca, M., Lakshminarayanan, B., Warde-Farley, D., & Mohamed, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.04987 · Cited by 217
PacGAN: The power of two samples in generative adversarial networks
Khetan, A. K. (2018) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 205
Triple Generative Adversarial Nets
Li, C., Xu, K., Zhu, J., & Zhang, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.02291 · Cited by 204
2004 IEEE Intelligent Vehicles Symposium
(2004) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2004.1336338 · Cited by 199
AdaGAN: Boosting Generative ModelsPreprint
Tolstikhin, I., Gelly, S., Bousquet, O., Simon-Gabriel, C.-J., & Schölkopf, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.02386 · Cited by 196
Banach Wasserstein GANPreprint
Adler, J., & Lunz, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.06621 · Cited by 189
Recent Advances of Generative Adversarial Networks in Computer Vision
Cao, Y.-J., Jia, L.-L., Chen, Y.-X., Lin, N., Yang, C., Zhang, B., Liu, Z., Li, X.-X., & Dai, H.-H. (2019) · IEEE Access · doi:10.1109/access.2018.2886814 · Cited by 188
CapsuleGAN: Generative Adversarial Capsule Network
Jaiswal, A., AbdAlmageed, W., Wu, Y., & Natarajan, P. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-11015-4_38 · Cited by 186
scIGANs: single-cell RNA-seq imputation using generative adversarial networks
Xu, Y., Zhang, Z., You, L., Liu, J., Fan, Z., & Zhou, X. (2020) · Nucleic Acids Research · doi:10.1093/nar/gkaa506 · Cited by 179
33rd International Conference on Machine Learning, ICML 2016
Luketina, J., Berglund, M., Greff, K., & Raiko, T. (2016) · International Conference on Machine Learning · Cited by 163
31st International Conference on Machine Learning, ICML 2014
Sejdinović, D., Strathmann, H., Lomelí, M., Andrieu, C., & Gretton, A. (2014) · International Conference on Machine Learning · Cited by 107
Deep and Hierarchical Implicit ModelsPreprint
Tran, D., Ranganath, R., & Blei, D. (2017) · arXiv.org · doi:10.48550/arxiv.1702.08896 · Cited by 103
Controllable Generative Adversarial Network
Lee, M., & Seok, J. (2019) · IEEE Access · doi:10.1109/access.2019.2899108 · Cited by 102
Generative Adversarial Network Technologies and Applications in Computer Vision
Jin, L., Tan, F., & Jiang, S. (2020) · Computational Intelligence and Neuroscience · doi:10.1155/2020/1459107 · Cited by 101
Dropout-GAN: Learning from a Dynamic Ensemble of DiscriminatorsPreprint
Mordido, G., Yang, H., & Meinel, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.11346 · Cited by 72
Bayesian GANPreprint
Saatçi, Y., & Wilson, A. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.09558 · Cited by 69
Adversarial Symmetric Variational AutoencoderPreprint
Pu, Y., Wang, W., Henao, R., Chen, L., Gan, Z., Li, C., & Carin, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04915 · Cited by 63
Improved generative adversarial networks with reconstruction loss
Li, Y., Xiao, N., & Ouyang, W. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.10.014 · Cited by 32
Gang of GANs: Generative Adversarial Networks with Maximum Margin RankingPreprint
Juefei-Xu, F., Boddeti, V. N., & Savvides, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.04865 · Cited by 31
Adversarial Framework for Unsupervised Learning of Motion Dynamics in Videos
Spampinato, C., Palazzo, S., D’Oro, P., Giordano, D., & Shah, M. A. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01246-5 · Cited by 21
The Six Fronts of the Generative Adversarial NetworksPreprint
Bissoto, A. E., Junior, E. A. D. V., & Avila, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.13076 · Cited by 21
Message Passing Multi-Agent GANsPreprint
Ghosh, A., Kulharia, V., & Namboodiri, V. P. (2016) · arXiv.org · doi:10.48550/arxiv.1612.01294 · Cited by 19
Hierarchical Adversarially Learned InferencePreprint
Belghazi, M. I., Rajeswar, S., Mastropietro, O., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.01071 · Cited by 17
Generative Adversarial Nets for Information Retrieval: Fundamentals and Advances
Zhang, W. (2018) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3209978.3210184 · Cited by 14
Fisher GANPreprint
Mroueh, Y., & Sercu, T. (2017) · arXiv (Cornell University) · Cited by 12
A Gentle Introduction to Support Vector Machines in Biomedicine: Case Studies
Statnikov, A., Aliferis, C., Hardin, D. P., & Guyon, I. (2011) · WORLD SCIENTIFIC eBooks · Cited by 9
Cited by
Recent advances and clinical applications of deep learning in medical image analysis
Chen, X., Wang, X., Zhang, K., Fung, K.-M., Thai, T. C., Moore, K., Mannel, R. S., Liu, H., Zheng, B., & Qiu, Y. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102444 · Cited by 1,080
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
Zhang, M., Cai, Z., Pan, L., Hong, F., Guo, X., Yang, L., & Liu, Z. (2022) · arXiv · doi:10.48550/arxiv.2208.15001 · Cited by 437
MotionDiffuse: Text-Driven Human Motion Generation With Diffusion Model
Zhang, M., Cai, Z., Pan, L., Hong, F., Guo, X., Yang, L., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3355414 · Cited by 424
Survey on Synthetic Data Generation, Evaluation Methods and GANs
Figueira, A., & Vaz, B. (2022) · Mathematics · doi:10.3390/math10152733 · Cited by 413
Deep Learning for Medical Anomaly Detection – A Survey
Fernando, T., Gammulle, H., Denman, S., Sridharan, S., & Fookes, C. (2022) · ACM Computing Surveys · doi:10.1145/3464423 · Cited by 399
Metadata
- Authors
- Saxena, D., & Cao, J.
- Venue
- ACM Computing Surveys
- Published
- 2021-05-08
- Volume
- 54
- Issue
- 3
- Pages
- 1-42
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W3161810785
- Cited by
- 689
- References
- 83
- FWCI
- 29.01
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.