Pros and cons of GAN evaluation measures
Open accessAbstract
Generative models , in particular generative adversarial networks (GANs), have gained significant attention in recent years. A number of GAN variants have been proposed and have been utilized in many applications. Despite large strides in terms of theoretical progress, evaluating and comparing GANs remains a daunting task. While several measures have been introduced, as of yet, there is no consensus as to which measure best captures strengths and limitations of models and should be used for fair model comparison. As in other areas of computer vision and machine learning , it is critical to settle on one or few good measures to steer the progress in this field. In this paper, I review and critically discuss more than 24 quantitative and 5 qualitative measures for evaluating generative models with a particular emphasis on GAN-derived models. I also provide a set of 7 desiderata followed by an evaluation of whether a given measure or a family of measures is compatible with them.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The diversity and quality of GAN-generated images are evaluated manually by a physician or quantitatively using similarity measures, generally FID, structural similarity index, and peak signal-to-noise ratio.
“The diversity and quality of the new images produced by GANs are evaluated manually by a physician or quantitatively by using similarity evaluation measurements, which are generally FID, structural similarity index, and peak signal-to-noise ratio (Borji 2019; Xu et al. 2018).”
From Medical Image Data Augmentation: Techniques, Comparisons and Interpretations (2023) · cites this paper as Borji 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138 · see citing works
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · see citing works
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · see citing works
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696 · see citing works
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235 · see citing works
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365 · see citing works
Deep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747 · see citing works
Improved Techniques for Training GANs
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016) · arXiv · doi:10.48550/arxiv.1606.03498 · Cited by 6,965 · see citing works
Improved training of Wasserstein GANs
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. (2017) · arXiv · doi:10.48550/arxiv.1704.00028 · Cited by 6,800 · see citing works
The Laplacian pyramid as a compact image code
Burt, P., & Adelson, E. (1983) · IEEE Transactions on Communications · doi:10.1109/tcom.1983.1095851 · Cited by 6,077 · see citing works
A universal image quality index
Wang, Z., & Bovik, A. C. (2002) · IEEE Signal Processing Letters · doi:10.1109/97.995823 · Cited by 5,757 · see citing works
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691 · see citing works
Testing Statistical Hypotheses
Niewiadomska-Bugaj, M., & Bartoszynski, R. (2021) · Wiley series in probability and statistics · doi:10.1002/9781119243830.ch12 · Cited by 5,188 · see citing works
Adversarial Discriminative Domain Adaptation
Tzeng, E., Hoffman, J., Saenko, K., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.316 · Cited by 5,154 · see citing works
Progressive Growing of GANs for Improved Quality, Stability, and Variation
Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017) · arXiv · doi:10.48550/arxiv.1710.10196 · Cited by 4,727 · see citing works
Reading digits in natural images with unsupervised feature learning
Netzer, Y. (2024) · doi:10.57702/e7wwo0zk · Cited by 4,261 · see citing works
Image information and visual quality
Sheikh, H. R., & Bovik, A. C. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2005.859378 · Cited by 4,104 · see citing works
(2012) · MPG.PuRe (Max Planck Society) · doi:10.5555/2188385.2188410 · Cited by 3,603 · see citing works
Striving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447 · see citing works
(2017) · International Conference on Learning Representations · Cited by 3,248 · see citing works
Relations between the statistics of natural images and the response properties of cortical cells
Field, D. J. (1987) · Journal of the Optical Society of America A · doi:10.1364/josaa.4.002379 · Cited by 3,241 · see citing works
Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures
Wang, Z., & Bovik, A. C. (2009) · IEEE Signal Processing Magazine · doi:10.1109/msp.2008.930649 · Cited by 3,175 · see citing works
Conditional Image Generation with PixelCNN DecodersPreprint
van den Oord, A., Kalchbrenner, N., Vinyals, O., Espeholt, L., Graves, A., & Kavukcuoglu, K. (2016) · Neural Information Processing Systems · doi:10.48550/arxiv.1606.05328 · Cited by 3,066 · see citing works
StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks
Zhang, H., Xu, T., Li, H., Zhang, S., Wang, X., Huang, X., & Metaxas, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.629 · Cited by 3,021 · see citing works
Conditional Image Synthesis With Auxiliary Classifier GANsPreprint
Odena, A., Olah, C., & Shlens, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.09585 · Cited by 2,968 · see citing works
Generative Adversarial Text to Image Synthesis
Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., & Lee, H. (2016) · arXiv · doi:10.48550/arxiv.1605.05396 · Cited by 2,494 · see citing works
Natural Image Statistics and Neural Representation
Simoncelli, E. P., & Olshausen, B. A. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.1193 · Cited by 2,460 · see citing works
Deep Generative Image Models using a Laplacian Pyramid of Adversarial NetworksPreprint
Denton, E., Chintala, S., Szlam, A., & Fergus, R. (2015) · Neural Information Processing Systems · doi:10.48550/arxiv.1506.05751 · Cited by 2,360 · see citing works
Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial NetworksPreprint
Zhu, J.-Y., Park, T., Isola, P., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.10593 · Cited by 2,278 · see citing works
Arjovsky, M., Chintala, S., & Bottou, L. (2017) · arXiv · doi:10.48550/arxiv.1701.07875 · Cited by 2,225 · see citing works
Conditional Image Generation with PixelCNN DecodersPreprint
van den Oord, A., Kalchbrenner, N., Vinyals, O., et al. (2016) · arXiv (Cornell University) · Cited by 1,586 · see citing works
Network Dissection: Quantifying Interpretability of Deep Visual Representations
Bau, D., Zhou, B., Khosla, A., Oliva, A., & Torralba, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.354 · Cited by 1,389 · see citing works
A note on the evaluation of generative modelsPreprint
Theis, L., van den Oord, A., & Bethge, M. (2015) · International Conference on Learning Representations · doi:10.48550/arxiv.1511.01844 · Cited by 1,345 · see citing works
Semantic Image Inpainting with Deep Generative Models
Yeh, R. A., Chen, C., Lim, T. Y., Schwing, A. G., Hasegawa‐Johnson, M., & Do, M. N. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.728 · Cited by 1,263 · see citing works
BEGAN: Boundary Equilibrium Generative Adversarial NetworksPreprint
Berthelot, D., Schumm, T., & Metz, L. (2017) · arXiv.org · doi:10.48550/arxiv.1703.10717 · Cited by 1,217 · see citing works
Neal, R. M. (2001) · Statistics and Computing · doi:10.1023/a:1008923215028 · Cited by 1,208 · see citing works
Are GANs Created Equal? A Large-Scale StudyPreprint
Lučić, M., Kurach, K., Michalski, M., Gelly, S., & Bousquet, O. (2017) · Neural Information Processing Systems · doi:10.48550/arxiv.1711.10337 · Cited by 1,175 · see citing works
Toward Multimodal Image-to-Image TranslationPreprint
Zhu, J.-Y., Zhang, R., Pathak, D., Darrell, T., Efros, A. A., Wang, O., & Shechtman, E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.11586 · Cited by 1,159 · see citing works
Unrolled Generative Adversarial NetworksPreprint
Metz, L., Poole, B., Pfau, D., & Sohl-Dickstein, J. N. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1611.02163 · Cited by 1,100 · see citing works
Adversarially Learned InferencePreprint
Dumoulin, V., Belghazi, M. I., Poole, B., Mastropietro, O., Lamb, A., Arjovsky, M., & Courville, A. C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00704 · Cited by 1,015 · see citing works
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial NetworkPreprint
Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A. P., Tejani, A., Totz, J., Wang, Z., et al. (2016) · arXiv (Cornell University) · doi:10.17863/cam.51996 · Cited by 989 · see citing works
The statistics of natural images
Ruderman, D. (1994) · Network Computation in Neural Systems · doi:10.1088/0954-898x_5_4_006 · Cited by 945 · see citing works
A feedforward architecture accounts for rapid categorization
Serre, T., Oliva, A., & Poggio, T. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0700622104 · Cited by 918 · see citing works
Statistics of natural image categories
Torralba, A. B., & Oliva, A. (2003) · Network Computation in Neural Systems · doi:10.1088/0954-898x_14_3_302 · Cited by 845 · see citing works
Object Detectors Emerge in Deep Scene CNNsPreprint
Zhou, B., Khosla, A., Lapedriza, À., Oliva, A., & Torralba, A. B. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6856 · Cited by 823 · see citing works
Assessing Generative Models via Precision and RecallPreprint
Sajjadi, M. S. M., Bachem, O., Lučić, M., Bousquet, O., & Gelly, S. (2018) · Neural Information Processing Systems · doi:10.48550/arxiv.1806.00035 · Cited by 815 · see citing works
Rosenholtz, R. E., Li, Y., & Nakano, L. (2007) · Journal of Vision · doi:10.1167/7.2.17 · Cited by 805 · see citing works
Visual Perception and the Statistical Properties of Natural Scenes
Geisler, W. S. (2007) · Annual Review of Psychology · doi:10.1146/annurev.psych.58.110405.085632 · Cited by 732 · see citing works
Generative Moment Matching NetworksPreprint
Li, Y., Swersky, K., & Zemel, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.02761 · Cited by 644 · see citing works
Demystifying MMD GANsPreprint
Bińkowski, M., Sutherland, D. J., Arbel, M., & Gretton, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.01401 · Cited by 614 · see citing works
Kernel Mean Embedding of Distributions: A Review and Beyond
Muandet, K., Fukumizu, K., Sriperumbudur, B. K., & Schölkopf, B. (2017) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000060 · Cited by 609 · see citing works
Mode Regularized Generative Adversarial NetworksPreprint
Che, T., Li, Y.-R., Jacob, A. P., Bengio, Y., & Li, W.-J. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1612.02136 · Cited by 600 · see citing works
Lossy Image Compression with Compressive AutoencodersPreprint
Theis, L., Shi, W., Cunningham, A., & Huszár, F. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00395 · Cited by 579 · see citing works
VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational LearningPreprint
Srivastava, A., Valkov, L., Russell, C., Gutmann, M. U., & Sutton, C. A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.07761 · Cited by 524 · see citing works
Color and spatial structure in natural scenes
Burton, G. J., & Moorhead, I. R. (1987) · Applied Optics · doi:10.1364/ao.26.000157 · Cited by 500 · see citing works
Oliva, A. (2005) · Neurobiology of Attention · doi:10.1016/b978-012375731-9/50045-8 · Cited by 491 · see citing works
On Advances in Statistical Modeling of Natural Images
Srivastava, A., Lee, A. B., Simoncelli, E. P., & Zhu, S. (2003) · Journal of Mathematical Imaging and Vision · doi:10.1023/a:1021889010444 · Cited by 483 · see citing works
Stacked Generative Adversarial Networks
Huang, X., Li, Y., Poursaeed, O., Hopcroft, J. E., & Belongie, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.202 · Cited by 462 · see citing works
Scale Mixtures of Gaussians and the Statistics of Natural Images
Wainwright, M. J., & Simoncelli, E. P. (1999) · Cited by 439 · see citing works
Synthesizing the preferred inputs for neurons in neural networks via deep generator networksPreprint
Nguyen, A., Dosovitskiy, A., Yosinski, J., Brox, T., & Clune, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.09304 · Cited by 435 · see citing works
Shmelkov, K., Schmid, C., & Alahari, K. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01216-8_14 · Cited by 394 · see citing works
A Note on the Inception ScorePreprint
Barratt, S. T., & Sharma, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.01973 · Cited by 366 · see citing works
Generative Multi-Adversarial NetworksPreprint
Durugkar, I., Gemp, I., & Mahadevan, S. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1611.01673 · Cited by 366 · see citing works
SalGAN: Visual Saliency Prediction with Generative Adversarial NetworksPreprint
Pan, J., Canton-Ferrer, C., McGuinness, K., O'Connor, N. E., Torres, J., Sayrol, E., & Giró-i-Nieto, X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.01081 · Cited by 337 · see citing works
DeLiGAN: Generative Adversarial Networks for Diverse and Limited Data
Gurumurthy, S., Sarvadevabhatla, R. K., & Babu, R. V. (2017) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2017.525 · Cited by 309 · see citing works
Deep Feature Interpolation for Image Content Changes
Upchurch, P., Gardner, J. R., Pleiss, G., Pless, R. B., Snavely, N., Bala, K., & Weinberger, K. Q. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.645 · Cited by 303 · see citing works
Preserved figure-ground segregation and symmetry perception in visual neglect
Driver, J., Baylis, G. C., & Rafal, R. D. (1992) · Nature · doi:10.1038/360073a0 · Cited by 302 · see citing works
LR-GAN: Layered Recursive Generative Adversarial Networks for Image GenerationPreprint
Yang, J., Kannan, A., Batra, D., & Parikh, D. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1703.01560 · Cited by 249 · see citing works
Image De-raining Using a Conditional Generative Adversarial NetworkPreprint
Zhang, H., Sindagi, V. A., & Patel, V. M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.05957 · Cited by 245 · see citing works
Generating images with recurrent adversarial networksPreprint
Im, D. J., Kim, C. D., Jiang, H., & Memisevic, R. (2016) · arXiv.org · doi:10.48550/arxiv.1602.05110 · Cited by 232 · see citing works
Generating Adversarial Examples with Adversarial NetworksPreprint
Xiao, C., Li, B., Zhu, J.-Y., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.02610 · Cited by 230 · see citing works
Occlusion Models for Natural Images: A Statistical Study of a Scale-Invariant Dead Leaves Model
Lee, A. B., Mumford, D. B., & Huang, J. (2001) · International Journal of Computer Vision · doi:10.1023/a:1011109015675 · Cited by 224 · see citing works
Cross-view image synthesis using conditional GANs
Regmi, K., & Borji, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00369 · Cited by 204 · see citing works
Self-citationDo GANs actually learn the distribution? An empirical studyPreprint
Arora, S., & Zhang, Y. (2017) · arXiv.org · doi:10.48550/arxiv.1706.08224 · Cited by 200 · see citing works
Convergence de la répartition empirique vers la répartition théorique
Fortet, R. M., & Mourier, É. (1953) · Annales Scientifiques de l École Normale Supérieure · doi:10.24033/asens.1013 · Cited by 192 · see citing works
Learning to generate images with perceptual similarity metrics
Snell, J., Ridgeway, K., Liao, R., Roads, B. D., Mozer, M. C., & Zemel, R. S. (2017) · 2017 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2017.8297089 · Cited by 186 · see citing works
How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?Preprint
Huszár, F. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.05101 · Cited by 181 · see citing works
Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every StepPreprint
Fedus, W., Rosca, M., Lakshminarayanan, B., Dai, A. M., Mohamed, S., & Goodfellow, I. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.08446 · Cited by 177 · see citing works
Generative Models and Model Criticism via Optimized Maximum Mean DiscrepancyPreprint
Sutherland, D. J., Tung, H.-Y. F., Strathmann, H., De, S., Ramdas, A., Smola, A. J., & Gretton, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.04488 · Cited by 174 · see citing works
Training generative neural networks via Maximum Mean Discrepancy optimizationPreprint
Dziugaite, G. K., Roy, D. M., & Ghahramani, Z. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.03906 · Cited by 171 · see citing works
Wang, Z., & Simoncelli, E. P. (2008) · Journal of Vision · doi:10.1167/8.12.8 · Cited by 138 · see citing works
On the Quantitative Analysis of Decoder-Based Generative ModelsPreprint
Wu, Y., Burda, Y., Salakhutdinov, R., & Grosse, R. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.04273 · Cited by 133 · see citing works
PacGAN: The Power of Two Samples in Generative Adversarial Networks
Lin, Z., Khetan, A., Fanti, G. C., & Oh, S. (2020) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2020.2983071 · Cited by 125 · see citing works
Semantically Decomposing the Latent Spaces of Generative Adversarial NetworksPreprint
Donahue, C., Balsubramani, A., McAuley, J., & Lipton, Z. C. (2017) · International Conference on Learning Representations · doi:10.48550/arxiv.1705.07904 · Cited by 122 · see citing works
Statistical modeling and conceptualization of visual patterns
Zhu, S. C. (2003) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2003.1201820 · Cited by 110 · see citing works
Precise Recovery of Latent Vectors from Generative Adversarial NetworksPreprint
Lipton, Z. C., & Tripathi, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.04782 · Cited by 91 · see citing works
Generative Models of Visually Grounded Imagination
Vedantam, R., Fischer, I., Huang, J., & Murphy, K. (2017) · arXiv · doi:10.48550/arxiv.1705.10762 · Cited by 81 · see citing works
Geometry Score: A Method For Comparing Generative Adversarial NetworksPreprint
Khrulkov, V., & Oseledets, I. V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.02664 · Cited by 76 · see citing works
Liu, Y., Qin, Z., Luo, Z., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.01908 · Cited by 68 · see citing works
On GANs and GMMsPreprint
Richardson, E., & Weiss, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.12462 · Cited by 67 · see citing works
How Sensitive Is the Human Visual System to the Local Statistics of Natural Images?
Gerhard, H. E., Wichmann, F. A., & Bethge, M. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002873 · Cited by 63 · see citing works
Sampling Generative Networks: Notes on a Few Effective Techniques
White, T. (2016) · arXiv.org · doi:10.26686/wgtn.12585362.v1 · Cited by 63 · see citing works
Metrics for Deep Generative ModelsPreprint
Chen, N., Klushyn, A., Kurle, R., Jiang, X., Bayer, J., & van der Smagt, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.01204 · Cited by 63 · see citing works
Revisiting Classifier Two-Sample TestsPreprint
López-Paz, D., & Oquab, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.06545 · Cited by 58 · see citing works
An Improved Evaluation Framework for Generative Adversarial NetworksPreprint
Liu, S.-H., Wei, Y., Lu, J.-W., & Zhou, J. (2018) · arXiv.org · doi:10.48550/arxiv.1803.07474 · Cited by 52 · see citing works
A Test of Relative Similarity For Model Selection in Generative ModelsPreprint
Bounliphone, W., Belilovsky, E., Blaschko, M. B., Antonoglou, I., & Gretton, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04581 · Cited by 51 · see citing works
Ensembles of Generative Adversarial NetworksPreprint
Wang, Y., Zhang, L., & van de Weijer, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00991 · Cited by 41 · see citing works
An information-theoretic analysis of deep latent-variable modelsPreprint
Alemi, A. A., Poole, B., Fischer, I., Dillon, J. V., Saurous, R. A., & Murphy, K. P. (2018) · arXiv (Cornell University) · Cited by 33 · see citing works
Gang of GANs: Generative Adversarial Networks with Maximum Margin RankingPreprint
Juefei-Xu, F., Boddeti, V. N., & Savvides, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.04865 · Cited by 31 · see citing works
Image Descriptors for Displays
Cohen, R. W., Gorog, I., & Carlson, C. R. (1975) · doi:10.21236/ada007585 · Cited by 22 · see citing works
Skill Rating for Generative ModelsPreprint
Olsson, C., Bhupatiraju, S., Brown, T., Odena, A., & Goodfellow, I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.04888 · Cited by 22 · see citing works
Statistics of Deep Generated ImagesPreprint
Zeng, Y., Lu, H., & Borji, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.02688 · Cited by 11 · see citing works
Self-citationA Classification-Based Study of Covariate Shift in GAN DistributionsPreprint
Santurkar, S., Schmidt, L., & Mądry, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00970 · Cited by 7 · see citing works
Decoupled Learning for Conditional Adversarial NetworksPreprint
Zhang, Z., Song, Y., & Qi, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.06790 · Cited by 6 · see citing works
Image Generation from Sketch Constraint Using Contextual GANPreprint
Lu, Y., Wu, S., Tai, Y., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.08972 · Cited by 2 · see citing works
Beyond Planar Symmetry: Modeling human perception of reflection and rotation symmetries in the wildPreprint
Funk, C., & Liu, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.03568 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 22
- L. A. Gatys, A. S. Ecker, M. Bethge, Image style transfer using convo- lutional neural networks, in: Computer Vision and Pattern Recognition (CVPR), 2016 IEEE Conference on, IEEE, 2016, pp. 2414-2423.
- J. Johnson, A. Alahi, L. Fei-Fei, Perceptual losses for real-time style transfer and super-resolution, in: European Conference on Computer Vision, Springer, 2016, pp. 694-711.
- P. Isola, J.-Y. Zhu, T. Zhou, A. A. Efros, Image-to-image translation with conditional adversarial networks, arXiv preprint.
- C. Vondrick, H. Pirsiavash, A. Torralba, Generating videos with scene dynamics, in: Advances In Neural Information Processing Systems, 2016, pp. 613-621.
- I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial nets, in: Advances in neural information processing systems, 2014, pp. 2672-2680.
- D. P. Kingma, S. Mohamed, D. J. Rezende, M. Welling, Semi-supervised learning with deep generative models, in: Advances in Neural Information Processing Systems, 2014, pp. 3581-3589.
- K. Kurach, M. Lucic, X. Zhai, M. Michalski, S. Gelly, The gan landscape: Losses, architectures, regularization, and normalization, arXiv preprint arXiv:1807.04720. · arXiv
- G. Huang, Y. Yuan, Q. Xu, C. Guo, Y. Sun, F. Wu, K. Weinberger, An empirical study on evaluation metrics of generative adversarial networks, International Conference on Learning RepresentationsRejected. URL https://openreview.net/forum?id=Sy1f0e-R-
- I. O. Tolstikhin, S. Gelly, O. Bousquet, C.-J. Simon-Gabriel, B. Schölkopf, Adagan: Boosting generative models, in: Advances in Neural Information Processing Systems, 2017, pp. 5430-5439.
- Z. Zhou, H. Cai, S. Rong, Y. Song, K. Ren, W. Zhang, J. Wang, Y. Yu, Activation maximization generative adversarial nets, International Confer- ence on Learning Representations. URL https://openreview.net/forum?id=HyyP33gAZ
- Z. Yi, H. Zhang, P. Tan, M. Gong, Dualgan: Unsupervised dual learning for image-to-image translation, arXiv preprint.
- X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, P. Abbeel, Infogan: Interpretable representation learning by information maximizing generative adversarial nets, in: Advances in Neural Information Processing Systems, 2016, pp. 2172-2180.
- M. F. Mathieu, J. J. Zhao, J. Zhao, A. Ramesh, P. Sprechmann, Y. LeCun, Disentangling factors of variation in deep representation using adversarial training, in: Advances in Neural Information Processing Systems, 2016, pp. 5040-5048.
- Z. Zhou, W. Zhang, J. Wang, Inception score, label smoothing, gradient vanishing and-log (d (x)) alternative, arXiv preprint arXiv:1708.01729. · arXiv
- C.-L. Li, W.-C. Chang, Y. Cheng, Y. Yang, B. Póczos, Mmd gan: Towards deeper understanding of moment matching network, in: Advances in Neural Information Processing Systems, 2017, pp. 2200-2210.
- R. Zhang, P. Isola, A. A. Efros, Colorful image colorization, in: European Conference on Computer Vision, Springer, 2016, pp. 649-666.
- T. Lesort, F. Bordes, J.-F. Goudou, D. Filliat, Evaluation of generative networks through their data augmentation capacity (2018). URL https://openreview.net/forum?id=HJ1HFlZAb
- M. E. Glickman, A comprehensive guide to chess ratings, American Chess Journal 3 (1995) 59-102.
- S. Ghebreab, S. Scholte, V. Lamme, A. Smeulders, A biologically plausi- ble model for rapid natural scene identification, in: Advances in Neural Information Processing Systems, 2009, pp. 629-637.
- J. Snell, K. Ridgeway, R. Liao, B. D. Roads, M. C. Mozer, R. S. Zemel, Learning to generate images with perceptual similarity metrics, arXiv preprint arXiv:1511.06409. · arXiv
- L. v. d. Maaten, G. Hinton, Visualizing data using t-sne, Journal of machine learning research 9 (Nov) (2008) 2579-2605.
- D. J. Im, A. H. Ma, G. W. Taylor, K. Branson, Quantitatively evaluating gans with divergences proposed for training, International Conference on Learning RepresentationsAccepted as poster. URL https://openreview.net/forum?id=SJQHjzZ0-
Cited by
Sorted by citations
A survey of unsupervised deep domain adaptation
Wilson, G., & Cook, D. J. (2020) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3400066 · Cited by 987 · see citing works
Medical Image Data Augmentation: Techniques, Comparisons and Interpretations
Goceri, E. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10453-z · Cited by 459 · see citing works
A Categorical Archive of ChatGPT Failures
Borji, A. (2023) · doi:10.21203/rs.3.rs-2895792/v1 · Cited by 514 · see citing works
Self-citationGenerative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
Liu, M.-Y., Huang, X., Yu, J., Wang, T.-C., & Mallya, A. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3049196 · Cited by 236 · see citing works
Cross-view image synthesis using conditional GANs
Regmi, K., & Borji, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00369 · Cited by 204 · see citing works
Self-citationSurvey on artificial intelligence (AI) techniques for vehicular ad-hoc networks (VANETs)
Mchergui, A., Moulahi, T., & Zeadally, S. (2022) · Vehicular Communications · doi:10.1016/j.vehcom.2021.100403 · Cited by 165 · see citing works
A Review of Artificial Intelligence in Enhancing Architectural Design Efficiency
Li, Y., Chen, H., Yu, P., & Yang, L. (2025) · Applied Sciences · doi:10.3390/app15031476 · Cited by 76 · see citing works
Sharing Biomedical Data: Strengthening AI Development in Healthcare
Pereira, T., Morgado, J., Silva, F., Pelter, M. M., Dias, V. R., Barros, R., Freitas, C., Negrão, E., Flor de Lima, B., Correia da Silva, M., et al. (2021) · Healthcare · doi:10.3390/healthcare9070827 · Cited by 54 · see citing works
“These, along with other issues such as the variety of probability criteria and the lack of perceptually meaningful image similarity measures, have hindered the evaluation of generative models [52].”
BackgroundAugmentation of small training data using GANs for enhancing the performance of image classification
Hung, S.-K., & Gan, J. Q. (2021) · 2020 25th International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr48806.2021.9412399 · Cited by 10 · see citing works
- Authors
- Borji, A.
- Corresponding author
- Borji, Ali
- ORCID
- Borji, Ali
- Venue
- Computer Vision and Image Understanding
- Publisher
- Elsevier BV
- Published
- 2019-02
- Volume
- 179
- Pages
- 41-65
- ISSN
- 1077-3142 (ISSN-L)
- OpenAlex
- W2787289806
- Cited by
- 963
- References
- 110
- Citations by year
- 2026: 46 · 2025: 113 · 2024: 121 · 2023: 161 · 2022: 138 · 2021: 182 · 2020: 123 · 2019: 58 · 2018: 21
- FWCI
- 48.01
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Computer Vision and Image Understanding (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- generative adversarial networks, GAN evaluation metrics, generative model evaluation, qualitative evaluation, model comparison, generative models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review