The DOI matches its DataCite record; 3 of the 3 references it cites were found in a registry or an index.
A Note on the Inception Score
Open accessAbstract
Deep generative models are powerful tools that have produced impressive results in recent years. These advances have been for the most part empirically driven, making it essential that we use high quality evaluation metrics. In this paper, we provide new insights into the Inception Score, a recently proposed and widely used evaluation metric for generative models, and demonstrate that it fails to provide useful guidance when comparing models. We discuss both suboptimalities of the metric itself and issues with its application. Finally, we call for researchers to be more systematic and careful when evaluating and comparing generative models, as the advancement of the field depends upon it.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“One drawback of this metric is that it does not reward covering the whole distribution or capturing diversity within a class”
From Diffusion Models Beat GANs on Image Synthesis (2021) · cites this paper as [3]
The cited source says what the claim attributes to it.
In the citing paper
“and models which memorize a small subset of the full dataset will still have high IS [3].”
From Diffusion Models Beat GANs on Image Synthesis (2021) · cites this paper as [3]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365
Research works printed but not linked: 20
- Arjovsky, M., Chintala, S., and Bottou, L. Wasserstein gan. arXiv preprint arXiv:1701.07875, 2017. · arXiv
- Che, T., Li, Y., Jacob, A. P., Bengio, Y., and Li, W. Mode reg- ularized generative adversarial networks. arXiv preprint arXiv:1612.02136, 2016. · arXiv
- Esteban, C., Hyland, S. L., and Rätsch, G. Real-valued (Medical) Time Series Generation with Recurrent Condi- tional GANs. ArXiv e-prints, June 2017.
- Farimani, A. B., Gomes, J., and Pande, V. S. Deep learning the physics of transport phenomena. CoRR, abs/1709.02432, 2017. URL http://arxiv.org/ abs/1709.02432. · arXiv
- Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial nets. In Advances in neural information processing systems, pp. 2672-2680, 2014a.
- Goodfellow, I. J., Shlens, J., and Szegedy, C. Explain- ing and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014b. · arXiv
- Guo, J., Lu, S., Cai, H., Zhang, W., Yu, Y., and Wang, J. Long text generation via adversarial training with leaked information. CoRR, abs/1709.08624, 2017. URL http://arxiv.org/abs/1709.08624. · arXiv
- Karras, T., Aila, T., Laine, S., and Lehtinen, J. Progres- sive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196, 2017. · arXiv
- Ledig, C., Theis, L., Huszar, F., Caballero, J., Aitken, A. P., Tejani, A., Totz, J., Wang, Z., and Shi, W. Photo-realistic single image super-resolution using a generative adver- sarial network. CoRR, abs/1609.04802, 2016. URL http://arxiv.org/abs/1609.04802. · arXiv
- Lucic, M., Kurach, K., Michalski, M., Gelly, S., and Bous- quet, O. Are gans created equal? a large-scale study. arXiv preprint arXiv:1711.10337, 2017. · arXiv
- Mogren, O. C-RNN-GAN: continuous recurrent neural net- works with adversarial training. CoRR, abs/1611.09904, 2016. URL http://arxiv.org/abs/1611. 09904. · arXiv
- Qiantong, X., Gao, H., Yang, Y., Chuan, G., Yu, S., Felix, W., and Weinberger, K. An empirical study on evaluation metrics of generative adversarial networks. arXiv preprint arXiv:1806.07755, 2018. · arXiv
- Rosca, M., Lakshminarayanan, B., Warde-Farley, D., and Mohamed, S. Variational approaches for auto- encoding generative adversarial networks. arXiv preprint arXiv:1706.04987, 2017. · arXiv
- Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. Imagenet large scale visual recognition chal- lenge. International Journal of Computer Vision, 115(3): 211-252, 2015.
- Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Rad- ford, A., and Chen, X. Improved techniques for training gans. In Advances in Neural Information Processing Systems, pp. 2234-2242, 2016.
- Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I. J., and Fergus, R. Intriguing properties of neural networks. CoRR, abs/1312.6199, 2013. URL http://arxiv.org/abs/1312.6199. · arXiv
- Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, pp. 2818-2826, 2016.
- Theis, L., Oord, A. v. d., and Bethge, M. A note on the evaluation of generative models. arXiv preprint arXiv:1511.01844, 2015. · arXiv
- van den Oord, A. and Dambre, J. Locally-connected trans- formations for deep gmms. In International Conference on Machine Learning (ICML): Deep learning Workshop, pp. 1-8, 2015.
- Zhu, J., Park, T., Isola, P., and Efros, A. A. Unpaired image- to-image translation using cycle-consistent adversarial networks. CoRR, abs/1703.10593, 2017. URL http: //arxiv.org/abs/1703.10593. · arXiv
Cited by
Sorted by citations
Diffusion Models Beat GANs on Image Synthesis
Dhariwal, P., & Nichol, A. (2021) · arXiv · doi:10.48550/arxiv.2105.05233 · Cited by 3,362
Pros and cons of GAN evaluation measures
Borji, A. (2019) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2018.10.009 · Cited by 963
Brain-inspired replay for continual learning with artificial neural networks
van de Ven, G. M., Siegelmann, H. T., & Tolias, A. S. (2020) · Nature Communications · doi:10.1038/s41467-020-17866-2 · Cited by 474
Racial faces in the wild: Reducing racial bias by information maximization adaptation network
Wang, M., Deng, W., Hu, J., Tao, X., & Huang, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00078 · Cited by 353
Self-Supervised GANs via Auxiliary Rotation Loss
Chen, T., Zhai, X., Ritter, M., Lucic, M., & Houlsby, N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01243 · Cited by 328
Metadata
- Authors
- Barratt, S., & Sharma, R.
- Affiliations
- Stanford University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1801.01973
- OpenAlex
- W2782980316
- Cited by
- 732
- References
- 3
- Citations by year
- 2026: 47 · 2025: 123 · 2024: 84 · 2023: 94 · 2022: 69 · 2021: 111 · 2020: 94 · 2019: 69 · 2018: 41
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (DGE-165651 8, 1656518)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- Inception Score, generative model evaluation, evaluation metrics, model comparison, deep generative models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Note on the Inception Score. https://app.cochat.ai/papers/10.48550/arxiv.1801.01973
- The paper
- Barratt, S., & Sharma, R. (2018). A Note on the Inception Score. arXiv. https://doi.org/10.48550/arxiv.1801.01973
