Conditional Image Generation with PixelCNN Decoders
Open accessThe DOI matches its OpenAlex record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
This work explores conditional image generation with a new image density model based on the PixelCNN architecture. The model can be conditioned on any vector, including descriptive labels or tags, or latent embeddings created by other networks. When conditioned on class labels from the ImageNet database, the model is able to generate diverse, realistic scenes representing distinct animals, objects, landscapes and structures. When conditioned on an embedding produced by a convolutional network given a single image of an unseen face, it generates a variety of new portraits of the same person with different facial expressions, poses and lighting conditions. We also show that conditional PixelCNN can serve as a powerful decoder in an image autoencoder. Additionally, the gated convolutional layers in the proposed model improve the log-likelihood of PixelCNN to match the state-of-the-art performance of PixelRNN on ImageNet, with greatly reduced computational cost.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267 · cited 1×
“The network was trained with a triplet loss function [23] that ensured that the embedding h produced for an image x of a specific person was closer to the embeddings for all other images of the same person than it was to any embedding of another person.”
MethodTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Self-citationA Neural Algorithm of Artistic Style
Gatys, L., Ecker, A., & Bethge, M. (2016) · Journal of Vision · doi:10.1167/16.12.326 · Cited by 2,612
Generative Adversarial Text to Image Synthesis
Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., & Lee, H. (2016) · arXiv · doi:10.48550/arxiv.1605.05396 · Cited by 2,494
Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction
Masci, J., Meier, U., Cireşan, D., & Schmidhuber, J. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-21735-7_7 · Cited by 1,955
NICE: Non-linear Independent Components Estimation
Dinh, L., Krueger, D., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1410.8516 · Cited by 1,493
Pixel Recurrent Neural Networks
van den Oord, A., Kalchbrenner, N., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.06759 · Cited by 1,437
Self-citationDRAW: A Recurrent Neural Network For Image Generation
Gregor, K., Danihelka, I., Graves, A., Rezende, D. J., & Wierstra, D. (2015) · International Conference on Machine Learning · Cited by 1,285
Self-citationSrivastava, R. K., Greff, K., & Schmidhuber, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.06228 · Cited by 1,246
A note on the evaluation of generative models
Theis, L., Oord, A. V. D., & Bethge, M. (2015) · arXiv · doi:10.48550/arxiv.1511.01844 · Cited by 1,026
Self-citationDeep unsupervised learning using nonequilibrium thermodynamics
Sohl‐Dickstein, J. (2024) · International Conference on Machine Learning · doi:10.57702/qrfwyszh · Cited by 764
(2016) · neural information processing systems · Cited by 735
Offline Arabic Handwriting Recognition with Multidimensional Recurrent Neural Networks
Graves, A. (2012) · Guide to OCR for Arabic Scripts · doi:10.1007/978-1-4471-4072-6_12 · Cited by 684
Self-citationThe Neural Autoregressive Distribution Estimator
Larochelle, H., & Murray, I. (2011) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 626
Inceptionism: Going Deeper into Neural Networks
Mordvintsev, A., Olah, C., & Tyka, M. (2015) · Cited by 574
Action-conditional video prediction using deep networks in Atari games
Oh, J., Guo, X., Lee, H., Lewis, R. L., & Singh, S. P. (2015) · neural information processing systems · Cited by 561
Kalchbrenner, N., Danihelka, I., & Graves, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.01526 · Cited by 267
Self-citationLearning with Hierarchical-Deep Models
Salakhutdinov, R., Tenenbaum, J. B., & Torralba, A. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.269 · Cited by 225
Kaiser, Ł., & Sutskever, I. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.08228 · Cited by 214
Towards Conceptual Compression
(2016) · arXiv (Cornell University) · Cited by 185
Gregor, K., Danihelka, I., Mnih, A., Blundell, C., & Wierstra, D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1310.8499 · Cited by 99
Factoring Variations in Natural Images with Deep Gaussian Mixture Models
van den Oord, A., & Schrauwen, B. (2014) · Ghent University Academic Bibliography (Ghent University) · Cited by 92
Self-citationGenerative Image Modeling Using Spatial LSTMs
Theis, L., & Bethge, M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03478 · Cited by 88
Locally-connected transformations for deep GMMs
van den Oord, A., & Dambre, J. (2015) · Ghent University Academic Bibliography (Ghent University) · Cited by 13
Self-citation
Cited by
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
Denoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712
“40 Gated PixelCNN [59] 4.”
Improved training of Wasserstein GANs
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. (2017) · arXiv · doi:10.48550/arxiv.1704.00028 · Cited by 6,800
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
“More recent advancements in autoregressive modeling such as masked convolutional architectures [18, 19] or self-attention networks [20] could help improve results further.”
BackgroundMethodSelf-citationWaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
“For images, the equivalent of a causal convolution is a masked convolution (van den Oord et al., 2016a) which can be implemented by constructing a mask tensor and doing an elementwise multiplication of this mask with the convolution kernel before applying it.”
BackgroundMethodInfluentialSelf-citation
Metadata
- Authors
- van den Oord, A., Kalchbrenner, N., Vinyals, O., Espeholt, L., Graves, A., & Kavukcuoglu, K.
- Venue
- arXiv (Cornell University)
- Publisher
- arXiv
- Published
- 2016
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1606.05328
- OpenAlex
- W2963636093
- Cited by
- 1,833
- References
- 26
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.