The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
InfoVAE: Balancing Learning and Inference in Variational Autoencoders
Open accessAbstract
A key advance in learning generative models is the use of amortized inference distributions that are jointly trained with the models. We find that existing training objectives for variational autoencoders can lead to inaccurate amortized inference distributions and, in some cases, improving the objective provably degrades the inference quality. In addition, it has been observed that variational autoencoders tend to ignore the latent variables when combined with a decoding distribution that is too flexible. We again identify the cause in existing training criteria and propose a new class of objectives (Info-VAE) that mitigate these problems. We show that our model can significantly improve the quality of the variational posterior and can make effective use of the latent features regardless of the flexibility of the decoding distribution. Through extensive qualitative and quantitative analyses, we demonstrate that our models outperform competing approaches on multiple performance metrics
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602 · cited 2×
“Generative models have shown great promise in modeling complex distributions such as natural images and text (Radford et al., 2015; Zhu et al., 2017; Yang et al., 2017; Li et al., 2017).”
BackgroundLearning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235
Improved Techniques for Training GANs
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016) · arXiv · doi:10.48550/arxiv.1606.03498 · Cited by 6,965
Improved training of Wasserstein GANs
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. (2017) · arXiv · doi:10.48550/arxiv.1704.00028 · Cited by 6,800 · cited 2×
“In this respect, adversarial autoencoder becomes less desirable because it takes longer to converge, and sometimes converges to poor results even with WGAN-GP discriminators (Gulrajani et al. 2017).”
MethodResult(2017) · International Conference on Learning Representations · Cited by 3,248
Arjovsky, M., Chintala, S., & Bottou, L. (2017) · arXiv · doi:10.48550/arxiv.1701.07875 · Cited by 2,225
A Kernel Method for the Two-Sample-Problem
Gretton, A., Borgwardt, K. M., Rasch, M., Schölkopf, B., & Smola, A. J. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0069 · Cited by 2,167
Generating Sentences from a Continuous Space
Bowman, S. R., Vilnis, L., Vinyals, O., Dai, A., Jozefowicz, R., & Bengio, S. (2016) · Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning · doi:10.18653/v1/k16-1002 · Cited by 2,061
Conditional Image Generation with PixelCNN Decoders
van den Oord, A., Kalchbrenner, N., Vinyals, O., Espeholt, L., Graves, A., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.05328 · Cited by 1,833
Pixel Recurrent Neural Networks
van den Oord, A., Kalchbrenner, N., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.06759 · Cited by 1,437
Semi-Supervised Learning with Deep Generative Models
Kingma, D. P., Rezende, D. J., Mohamed, S., & Welling, M. (2014) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1406.5298 · Cited by 1,415
f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
Nowozin, S., Cseke, B., & Tomioka, R. (2016) · arXiv · doi:10.48550/arxiv.1606.00709 · Cited by 1,183 · cited 2×
“However we can obtain unbiased samples from it by first sampling x ∼ pD, then z ∼ qφ(z|x), so we can optimize it by likelihood free optimization techniques (Goodfellow et al. 2014; Nowozin, Cseke, and Tomioka 2016; Arjovsky, Chintala, and Bottou 2017; Gretton et al. 2007).”
MethodImportance Weighted Autoencoders
Burda, Y., Grosse, R., & Salakhutdinov, R. (2015) · arXiv · doi:10.48550/arxiv.1509.00519 · Cited by 644
Salimans, T., Karpathy, A., Chen, X., & Kingma, D. P. (2017) · arXiv · doi:10.48550/arxiv.1701.05517 · Cited by 584
Generative Moment Matching Networks
Li, Y., Swersky, K., & Zemel, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.02761 · Cited by 517
On the quantitative analysis of deep belief networks
Salakhutdinov, R., & Murray, I. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390266 · Cited by 411
PixelVAE: A Latent Variable Model for Natural ImagesPreprint
Gulrajani, I., Kumar, K., Ahmed, F., Taïga, A. A., Visin, F., Vázquez, D., & Courville, A. C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05013 · Cited by 355
InfoVAE: Information Maximizing Variational Autoencoders
Zhao, S., Song, J., & Ermon, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02262 · Cited by 328
Self-citationImproved Variational Autoencoders for Text Modeling using Dilated ConvolutionsPreprint
Yang, Z., Hu, Z., Salakhutdinov, R., & Berg-Kirkpatrick, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08139 · Cited by 319
Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
Liu, Q., & Wang, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.04471 · Cited by 292
Training generative neural networks via Maximum Mean Discrepancy optimization
Dziugaite, G. K., Roy, D. M., & Ghahramani, Z. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.03906 · Cited by 159
Amortized Inference RegularizationPreprint
Shu, R., Hung, H.-B., Zhao, S.-J., Kochenderfer, M. J., & Ermon, S. (2018) · Neural Information Processing Systems · doi:10.48550/arxiv.1805.08913 · Cited by 88
Self-citationThe Information Autoencoding Family: A Lagrangian Perspective on Latent Variable Generative ModelsPreprint
Zhao, S., Song, J., & Ermon, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.06514 · Cited by 41
Self-citationAn information-theoretic analysis of deep latent-variable modelsPreprint
Alemi, A. A., Poole, B., Fischer, I., Dillon, J. V., Saurous, R. A., & Murphy, K. P. (2018) · arXiv (Cornell University) · Cited by 35
Cited by
Trajectron++: Dynamically-Feasible Trajectory Forecasting with Heterogeneous Data
Salzmann, T., Ivanovic, B., Chakravarty, P., & Pavone, M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58523-5_40 · Cited by 1,076
Bond-Taylor, S., Leach, A., Long, Y., & Willcocks, C. G. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3116668 · Cited by 703
Deep Semi-Supervised Anomaly Detection
Ruff, L., Vandermeulen, R. A., Görnitz, N., Binder, A., Müller, E., Müller, K.-R., & Kloft, M. (2019) · arXiv · doi:10.48550/arxiv.1906.02694 · Cited by 388
Latent Embedding Feedback and Discriminative Features for Zero-Shot Classification
Narayan, S., Gupta, A., Khan, F. S., Snoek, C. G. M., & Shao, L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58542-6_29 · Cited by 260
Structured Event Memory: A neuro-symbolic model of event cognition.
Franklin, N. T., Norman, K. A., Ranganath, C., Zacks, J. M., & Gershman, S. J. (2020) · Psychological Review · doi:10.1037/rev0000177 · Cited by 177
Metadata
- Authors
- Zhao, S., Song, J., & Ermon, S.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2019-07-17
- Volume
- 33
- Issue
- 01
- Pages
- 5885-5892
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2903782687
- Cited by
- 265
- References
- 29
- FWCI
- 13.92
- Type
- conference paper
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.