The DOI matches its DataCite record; 13 of the 14 references it cites were found in a registry or an index.
Auto-Encoding Variational Bayes
Open accessAbstract
How can we perform efficient inference and learning in directed probabilistic models, in the presence of continuous latent variables with intractable posterior distributions, and large datasets? We introduce a stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case. Our contributions are two-fold. First, we show that a reparameterization of the variational lower bound yields a lower bound estimator that can be straightforwardly optimized using standard stochastic gradient methods. Second, we show that for i.i.d. datasets with continuous latent variables per datapoint, posterior inference can be made especially efficient by fitting an approximate inference model (also called a recognition model) to the intractable posterior using the proposed lower bound estimator. Theoretical advantages are reflected in experimental results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In fact, displaced vacuum states have Gaussian distributions in phase space, so the network has a resemblance to a variational autoencoder [121].”
From Continuous-variable quantum neural networks (2019) · cites this paper as [121]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A large amount of machine learning research is devoted to circumventing this problem by instead maximizing a suitable lower bound on this quantity, referred to as the evidence lower bound (ELBO) and sometimes as negative variational free energy.7”
From The overfitted brain hypothesis (2021) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We implement action chunking policy with Transformers [69], an architecture designed for sequence modeling, and train it as a conditional VAE (CVAE) [59, 34] to capture the variability in human data.”
From Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware (2023) · cites this paper as [34]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The forward process variances β t can be learned by reparameterization [33] or held constant as hyperparameters”
From Denoising Diffusion Probabilistic Models (2020) · cites this paper as [33]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Variational Autoencoders (VAEs) [99] are effective methods to model the posterior.”
From A review of uncertainty quantification in deep learning: Techniques, applications and challenges (2021) · cites this paper as [99]
The cited source says what the claim attributes to it.
In the citing paper
“Earlier efforts in this field focused on deep-learning-based generative models, including Generative Adversarial Networks (GANs) [268], Variational AutoEncoders (VAEs) [269], Normalizing Flows [270], and diffusion-based models [271] for high-quality image generation.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [269]
The cited source says what the claim attributes to it.
In the citing paper
“Generative approaches, such as auto-encoders (Tran et al. 2017; Lee et al. 2019), GANs (Goodfellow et al. 2014), and VAEs (Kingma and Welling 2013), offer a straightforward solution to handle this setting, but these methods are neithor flexible nor efficient as SMIL.”
From SMIL: Multimodal Learning with Severely Missing Modality (2021) · cites this paper as Kingma and Welling 2013
The cited source says what the claim attributes to it.
In the citing paper
“We highlight that our method is better than typical generative designs, such as Autoencoder (AE) (Tran et al. 2017), Variational Autoencoder (VAE) (Kingma and Welling 2013), or Generative Adversarial Network (GAN) (Goodfellow et al. 2014), since they often require a significant amount of full-modality data to learn from, which is usually not available in severely missing modality learning.”
From SMIL: Multimodal Learning with Severely Missing Modality (2021) · cites this paper as Kingma and Welling 2013
The cited source says what the claim attributes to it.
In the citing paper
“In contrast to deterministic approaches, [Liao et al., 2021] adopts probabilistic entity representations, based on variational Bayesian inference [Kingma and Welling, 2013], to jointly model the entity features and the uncertainty.”
From Temporal Knowledge Graph Completion: A Survey (2023) · cites this paper as Kingma and Welling, 2013
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Duane, S., Kennedy, A. D., Pendleton, B. J., & Roweth, D. (1987) · Physics Letters B · doi:10.1016/0370-2693(87)91197-x · Cited by 4,081
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051
Stochastic Variational InferencePreprint
Hoffman, M. D., Blei, D. M., Wang, C., & Paisley, J. W. (2012) · arXiv (Cornell University) · doi:10.5555/2567709.2502622 · Cited by 1,991
EM Algorithms for PCA and SPCA
Roweis, S. T. (1997) · Cited by 780
Black Box Variational InferencePreprint
Ranganath, R., Gerrish, S., & Blei, D. M. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1401.0118 · Cited by 735
Variational Bayesian Inference with Stochastic SearchPreprint
Paisley, J. W., Blei, D. M., & Jordan, M. I. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6430 · Cited by 372
Efficient Learning of Deep Boltzmann Machines
Salakhutdinov, R., & Larochelle, H. (2010) · Cited by 352
Fast Inference in Sparse Coding Algorithms with Applications to Object RecognitionPreprint
Kavukcuoglu, K., Ranzato, M., & LeCun, Y. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1010.3467 · Cited by 222
Sample-based non-uniform random variate generation
Devroye, L. (1986) · Winter Simulation Conference · doi:10.1145/318242.318443 · Cited by 216
Deep AutoRegressive NetworksPreprint
Gregor, K., Danihelka, I., Mnih, A., Blundell, C., & Wierstra, D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1310.8499 · Cited by 192
Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression
Salimans, T., & Knowles, D. A. (2013) · Bayesian Analysis · doi:10.1214/13-ba858 · Cited by 173
An Application of the Principle of Maximum Information Preservation to Linear Systems
Linsker, R. (1988) · Cited by 150
Research works printed but not linked: 4
- [BTL13] Yoshua Bengio and ´ Eric Thibodeau-Laufer. Deep generative stochastic networks trainable by backprop. arXiv preprint arXiv:1306.1091 , 2013. · arXiv
- [DHS10] John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research , 12:21212159, 2010.
- [HDFN95] Geoffrey E Hinton, Peter Dayan, Brendan J Frey, and Radford M Neal. The' wakesleep' algorithm for unsupervised neural networks. SCIENCE , pages 1158-1158, 1995.
- [RMW14] Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and variational inference in deep latent gaussian models. arXiv preprint arXiv:1401.4082 , 2014. · arXiv
Cited by
Sorted by citations
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Self-citationUnpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 20,507
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
Metadata
- Authors
- Kingma, D. P., & Welling, M.
- Affiliations
- University of Amsterdam (NL)
- ORCID
- Welling, Max
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2013
- arXiv
- 1312.6114
- OpenAlex
- W1959608418
- Cited by
- 20,067
- References
- 14
- Citations by year
- 2026: 414 · 2025: 1878 · 2024: 2998 · 2023: 2873 · 2022: 2374 · 2021: 3248 · 2020: 2621 · 2019: 1977 · 2018: 962 · 2017: 461 · 2016: 160 · 2015: 74 · 2014: 19 · 2013: 1
- Type
- posted content
- Language
- en
- Open access
- green
- dare.uva.nl
- Licence
- Locations
- UvA-DARE (University of Amsterdam) (publishedVersion, other-oa)
- arXiv (Cornell University) (submittedVersion)
- UvA-DARE (University of Amsterdam) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
- Keywords
- autoencoder, variational inference, stochastic variational inference, variational lower bound, reparameterization, continuous latent variables, recognition model, large-scale machine learning, stochastic gradient optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
