Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference
Open accessAbstract
We propose a simple and general variant of the standard reparameterized gradient estimator for the variational evidence lower bound. Specifically, we remove a part of the total derivative with respect to the variational parameters that corresponds to the score function. Removing this term produces an unbiased gradient estimator whose variance approaches zero as the approximate posterior approaches the exact posterior. We analyze the behavior of this gradient estimator theoretically and empirically, and generalize it to more complex variational distributions such as mixtures and importance-weighted posteriors.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
When K > 1, the first term within parenthesis in (7) need not be zero even when the approximate posterior matches the true posterior everywhere, which can contribute to significant variance in the gradient estimator.
“Roeder et al. (2017) find that when K > 1 , the first term within paranthesis in (7) need not be zero even when the approximate posterior matches true posterior everywhere, which can contribute to significant variance in the gradient estimator.”
From Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models (2019) · cites this paper as Roeder et al., 2017
Section 2 (p. 3) says the score-function component of its gradient estimator need not vanish at an exact posterior, leaving nonzero variance. The paper discusses a multi-sample importance-weighted estimator in Section 4 (pp. 5–6), but our copy does not establish the more specific claim about the first term in the citing paper’s equation (7) when K > 1. (LLM, conf=0.83)
Experimental — accuracy not yet measuredThe standard gradient estimator of IWAE can have undesirably high variance.
“The standard gradient estimator of IWAE can have undesirably high variance (Rainforth et al., 2018; Roeder et al., 2017).”
From Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models (2019) · cites this paper as Roeder et al., 2017
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 16
- James Bergstra, Olivier Breuleux, Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, Guillaume Desjardins, Joseph Turian, David Warde-Farley, and Yoshua Bengio. Theano: A cpu and gpu math compiler in python. In Proc. 9th Python in Science Conf , pages 1-7, 2010.
- Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov. Importance weighted autoencoders. arXiv preprint arXiv:1509.00519 , 2015. · arXiv
- Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. arXiv preprint arXiv:1605.08803 , 2016. · arXiv
- Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Aistats , volume 9, pages 249-256, 2010.
- Shaobo Han, Xuejun Liao, David B Dunson, and Lawrence Carin. Variational gaussian copula inference. In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics , volume 51, pages 829-838, 2016.
- Danilo Jimenez Rezende and Shakir Mohamed. Variational inference with normalizing flows. In The 32nd International Conference on Machine Learning , 2015.
- Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. Proceedings of the 3rd international conference on learning representations , 2015.
- Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 , 2013. · arXiv
- Diederik P. Kingma, Tim Salimans, and Max Welling. Improving variational inference with inverse autoregressive flow. Advances in Neural Information Processing Systems 29 , 2016.
- Brenden M Lake. Towards more human-like concept learning in machines: Compositionality, causality, and learning-to-learn . PhD thesis, Massachusetts Institute of Technology, 2014.
- Rajesh Ranganath, Sean Gerrish, and David M Blei. Black box variational inference. In AISTATS , pages 814-822, 2014.
- Danilo J Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In Proceedings of the 31st International Conference on Machine Learning (ICML-14) , pages 1278-1286, 2014.
- Francisco JR Ruiz, Michalis K Titsias, and David M Blei. The generalized reparameterization gradient. arXiv preprint arXiv:1610.02287 , 2016. · arXiv
- Linda SL Tan and David J Nott. Gaussian variational approximation with sparse precision matrices. Statistics and Computing , pages 1-17, 2017.
- Dustin Tran, Alp Kucukelbir, Adji B. Dieng, Maja Rudolph, Dawen Liang, and David M. Blei. Edward: A library for probabilistic modeling, inference, and criticism. arXiv preprint arXiv:1610.09787 , 2016. · arXiv
- Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning , 8(3-4):229-256, 1992.
Cited by
Sorted by citations
An Introduction to Variational Autoencoders
Kingma, D. P., & Welling, M. (2019) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000056 · Cited by 2,609 · see citing works
Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models
Shi, Y., Siddharth, N., Paige, B., & Torr, P. H. S. (2019) · arXiv · doi:10.48550/arxiv.1911.03393 · Cited by 87 · see citing works
“To see this, we can expand (6) as: Roeder et al. (2017) find that when K > 1 , the first term within paranthesis in (7) need not be zero even when the approximate posterior matches true posterior everywhere, which can contribute to significant variance in the gradient estimator.”
Background
- Authors
- Roeder, G., Wu, Y., & Duvenaud, D.
- Affiliations
- Princeton University (US)
- University of Toronto (CA)
- ORCID
- Roeder, Geoffrey
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- Volume
- 30
- Pages
- 6925-6934
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1703.09194
- OpenAlex
- W2743472713
- Cited by
- 84
- References
- 0
- Citations by year
- 2025: 3 · 2024: 5 · 2023: 2 · 2022: 5 · 2021: 12 · 2020: 22 · 2019: 21 · 2018: 12 · 2017: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- neural information processing systems
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- variational inference, score function, unbiased gradient estimation, gradient variance reduction, variational posterior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
