The DOI matches its Crossref record; 185 of the 185 references it cites were found in a registry or an index.
Advances in Variational Inference
Open accessAbstract
Many modern unsupervised or semi-supervised machine learning algorithms rely on Bayesian probabilistic models. These models are usually intractable and thus require approximate inference. Variational inference (VI) lets us approximate a high-dimensional Bayesian posterior with a simpler variational distribution by solving an optimization problem. This approach has been successfully applied to various models and large-scale applications. In this review, we give an overview of recent trends in variational inference. We first introduce standard mean field variational inference, then review recent advances focusing on the following aspects: (a) scalable VI, which includes stochastic approximations, (b) generic VI, which extends the applicability of VI to a large class of otherwise intractable models, such as non-conjugate models, (c) accurate VI, which includes variational models beyond the mean field approximation or with atypical divergences, and (d) amortized VI, which implements the inference over local latent variables with inference networks. Finally, we provide a summary of promising future research directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 22,066
On Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
A Stochastic Approximation Method
Robbins, H., & Monro, S. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729586 · Cited by 9,643
(2010) · Cited by 8,261
Stan : A Probabilistic Programming Language
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddell, A. (2017) · Journal of Statistical Software · doi:10.18637/jss.v076.i01 · Cited by 7,754 · cited 2×
“Popular probabilistic programming tools include but are not limited to: Stan [28], which covers a large range of advanced VI and MCMC methods, Infer.”
MethodLarge-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795
ADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276
Variational Inference: A Review for Statisticians
Blei, D. M., Kucukelbir, A., & McAuliffe, J. D. (2017) · Journal of the American Statistical Association · doi:10.1080/01621459.2017.1285773 · Cited by 3,811
An Introduction to Variational Methods for Graphical Models
Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., & Saul, L. K. (1999) · Machine Learning · doi:10.1023/a:1007665907178 · Cited by 3,791
Spin Glass Theory and Beyond: An Introduction to the Replica Method and Its Applications
Mezard, M., Parisi, G., & Virasoro, M. (1986) · World Scientific Lecture Notes in Physics · doi:10.1142/0271 · Cited by 3,653
Categorical Reparameterization with Gumbel-Softmax
Jang, E., Gu, S., & Poole, B. (2016) · arXiv · doi:10.48550/arxiv.1611.01144 · Cited by 3,616
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Handbook of Markov Chain Monte Carlo
Brooks, S., Gelman, A., Jones, G., & Meng, X.-L. (2011) · Chapman & Hall/CRC eBooks · doi:10.1201/b10905 · Cited by 2,993
Graphical Models, Exponential Families, and Variational Inference
Wainwright, M. J., & Jordan, M. I. (2007) · now publishers, Inc. eBooks · doi:10.1561/9781601981851 · Cited by 2,959
Introduction to Mathematical Statistics.
AA, Hogg, R. V., & Craig, A. T. (1996) · Journal of the American Statistical Association · doi:10.2307/2291439 · Cited by 2,888
Mezard, M., Parisi, G., Virasoro, M. A., & Thouless, D. J. (1988) · Physics Today · doi:10.1063/1.2811676 · Cited by 2,682
Natural Gradient Works Efficiently in Learning
Amari, S.-I. (1998) · Neural Computation · doi:10.1162/089976698300017746 · Cited by 2,681
Pattern recognition and machine learning
(2007) · Choice Reviews Online · doi:10.5860/choice.44-5091 · Cited by 2,650
The information bottleneck method
Tishby, N., Pereira, F. C., & Bialek, W. (2000) · arXiv · doi:10.48550/arxiv.physics/0004057 · Cited by 2,626 · cited 2×
“For example, the information bottleneck [193] has recently led to the deep variational”
BackgroundStochastic Backpropagation and Approximate Inference in Deep Generative Models
Rezende, D. J., Mohamed, S., & Wierstra, D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1401.4082 · Cited by 2,405
Blei, D. M., & Lafferty, J. D. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143859 · Cited by 2,386
Accurate Approximations for Posterior Moments and Marginal Densities
Tierney, L., & Kadane, J. B. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478240 · Cited by 2,129
Generating Sentences from a Continuous Space
Bowman, S. R., Vilnis, L., Vinyals, O., Dai, A., Jozefowicz, R., & Bengio, S. (2016) · Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning · doi:10.18653/v1/k16-1002 · Cited by 2,061
Probabilistic machine learning and artificial intelligence
Ghahramani, Z. (2015) · Nature · doi:10.1038/nature14541 · Cited by 1,995
Variational Inference: A Review for Statisticians
Blei, D. M., Kucukelbir, A., & McAuliffe, J. (2017) · Figshare · doi:10.6084/m9.figshare.5203696 · Cited by 1,919
Accelerating Stochastic Gradient Descent using Predictive Variance Reduction
Johnson, R., & Zhang, T. (2013) · Cited by 1,889
Graphical Models, Exponential Families, and Variational Inference
Wainwright, M. J., & Jordan, M. I. (2008) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000001 · Cited by 1,820
Differential-Geometrical Methods in Statistics
Amari, S.-I. (1985) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-5056-2 · Cited by 1,673
Bayesian Learning via Stochastic Gradient Langevin Dynamics
Welling, M., & Teh, Y. W. (2012) · Cited by 1,551
NICE: Non-linear Independent Components Estimation
Dinh, L., Krueger, D., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1410.8516 · Cited by 1,493
Eddy, S. R. (1996) · Current Opinion in Structural Biology · doi:10.1016/s0959-440x(96)80056-x · Cited by 1,430
Sparse Gaussian Processes using Pseudo-inputs
Snelson, E., & Ghahramani, Z. (2005) · Cited by 1,415
Loopy Belief Propagation for Approximate Inference: An Empirical Study
Murphy, K. P., Weiss, Y., & Jordan, M. I. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.6725 · Cited by 1,397
Expectation Propagation for approximate Bayesian inference
Minka, T. P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.2294 · Cited by 1,392
Online Learning for Latent Dirichlet Allocation
Hoffman, M. D., Blei, D. M., & Bach, F. (2010) · Cited by 1,363
The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
Maddison, C. J., Mnih, A., & Teh, Y. W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.00712 · Cited by 1,359
Options: A Monte Carlo approach
Boyle, P. P. (1977) · Journal of Financial Economics · doi:10.1016/0304-405x(77)90005-8 · Cited by 1,336
Stochastic variational inference
Hoffman, M. D., Blei, D. M., Wang, C., & Paisley, J. W. (2013) · Journal of Machine Learning Research · doi:10.5555/2567709.2502622 · Cited by 1,333
Solution of 'Solvable model of a spin glass'
Thouless, D. J., Anderson, P. W., & Palmer, R. G. (1977) · Philosophical Magazine · doi:10.1080/14786437708235992 · Cited by 1,286
Variational Inference with Normalizing Flows
Rezende, D. J., & Mohamed, S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05770 · Cited by 1,253
Dayan, P., Hinton, G. E., Neal, R. M., & Zemel, R. S. (1995) · Neural Computation · doi:10.1162/neco.1995.7.5.889 · Cited by 1,207
A General Class of Coefficients of Divergence of One Distribution from Another
Ali, S. M., & Silvey, S. D. (1966) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1966.tb00626.x · Cited by 1,205
Variational Learning of Inducing Variables in Sparse Gaussian Processes
Titsias, M. K. (2009) · Cited by 1,083
Probabilistic Inference Using Markov Chain Monte Carlo Methods
Neal, R. M. (2011) · Cited by 1,040
Stein, C. (1972) · Probability Theory · doi:10.1525/9780520423671-036 · Cited by 1,029
Lafferty, J., & Blei, D. M. (2005) · neural information processing systems · Cited by 917
Itzykson, C., & Drouffe, J.-M. (1989) · doi:10.1017/cbo9780511622779 · Cited by 885
(2018) · Physics Subject Headings (PhySH) · doi:10.29172/0660b3fe-654b-49c9-a7fa-79f18efe2aa0 · Cited by 882
Simple statistical gradient-following algorithms for connectionist reinforcement learning
Williams, R. J. (1992) · Machine Learning · doi:10.1023/a:1022672621406 · Cited by 817
DRAW: A Recurrent Neural Network For Image Generation
Gregor, K., Danihelka, I., Graves, A., Rezende, D. J., & Wierstra, D. (2015) · International Conference on Machine Learning · Cited by 785
Parameter estimation for text analysis
Heinrich, G. (2009) · Cited by 700
(2005) · Edinburgh Research Explorer (University of Edinburgh) · doi:10.5555/1046920.1088695 · Cited by 650
Importance Weighted Autoencoders
Burda, Y., Grosse, R., & Salakhutdinov, R. (2015) · arXiv · doi:10.48550/arxiv.1509.00519 · Cited by 644
Fast collapsed gibbs sampling for latent dirichlet allocation
Porteous, I., Newman, D., Ihler, A., Asuncion, A., Smyth, P., & Welling, M. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401960 · Cited by 560
Integral Probability Metrics and Their Generating Classes of Functions
Müller, A. (1997) · Advances in Applied Probability · doi:10.1017/s000186780002807x · Cited by 548
Autoencoding Variational Inference For Topic ModelsPreprint
Srivastava, A., & Sutton, C. A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01488 · Cited by 545
Simultaneous image classification and annotation
Chong, W., Blei, D., & Li, F.-F. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206800 · Cited by 510
A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation
Teh, Y. W., Newman, D., & Welling, M. (2007) · doi:10.21236/ada629956 · Cited by 490
Artificial Intelligence with Uncertainty
Li, D., & Du, Y. (2017) · CRC Press eBooks · doi:10.1201/9781315366951 · Cited by 481
Divergence measures and message passing
Minka, T. P. (2005) · Cited by 477
Learning in Implicit Generative ModelsPreprint
Mohamed, S., & Lakshminarayanan, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.03483 · Cited by 469
An Introduction to Mathematical Statistics
Rozanov, Y. A. (1995) · Probability Theory, Random Processes and Mathematical Statistics · doi:10.1007/978-94-011-0449-4_3 · Cited by 434
Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial NetworksPreprint
Mescheder, L., Nowozin, S., & Geiger, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.04722 · Cited by 425
Gaussian Processes for Big Data
Hensman, J. J., Fusi, N., & Lawrence, N. D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1309.6835 · Cited by 393
Advanced mean field methods: theory and practice
Opper, M., & Saad, D. (2001) · Cited by 385
Amortized Inference in Probabilistic Reasoning
Gershman, S. J., & Goodman, N. D. (2014) · eScholarship (California Digital Library) · Cited by 384
A deterministic annealing approach to clustering
Rose, K., Gurewitz, E., & Fox, G. (1990) · Pattern Recognition Letters · doi:10.1016/0167-8655(90)90010-y · Cited by 382
A New Class of Upper Bounds on the Log Partition Function
Wainwright, M. J., Jaakkola, T. S., & Willsky, A. S. (2005) · IEEE Transactions on Information Theory · doi:10.1109/tit.2005.850091 · Cited by 381
Online Model Selection Based on the Variational Bayes
Sato, M.-A. (2001) · Neural Computation · doi:10.1162/089976601750265045 · Cited by 375
Sample size selection in optimization methods for machine learning
Byrd, R. H., Chin, G. M., Nocedal, J., & Wu, Y. (2012) · Mathematical Programming · doi:10.1007/s10107-012-0572-5 · Cited by 369
Stochastic Gradient Descent as Approximate Bayesian Inference
Mandt, S., Hoffman, M. D., & Blei, D. M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.04289 · Cited by 369
Self-citationReinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1805.00909 · Cited by 366
Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
Salimans, T., Kingma, D. P., & Welling, M. (2014) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1410.6460 · Cited by 343
Advanced Mean Field Methods: Theory and Practice
(2001) · The MIT Press eBooks · doi:10.7551/mitpress/1100.001.0001 · Cited by 336
Online Variational Inference for the Hierarchical Dirichlet Process
Wang, C., Paisley, J. W., & Blei, D. M. (2011) · Cited by 328
Mean Field Theory for Sigmoid Belief Networks
Saul, L. K., Jaakkola, T., & Jordan, M. I. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.251 · Cited by 327
Convergence condition of the TAP equation for the infinite-ranged Ising spin glass model
Plefka, T. (1982) · Journal of Physics A: Mathematical and General · doi:10.1088/0305-4470/15/6/035 · Cited by 327
Hybrid deterministic-stochastic methods for data fitting
Friedlander, M. P., & Schmidt, M. (2011) · Cited by 314
The IM Algorithm: A Variational Approach to Information Maximization.
Barber, D., & Agakov, F. V. (2003) · UCL Discovery (University College London) · Cited by 295
Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
Liu, Q., & Wang, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.04471 · Cited by 292
Variational Bayesian Inference with Stochastic Search
Paisley, J. W., Blei, D. M., & Jordan, M. I. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6430 · Cited by 275
Streaming Variational BayesPreprint
Broderick, T., Boyd, N. R., Wibisono, A. Y., Wilson, A., & Jordan, M. I. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1307.6769 · Cited by 265
Hybrid Deterministic-Stochastic Methods for Data Fitting
Friedlander, M. P., & Schmidt, M. (2012) · SIAM Journal on Scientific Computing · doi:10.1137/110830629 · Cited by 254
Hierarchical Implicit Models and Likelihood-Free Variational InferencePreprint
Tran, D., Ranganath, R., & Blei, D. M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08896 · Cited by 252
Damianou, A., & Lawrence, N. D. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1211.0358 · Cited by 249
Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression
Salimans, T., & Knowles, D. A. (2013) · Bayesian Analysis · doi:10.1214/13-ba858 · Cited by 246
Stochastic Gradient VB and the Variational Auto-EncoderPreprint
Kingma, D. P., & Welling, M. (2013) · arXiv (Cornell University) · Cited by 234
$\alpha$-Divergence Is Unique, Belonging to Both $f$-Divergence and Bregman Divergence Classes
Амари, Ш.-И. (2009) · IEEE Transactions on Information Theory · doi:10.1109/tit.2009.2030485 · Cited by 230
Stochastic Optimization with Importance Sampling for Regularized Loss Minimization
Zhao, P., & Zhang, T. (2015) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · Cited by 228
Chen, X., Kingma, D. P., Salimans, T., Duan, Y., Dhariwal, P., Schulman, J., Sutskever, I., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02731 · Cited by 226
Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in AdamPreprint
Khan, M. E., Nielsen, D., Tangkaratt, V., Wu, L., Gal, Y., & Srivastava, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.04854 · Cited by 211
Neural Variational Inference for Text Processing
Miao, Y., Yu, L., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06038 · Cited by 210
Krishnan, R. G., Shalit, U., & Sontag, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05121 · Cited by 206
A kernelized stein discrepancy for goodness-of-fit tests
Liu, Q., Lee, J. D., & Jordan, M. I. (2016) · International Conference on Machine Learning · Cited by 206
REBAR: Low-variance, unbiased gradient estimates for discrete latent variable modelsPreprint
Tucker, G., Mnih, A., Maddison, C. J., Lawson, D., & Sohl‐Dickstein, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07370 · Cited by 201
Edward: A library for probabilistic modeling, inference, and criticism
Tran, D., Kucukelbir, A., Dieng, A. B., Rudolph, M., Liang, D., & Blei, D. M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.09787 · Cited by 199
Bayesian Posterior Sampling via Stochastic Gradient Fisher Scoring
Ahn, S., Korattikara, A., & Welling, M. (2012) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1206.6380 · Cited by 195
Memoir on the Probability of the Causes of Events
Laplace, P. S. (1986) · Statistical Science · doi:10.1214/ss/1177013621 · Cited by 193
Variational Heteroscedastic Gaussian Process Regression
Titsias, M. K., & Lázaro-Gredilla, M. (2011) · International Conference on Machine Learning · Cited by 190
Dynamic Word EmbeddingsPreprint
Bamler, R., & Mandt, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08359 · Cited by 184
Self-citationKorattikara, A., Rathod, V., Murphy, K., & Welling, M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.04416 · Cited by 182
Frequentist Consistency of Variational Bayes
Wang, Y., & Blei, D. M. (2019) · Journal of the American Statistical Association · doi:10.1080/01621459.2018.1473776 · Cited by 174
Variational Inference in Nonconjugate Models
Wang, C., & Blei, D. M. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1209.4360 · Cited by 169
Mescheder, L., Nowozin, S., & Geiger, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.10461 · Cited by 165
Control Functionals for Monte Carlo Integration
Oates, C. J., Girolami, M., & Chopin, N. (2016) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/rssb.12185 · Cited by 153
PixelVAE: A Latent Variable Model for Natural ImagesPreprint
Gulrajani, I., Kumar, K., Ahmed, F., Taïga, A. A., Visin, F., Vázquez, D., & Courville, A. C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05013 · Cited by 140
Variational Inference using Implicit DistributionsPreprint
Huszár, F. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08235 · Cited by 138
Expectation Propagation for Exponential Families
Seeger, M. (2005) · Cited by 131
Stein Variational Policy GradientPreprint
Liu, Y., Ramachandran, P., Liu, Q., & Peng, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.02399 · Cited by 125
Mr. LDA: a flexible large scale topic modeling package using variational inference in MapReduce
Zhai, K., Boyd-Graber, J. L., Asadi, N., & Alkhouja, M. (2012) · International World Wide Web Conference (WWW) · doi:10.1145/2187836.2187955 · Cited by 124
Black-Box α-divergence minimizationPreprint
Hernández-Lobato, J. M., Li, Y., Rowland, M., Hernández-Lobato, D., Bui, T. D., & Turner, R. E. (2015) · arXiv (Cornell University) · doi:10.17863/cam.9689 · Cited by 118
Towards Deeper Understanding of Variational Autoencoding Models
Zhao, S., Song, J., & Ermon, S. (2017) · arXiv · doi:10.48550/arxiv.1702.08658 · Cited by 116
Variational Policy Search via Trajectory Optimization
Levine, S., & Koltun, V. (2013) · neural information processing systems · Cited by 113
Rényi divergence variational inference
Li, Y., & Turner, R. E. (2016) · Apollo (University of Cambridge) · doi:10.17863/cam.21344 · Cited by 113
Nonparametric variational inferencePreprint
Gershman, S. J., Hoffman, M., & Blei, D. M. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.4665 · Cited by 111
Variational Auto-encoded Deep Gaussian ProcessesPreprint
Dai, Z., Damianou, A., González, J., & Lawrence, N. D. (2015) · arXiv (Cornell University) · doi:10.17863/cam.47957 · Cited by 106
Improving the Mean Field Approximation Via the Use of Mixture Distributions
Jaakkola, T., & Jordan, M. I. (1998) · Springer eBooks · doi:10.1007/978-94-011-5014-9_6 · Cited by 104
Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial LearningPreprint
Wang, D., & Liu, Q. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01722 · Cited by 99
Overlapping Mixtures of Gaussian Processes for the data association problem
Lázaro-Gredilla, M., Vaerenbergh, S. V., & Lawrence, N. D. (2011) · Pattern Recognition · doi:10.1016/j.patcog.2011.10.004 · Cited by 97
Reinterpreting Importance-Weighted AutoencodersPreprint
Cremer, C., Morris, Q., & Duvenaud, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.02916 · Cited by 97
Variance Reduction for Stochastic Gradient Optimization
Wang, C., Chen, X., Smola, A. J., & Xing, E. P. (2018) · doi:10.1184/r1/6476438 · Cited by 95
Patterns of Scalable Bayesian Inference
Angelino, E. L., Johnson, M. J., & Adams, R. P. (2016) · now publishers, Inc. eBooks · doi:10.1561/9781680832198 · Cited by 92
Non-conjugate Variational Message Passing for Multinomial and Binary Regression
Knowles, D. A., & Minka, T. (2011) · Cited by 88
Opper, M., & Winther, O. (2001) · Physical Review Letters · doi:10.1103/physrevlett.86.3695 · Cited by 87
Scalable Inference of Overlapping Communities
Gopalan, P., Gerrish, S., Freedman, M. J., Blei, D. M., & Mimno, D. (2012) · Cited by 83
Azevedo-Filho, A., & Shachter, R. D. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-332-5.50009-2 · Cited by 82
Accelerating Minibatch Stochastic Gradient Descent using Stratified SamplingPreprint
Zhao, P., & Zhang, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.3080 · Cited by 82
Fast Variational Inference in the Conjugate Exponential FamilyPreprint
Hensman, J. J., Rattray, M., & Lawrence, N. D. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.5162 · Cited by 74
An Adaptive Learning Rate for Stochastic Variational Inference
Ranganath, R., Wang, C., Blei, D. M., & P Xing, E. (2018) · doi:10.1184/r1/6475463 · Cited by 74
Stochastic Expectation Propagation
Li, Y., Hernández-Lobato, J. M., & Turner, R. E. (2015) · Apollo (University of Cambridge) · doi:10.17863/cam.21346 · Cited by 73
A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation
Teh, Y. W., Newman, D. J., & Welling, M. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0174 · Cited by 73
Boosting Variational InferencePreprint
Guo, F., Wang, X., Fan, K., Broderick, T., & Dunson, D. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05559 · Cited by 73
On Nesting Monte Carlo EstimatorsPreprint
Rainforth, T., Cornish, R., Yang, H., Warrington, A., & Wood, F. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.06181 · Cited by 73
Stochastic Variational Inference for Hidden Markov ModelsPreprint
Foti, N. J., Xu, J., Laird, D., & Fox, E. B. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.1670 · Cited by 72
Information Geometry of Mean-Field Approximation
Tanaka, T. (2000) · Neural Computation · doi:10.1162/089976600300015213 · Cited by 71
A Variational Analysis of Stochastic Gradient AlgorithmsPreprint
Mandt, S., Hoffman, M. D., & Blei, D. M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02666 · Cited by 71
Self-citationHow to Train Deep Variational Autoencoders and Probabilistic Ladder NetworksPreprint
Sønderby, C. K., Raiko, T., Maaløe, L., Sønderby, S. K., & Winther, O. (2016) · arXiv (Cornell University) · Cited by 69
Approximate Riemannian Conjugate Gradient Learning for Fixed-Form Variational Bayes
Honkela, A., Raiko, T., Kuusela, M., Tornio, M., & Karhunen, J. (2010) · doi:10.5555/1756006.1953035 · Cited by 67
Tighter Variational Bounds are Not Necessarily Better
Rainforth, T., Kosiorek, A. R., Le, T. A., Maddison, C. J., Igl, M., Wood, F., & Teh, Y. W. (2018) · arXiv · doi:10.48550/arxiv.1802.04537 · Cited by 66
Operator Variational InferencePreprint
Ranganath, R., Altosaar, J., Tran, D., & Blei, D. M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.09033 · Cited by 65
Factorized Variational Autoencoders for Modeling Audience Reactions to Movies
Deng, Z., Navarathna, R., Carr, P., Mandt, S., Yue, Y., Matthews, I., & Mori, G. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.637 · Cited by 63
Self-citationEarly Stopping as Nonparametric Variational Inference
Duvenaud, D., Maclaurin, D., & Adams, R. P. (2016) · International Conference on Artificial Intelligence and Statistics · Cited by 60
Stochastic Variational Inference for Bayesian Time Series Models
Johnson, M., & Willsky, A. S. (2014) · Cited by 59
Structured Stochastic Variational InferencePreprint
Hoffman, M. D., & Blei, D. M. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.4114 · Cited by 58
Tensor Space Learning for Analyzing Activity Patterns from Video Sequences
Wang, L., Leckie, C. A., Wang, X., Ramamohanarao, K., & Bezdek, J. (2007) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2007.70 · Cited by 58
Variational Boosting: Iteratively Refining Posterior ApproximationsPreprint
Miller, A. C., Foti, N. J., & Adams, R. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.06585 · Cited by 58
Approximate Inference with Amortised MCMCPreprint
Li, Y., Turner, R. E., & Liu, Q. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08343 · Cited by 57
Big Batch SGD: Automated Inference using Adaptive Batch SizesPreprint
De, S., Yadav, A. K., Jacobs, D. W., & Goldstein, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.05792 · Cited by 56
Variational Bayes Solution of Linear Neural Networks and Its Generalization Performance
Nakajima, S., & Watanabe, S. (2007) · Neural Computation · doi:10.1162/neco.2007.19.4.1112 · Cited by 50
Local expectation gradients for black box variational inference
Titsias, M. K., & Lázaro-Gredilla, M. (2015) · neural information processing systems · Cited by 50
Stochastic gradient variational Bayes for gamma approximating distributionsPreprint
Knowles, D. A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.01631 · Cited by 49
Minka, T. (2004) · Cited by 49
Approximate inference for the loss-calibrated Bayesian
Lacoste-Julien, S., Huszár, F., & Ghahramani, Z. (2011) · Cited by 45
Information geometric measurements of generalisation
Zhu, H., & Rohwer, R. (1995) · Aston Publications Explorer (Aston University) · Cited by 44
ZhuSuan: A Library for Bayesian Deep LearningPreprint
Shi, J., Chen, J., Zhu, J., Sun, S., Luo, Y., Gu, Y., & Zhou, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.05870 · Cited by 44
Active Mini-Batch Sampling Using Repulsive Point Processes
Zhang, C., Öztireli, C., Mandt, S., & Salvi, G. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33015741 · Cited by 44
Self-citationImproving Variational Autoencoders with Inverse Autoregressive Flow
Kingma, D. P., Salimans, T., Józefowicz, R., Chen, X., Sutskever, I., & Welling, M. (2016) · UvA-DARE (University of Amsterdam) · Cited by 39
Natural Conjugate Gradient in Variational Inference
Honkela, A., Tornio, M., Raiko, T., & Karhunen, J. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-69162-4_32 · Cited by 36
Nallapati, R. M., Cohen, W. W., & Lafferty, J. (2007) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2007.33 · Cited by 34
Nowozin, S. (2018) · International Conference on Learning Representations · Cited by 34
Latent-Space Variational Bayes
Sung, J., Ghahramani, Z., & Bang, S.-Y. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.157 · Cited by 33
A Theory of Mean Field Approximation
Tanaka, T. (1998) · Cited by 29
Faster Coordinate Descent via Adaptive Importance SamplingPreprint
Perekrestenko, D., Cevher, V., & Jaggi, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.02518 · Cited by 29
Variational Message Passing with Structured Inference NetworksPreprint
Wu, L., Hubacher, N., & Khan, M. E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.05589 · Cited by 29
Variational Gaussian Copula InferencePreprint
Han, S., Liao, X., Dunson, D. B., & Carin, L. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.05860 · Cited by 28
Fast Learning by Bounding Likelihoods in Sigmoid Type Belief Networks
Jaakkola, T., Saul, L. K., & Jordan, M. I. (1995) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 24
Stochastic variational inference for large-scale discrete choice models using adaptive batch sizes
Tan, L. S. L. (2015) · Statistics and Computing · doi:10.1007/s11222-015-9618-x · Cited by 23
Kullback-Leibler proximal variational inference
Khan, M. E., Baqué, P., Fleuret, F., & Fua, P. (2015) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 22
Adversarial Message Passing For Graphical ModelsPreprint
Karaletsos, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.05048 · Cited by 22
Stein Variational Adaptive Importance SamplingPreprint
Han, J. G., & Liu, Q. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.05201 · Cited by 22
A trust-region method for stochastic variational inference with applications to streaming dataPreprint
Theis, L., & Hoffman, M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.07649 · Cited by 21
Perturbative Black Box Variational InferencePreprint
Bamler, R., Zhang, C., Opper, M., & Mandt, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.07433 · Cited by 19
Self-citationStructured VAEs: Composing Probabilistic Graphical Models and Variational AutoencodersPreprint
Johnson, M., Duvenaud, D., Wiltschko, A. B., Datta, S. R., & Adams, R. P. (2016) · arXiv (Cornell University) · Cited by 14
Learning Model Reparametrizations: Implicit Variational Inference by Fitting MCMC distributionsPreprint
Titsias, M. K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.01529 · Cited by 14
Two Methods For Wild Variational InferencePreprint
Liu, Q., & Feng, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00081 · Cited by 9
Structured Black Box Variational Inference for Latent Time Series ModelsPreprint
Bamler, R., & Mandt, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.01069 · Cited by 8
Self-citationKucukelbir, A., & Blei, D. M. (2015) · Uncertainty in Artificial Intelligence · Cited by 7
Approximate Inference: New Visions
Li, Y. (2017) · Apollo (University of Cambridge) · doi:10.17863/cam.24865 · Cited by 4
Cited by
Generalizing from a Few Examples: A Survey on Few-shot Learning
Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2021) · ACM Computing Surveys · doi:10.1145/3386252 · Cited by 3,165
MOFA+: a statistical framework for comprehensive integration of multi-modal single-cell data
Argelaguet, R., Arnol, D., Bredikhin, D., Deloro, Y., Velten, B., Marioni, J. C., & Stegle, O. (2020) · Genome Biology · doi:10.1186/s13059-020-02015-1 · Cited by 1,091
Generalizing from a Few Examples: A Survey on Few-Shot Learning
Wang, Y., Yao, Q., Kwok, J. T.-Y., & Ni, L. M.-S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.05046 · Cited by 772
Autoencoder and Its Various Variants
Zhai, J., Zhang, S., Chen, J., & He, Q. (2018) · 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC) · doi:10.1109/smc.2018.00080 · Cited by 339
Load Forecasting Techniques for Power System: Research Challenges and Survey
Ahmad, N., Ghadi, Y., Adnan, M., & Ali, M. (2022) · IEEE Access · doi:10.1109/access.2022.3187839 · Cited by 323
Metadata
- Authors
- Zhang, C., Butepage, J., Kjellstrom, H., & Mandt, S.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2019-08-01
- Volume
- 41
- Issue
- 8
- Pages
- 2008-2026
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2962851448
- Cited by
- 671
- References
- 185
- FWCI
- 32.53
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.