CoChat
Sign Up
PreprintCited by 6

The DOI matches its DataCite record; 2 of the 2 references it cites were found in a registry or an index.

Composing graphical models with neural networks for structured representations and fast inference

Johnson, Matthew J., Duvenaud, David, Wiltschko, Alexander B., Datta, Sandeep R., Adams, Ryan P.·2016·arXiv
DOI 10.48550/arxiv.1603.06277arXiv 1603.06277
Open this paper in CoChat — free account

Abstract

We propose a general modeling and inference framework that composes probabilistic graphical models with deep learning methods and combines their respective strengths. Our model family augments graphical structure in latent variables with neural network observation models. For inference, we extend variational autoencoders to use graphical model approximating distributions with recognition networks that output conjugate potentials. All components of these models are learned simultaneously with a single objective, giving a scalable algorithm that leverages stochastic variational inference, natural gradients, graphical model message passing, and the reparameterization trick. We illustrate this framework with several example models and an application to mouse behavioral phenotyping.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1603.06277

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 2 references

Research works printed but not linked: 26

  1. Daphne Koller and Nir Friedman. Probabilistic graphical models: principles and techniques. MIT Press, 2009.
  2. Kevin P Murphy. Machine Learning: a Probabilistic Perspective. MIT Press, 2012.
  3. Alexander B. Wiltschko, Matthew J. Johnson, Giuliano Iurilli, Ralph E. Peterson, Jesse M. Katon, Stan L. Pashkovski, Victoria E. Abraira, Ryan P. Adams, and Sandeep Robert Datta. "Mapping Sub-Second Structure in Mouse Behavior". In: Neuron 88.6 (2015), pp. 1121-1135.
  4. Li Deng. "Computational models for speech production". In: Computational Models of Speech Pattern Processing. Springer, 1999, pp. 199-213.
  5. Li Deng. "Switching dynamic system models for speech articulation and acoustics". In: Mathematical Foundations of Speech and Language Processing. Springer, 2004, pp. 115-133.
  6. Danilo J Rezende, Shakir Mohamed, and Daan Wierstra. "Stochastic Backpropagation and Approximate Inference in Deep Generative Models". In: Proceedings of the 31st International Conference on Machine Learning. 2014, pp. 1278-1286.
  7. Matthew J. Johnson and Alan S. Willsky. "Stochastic Variational Inference for Bayesian Time Series Models". In: International Conference on Machine Learning. 2014.
  8. Matthew D. Hoffman, David M. Blei, Chong Wang, and John Paisley. "Stochastic variational inference". In: Journal of Machine Learning Research (2013).
  9. James Martens. "New insights and perspectives on the natural gradient method". In: arXiv preprint arXiv:1412.1193 (2015). · arXiv
  10. David J.C. MacKay and Mark N. Gibbs. "Density networks". In: Statistics and neural networks: advances at the interface. Oxford University Press, Oxford (1999), pp. 129-144.
  11. Tomoharu Iwata, David Duvenaud, and Zoubin Ghahramani. "Warped Mixtures for Nonpara- metric Cluster Shapes". In: Conference on Uncertainty in Artificial Intelligence (UAI). 2013, pp. 311-319.
  12. E.B. Fox, E.B. Sudderth, M.I. Jordan, and A.S. Willsky. "Bayesian Nonparametric Inference of Switching Dynamic Linear Models". In: IEEE Transactions on Signal Processing 59.4 (2011).
  13. Shun-Ichi Amari. "Natural gradient works efficiently in learning". In: Neural computation 10.2 (1998), pp. 251-276.
  14. James Martens and Roger Grosse. "Optimizing Neural Networks with Kronecker-factored Approximate Curvature". In: arXiv preprint arXiv:1503.05671 (2015). · arXiv
  15. Rahul G Krishnan, Uri Shalit, and David Sontag. "Deep Kalman Filters". In: arXiv preprint arXiv:1511.05121 (2015). · arXiv
  16. Evan Archer, Il Memming Park, Lars Buesing, John Cunningham, and Liam Paninski. "Black box variational inference for state space models". In: arXiv preprint arXiv:1511.07367 (2015). · arXiv
  17. Karol Gregor, Ivo Danihelka, Alex Graves, and Daan Wierstra. "DRAW: A recurrent neural network for image generation". In: arXiv preprint arXiv:1502.04623 (2015). · arXiv
  18. Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Bengio. "A recurrent latent variable model for sequential data". In: Advances in Neural information processing systems. 2015, pp. 2962-2970.
  19. Mohammad E Khan, Pierre Baqué, François Fleuret, and Pascal Fua. "Kullback-Leibler proximal variational inference". In: Advances in Neural Information Processing Systems. 2015, pp. 3402-3410.
  20. Mohammad Emtiyaz Khan, Reza Babanezhad, Wu Lin, Mark Schmidt, and Masashi Sugiyama. "Faster Stochastic Variational Inference using Proximal-Gradient Methods with General Diver- gence Functions". In: Conference on Uncertainty in Artificial Intelligence (UAI). 2016.
  21. David A Knowles and Tom Minka. "Non-conjugate variational message passing for multino- mial and binary regression". In: Advances in Neural Information Processing Systems. 2011, pp. 1701-1709.
  22. John M. Danskin. The theory of max-min and its application to weapons allocation problems. Springer-Verlag, New York, 1967.
  23. Anthony-V Fiacco. Introduction to sensitivity and stability analysis in nonlinear programming. Academic Press, Inc., 1984.
  24. J Frederic Bonnans and Alexander Shapiro. Perturbation Analysis of Optimization Problems. Springer Science & Business Media, 2000.
  25. James Martens and Roger Grosse. "Optimizing Neural Networks with Kronecker-factored Approximate Curvature". In: Proceedings of the 32nd International Conference on Machine Learning. 2015.
  26. David Duvenaud and Ryan P. Adams. "Black-box stochastic variational inference in five lines of Python". In: NIPS Workshop on Black-box Learning and Inference (2015).

Cited by

Sorted by citations

Metadata

Authors
Johnson, M. J., Duvenaud, D., Wiltschko, A. B., Datta, S. R., & Adams, R. P.
Venue
arXiv
Publisher
arXiv
Published
2016
DOI
10.48550/arxiv.1603.06277
arXiv
1603.06277
References
2
Type
Preprint
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review