The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Structured Inference Networks for Nonlinear State Space Models
Open accessAbstract
Gaussian state space models have been used for decades as generative models of sequential data. They admit an intuitive probabilistic interpretation, have a simple functional form, and enjoy widespread adoption. We introduce a unified algorithm to efficiently learn a broad class of linear and non-linear state space models, including variants where the emission and transition distributions are modeled by deep neural networks. Our learning algorithm simultaneously learns a compiled inference network and the generative model, leveraging a structured variational approximation parameterized by recurrent neural networks to mimic the posterior distribution. We apply the learning algorithm to both synthetic and real-world datasets, demonstrating its scalability and versatility. We find that using the structured approximation to the posterior results in models with significantly higher held-out likelihood.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
The unscented Kalman filter for nonlinear estimation
Wan, E. A., & Van Der Merwe, R. (2002) · Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373) · doi:10.1109/asspcc.2000.882463 · Cited by 4,001
Stochastic Backpropagation and Approximate Inference in Deep Generative Models
Rezende, D. J., Mohamed, S., & Wierstra, D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1401.4082 · Cited by 2,405
Static Analysis of Shape in TensorFlow Programs
The Theano Development Team, Al‐Rfou, R., Alain, G., Almahairi, A., Angermueller, C., Bahdanau, D., Ballas, N., Bastien, F., Bayer, J., Belikov, A., et al. (2016) · arXiv (Cornell University) · doi:10.4230/lipics.ecoop.2020.15 · Cited by 1,589
DRAW: A Recurrent Neural Network For Image Generation
Gregor, K., Danihelka, I., Graves, A., Rezende, D. J., & Wierstra, D. (2015) · International Conference on Machine Learning · Cited by 1,285
Variational Inference with Normalizing Flows
Rezende, D. J., & Mohamed, S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05770 · Cited by 1,253
The "Wake-Sleep" Algorithm for Unsupervised Neural Networks
Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. M. (1995) · Science · doi:10.1126/science.7761831 · Cited by 1,070
A Recurrent Latent Variable Model for Sequential Data
Chung, J., Kastner, K., Dinh, L., Goel, K., Courville, A. C., & Bengio, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.02216 · Cited by 635
System identification of nonlinear state-space models
Schön, T. B., Wills, A., & Ninness, B. (2011) · Automatica · doi:10.1016/j.automatica.2010.10.013 · Cited by 563
Boulanger-Lewandowski, N., Bengio, Y., & Vincent, P. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6392 · Cited by 459
Learning Nonlinear Dynamical Systems Using an EM Algorithm
Ghahramani, Z., & Roweis, S. T. (1998) · Cited by 232
Sequential Neural Models with Stochastic LayersPreprint
Fraccaro, M., Sønderby, S. K., Paquet, U., & Winther, O. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07571 · Cited by 221
Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images
Watter, M., Springenberg, J. T., Boedecker, J., & Riedmiller, M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.07365 · Cited by 212
Krishnan, R. G., Shalit, U., & Sontag, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05121 · Cited by 206 · cited 2×
“The current paper instantiates the inference scheme presented in (Krishnan et al., 2015) [Thm.”
MethodSelf-citationLearning Stochastic Recurrent Networks
Bayer, J., & Osendorfer, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.7610 · Cited by 181
Variational Recurrent Auto-Encoders
Fabius, O., & van Amersfoort, J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6581 · Cited by 169
Black box variational inference for state space modelsPreprint
Archer, E., Park, I. M., Buesing, L., Cunningham, J. P., & Paninski, L. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.07367 · Cited by 147
An Unsupervised Ensemble Learning Method for Nonlinear Dynamic State-Space Models
Harri Valpola, B., & Karhunen, J. (2002) · Neural Computation · doi:10.1162/089976602760408017 · Cited by 138
Neural Adaptive Sequential Monte CarloPreprint
Gu, S., Ghahramani, Z., & Turner, R. E. (2015) · arXiv (Cornell University) · doi:10.17863/cam.17371 · Cited by 87
Deep Temporal Sigmoid Belief Networks for Sequence ModelingPreprint
Gan, Z., Li, C., Henao, R., Carlson, D., & Carin, L. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.07087 · Cited by 80
Dual Kalman Filtering Methods for Nonlinear Prediction, Smoothing, and Estimation
Wan, E. A., & Nelson, A. (1997) · Cited by 67
Variational Bayesian learning of nonlinear hidden state-space models for model predictive control
Raiko, T., & Tornio, M. (2009) · Neurocomputing · doi:10.1016/j.neucom.2009.06.009 · Cited by 35
Structured VAEs: Composing Probabilistic Graphical Models and Variational AutoencodersPreprint
Johnson, M., Duvenaud, D., Wiltschko, A. B., Datta, S. R., & Adams, R. P. (2016) · arXiv (Cornell University) · Cited by 14
Cited by
An Introduction to Variational Autoencoders
Kingma, D. P., & Welling, M. (2019) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000056 · Cited by 2,609
“, 2016a), learning nonlinear state space models (Krishnan et al., 2017), voice”
BackgroundKalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics
Revach, G., Shlezinger, N., Ni, X., Escoriza, A. L., van Sloun, R. J. G., & Eldar, Y. C. (2022) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2022.3158588 · Cited by 599
Pyro: Deep Universal Probabilistic Programming
Bingham, E., Chen, J. P., Jankowiak, M., Obermeyer, F. H., Pradhan, N., Karaletsos, T., Singh, R., Szerlip, P., Horsfall, P., & Goodman, N. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.09538 · Cited by 536
Deep learning and its application to LHC physics
Guest, D., Cranmer, K., & Whiteson, D. (2018) · Annual Review of Nuclear and Particle Science · doi:10.1146/annurev-nucl-101917-021019 · Cited by 417
Linearly Recurrent Autoencoder Networks for Learning Dynamics
Otto, S. E., & Rowley, C. W. (2019) · SIAM Journal on Applied Dynamical Systems · doi:10.1137/18m1177846 · Cited by 345
Metadata
- Authors
- Krishnan, R., Shalit, U., & Sontag, D.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2017-02-13
- Volume
- 31
- Issue
- 1
- ISSN
- 2374-3468
- 2159-5399
- OpenAlex
- W2964232608
- Cited by
- 346
- References
- 25
- FWCI
- 18.53
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.