Two Distributed-State Models For Generating High-Dimensional Time Series
The DOI matches its OpenAlex record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
In this paper we develop a class of nonlinear generative models for high-dimensional time series. We first propose a model based on the restricted Boltzmann machine (RBM) that uses an undirected model with binary latent variables and real-valued “visible” variables. The latent and visible variables at each time step receive directed connections from the visible variables at the last few time-steps. This “conditional” RBM (CRBM) makes on-line inference efficient and allows us to use a simple approximate learning procedure. We demonstrate the power of our approach by synthesizing various sequences from a model trained on motion capture data and by performing on-line filling in of data lost during capture. We extend the CRBM in a way that preserves its most important computational properties and introduces multiplicative three-way interactions that allow the effective interaction weight between two variables to be modulated by the dynamic state of a third variable. We introduce a factoring of the implied three-way weight tensor to permit a more compact parameterization. The resulting model can capture diverse styles of motion with a single set of parameters, and the three-way interactions greatly improve its ability to blend motion styles or to transition smoothly among them. Videos and source code can be found at
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 4×
“Keywords: unsupervised learning, restricted Boltzmann machines, time series, generative models, motion capture”
BackgroundInfluentialSelf-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841 · cited 2×
“Keywords: unsupervised learning, restricted Boltzmann machines, time series, generative models, motion capture”
BackgroundMethodSparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642
A learning algorithm for Boltzmann machines
Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985) · Cognitive Science · doi:10.1207/s15516709cog0901_7 · Cited by 3,402
Self-citationA Practical Guide to Training Restricted Boltzmann Machines
Hinton, G. E. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_32 · Cited by 2,852 · cited 3×
“Models were trained using CD-1 (see Section 3.1) for a fixed number of epochs (complete passes through the data).”
BackgroundInfluentialSelf-citationA View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants
Neal, R. M., & Hinton, G. E. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_12 · Cited by 2,187 · cited 2×
“Keywords: unsupervised learning, restricted Boltzmann machines, time series, generative models, motion capture”
BackgroundMethodSelf-citationRestricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Self-citationEfficient Learning of Sparse Representations with an Energy-Based Model
Ranzato, M., Poultney, C., Chopra, S., & LeCun, Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0147 · Cited by 1,106
The "Wake-Sleep" Algorithm for Unsupervised Neural Networks
Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. M. (1995) · Science · doi:10.1126/science.7761831 · Cited by 1,070
Self-citation
Cited by
A survey of deep neural network architectures and their applications
Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., & Alsaadi, F. E. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.12.038 · Cited by 3,239
“In [153], an undirected model is established with real-valued visible variables and binary latent ones.”
BackgroundA deep learning framework for character motion synthesis and editing
Holden, D., Saito, J., & Komura, T. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925975 · Cited by 655
Deep learning for estimating building energy consumption
Mocanu, E., Nguyen, P. H., Gibescu, M., & Kling, W. L. (2016) · Sustainable Energy, Grids and Networks · doi:10.1016/j.segan.2016.02.005 · Cited by 622
Review on the research and practice of deep learning and reinforcement learning in smart grids
China Electric Power Research Institute, Zhang, D., Han, X., Taiyuan University of Technology, Deng, C., & China Electric Power Research Institute (2018) · CSEE Journal of Power and Energy Systems · doi:10.17775/cseejpes.2018.00520 · Cited by 561
Deep neural networks for energy load forecasting
Amarasinghe, K., Marino, D. L., & Manic, M. (2017) · 2017 IEEE 26th International Symposium on Industrial Electronics (ISIE) · doi:10.1109/isie.2017.8001465 · Cited by 336
Metadata
- Authors
- Taylor, G. W., Hinton, G. E., & Roweis, S. T.
- Published
- 2011-02-01
- Volume
- 12
- Issue
- 28
- Pages
- 1025-1068
- DOI
- 10.5555/1953048.2021035
- OpenAlex
- W97016928
- Cited by
- 115
- References
- 15
- FWCI
- 13.28
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Two Distributed-State Models For Generating High-Dimensional Time Series. https://app.cochat.ai/papers/10.5555/1953048.2021035
- The paper
- Taylor, G. W., Hinton, G. E., & Roweis, S. T. (2011). Two Distributed-State Models For Generating High-Dimensional Time Series. https://doi.org/10.5555/1953048.2021035
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.