On the Challenges of Physical Implementations of RBMs
Open accessAbstract
Restricted Boltzmann machines (RBMs) are powerful machine learning models, but learning and some kinds of inference in the model require sampling-based approximations, which, in classical digital computers, are implemented using expensive MCMC. Physical computation offers the opportunity to reduce the costof sampling by building physical systems whose natural dynamics correspond to drawing samples from the desired RBM distribution. Such a system avoids the burn-in and mixing cost of a Markov chain. However, hardware implementations of this variety usually entail limitations such as low-precision and limited range of the parameters and restrictions on the size and topology of the RBM. We conduct software simulations to determine how harmful each of these restrictions is. Our simulations are based on the D-Wave Two computer, but the issues we investigate arise in most forms of physical computation.Our findings suggest that designers of new physical computing hardware and algorithms for physical computers should focus their efforts on overcoming the limitations imposed by the topology restrictions of currently existing physical computers.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Quantum algorithms have been proposed using a D-Wave machine or a circuit algorithm that sample from the Boltzmann distribution quadratically faster than is possible classically.
“Quantum algorithms have been proposed, using a D-Wave machine [101-103] or a circuit algorithm [104]; that sample from the Boltzmann distribution quadratically faster than it is possible classically [23].”
From The prospects of quantum computing in computational molecular biology (2021) · cites this paper as [102]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Self-citationA fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · see citing works
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382 · see citing works
Beitrag zur Theorie des Ferromagnetismus
Ising, E. (1925) · Zeitschrift für Physik · doi:10.1007/bf02980577 · Cited by 3,871 · see citing works
Information processing in dynamical systems: foundations of harmony theory
Smolensky, P. (1986) · The MIT Press eBooks · Cited by 1,673 · see citing works
Deep Learning of Representations for Unsupervised and Transfer Learning.
Bengio, Y. (2011) · Cited by 885 · see citing works
Self-citationTraining restricted Boltzmann machines using approximations to the likelihood gradient
Tieleman, T. (2008) · International Conference on Machine Learning · doi:10.1145/1390156.1390290 · Cited by 847 · see citing works
(2011) · Cited by 514 · see citing works
On the quantitative analysis of deep belief networks
Salakhutdinov, R., & Murray, I. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390266 · Cited by 411 · see citing works
Using fast weights to improve persistent contrastive divergence
Tieleman, T., & Hinton, G. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553506 · Cited by 274 · see citing works
On the convergence of markovian stochastic algorithms with rapidly decreasing ergodicity rates
Younès, L. (1999) · Stochastics and stochastics reports · doi:10.1080/17442509908834179 · Cited by 146 · see citing works
Multi-Prediction Deep Boltzmann Machines
Goodfellow, I., Mirza, M., Courville, A. C., & Bengio, Y. (2013) · neural information processing systems · Cited by 120 · see citing works
Self-citationTempered Markov Chain Monte Carlo for training of Restricted Boltzmann Machines
Desjardins, G., Courville, A. C., Bengio, Y., et al. (2010) · International Conference on Artificial Intelligence and Statistics · Cited by 105 · see citing works
Self-citationLearning in Markov Random Fields using Tempered Transitions
Salakhutdinov, R. (2009) · Cited by 82 · see citing works
Parallel tempering is efficient for learning restricted Boltzmann machines
Cho, K., Raiko, T., & Ilin, A. (2010) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596837 · Cited by 82 · see citing works
Learning Deep Boltzmann Machines using Adaptive MCMC
Salakhutdinov, R. (2010) · Cited by 73 · see citing works
Restricted Boltzmann Machines are Hard to Approximately Evaluate or Simulate
Long, P. M., & Servedio, R. A. (2010) · Cited by 54 · see citing works
Toward the Implementation of a Quantum RBM
de Freitas, N. (2011) · Cited by 34 · see citing works
Tractable Multivariate Binary Density Estimation and the Restricted Boltzmann Forest
Larochelle, H., Bengio, Y., & Turian, J. P. (2010) · Neural Computation · doi:10.1162/neco_a_00014 · Cited by 26 · see citing works
Self-citationAn optoelectronic CMOS circuit implementing a simulated annealing algorithm
Dupret, A., Belhaire, E., Rodier, J.-C., et al. (1996) · IEEE Journal of Solid-State Circuits · doi:10.1109/4.508220 · Cited by 13 · see citing works
Toward the Implementation of a Quantum RBM
Denil, M., & Freitas, N. D. (2014) · Cited by 11 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Biamonte, J., Wittek, P., Pancotti, N., Rebentrost, P., Wiebe, N., & Lloyd, S. (2017) · Nature · doi:10.1038/nature23474 · Cited by 4,779 · see citing works
A Survey on quantum computing technology
Gyongyosi, L., & Imre, S. (2019) · Computer Science Review · doi:10.1016/j.cosrev.2018.11.002 · Cited by 646 · see citing works
The prospects of quantum computing in computational molecular biology
Outeiral, C., Strahm, M., Shi, J., Morris, G. M., Benjamin, S. C., & Deane, C. M. (2021) · WIREs Computational Molecular Science · doi:10.1002/wcms.1481 · Cited by 232 · see citing works
Information Perspective to Probabilistic Modeling: Boltzmann Machines versus Born Machines
Cheng, S., Chen, J., & Wang, L. (2018) · Entropy · doi:10.3390/e20080583 · Cited by 104 · see citing works
Pal, S., Bhattacharya, M., Lee, S.-S., & Chakraborty, C. (2024) · Molecular Biotechnology · doi:10.1007/s12033-023-00765-4 · Cited by 84 · see citing works
- Authors
- Dumoulin, V., Goodfellow, I., Courville, A., & Bengio, Y.
- Affiliations
- Université de Montréal (CA)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2014-06-21
- Volume
- 28
- Issue
- 1
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2153346333
- Cited by
- 46
- References
- 21
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 3 · 2023: 8 · 2022: 5 · 2021: 4 · 2020: 6 · 2019: 3 · 2018: 6 · 2017: 3 · 2016: 4 · 2015: 1
- FWCI
- 1.56
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Quantum Computing Algorithms and Architecture
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Keywords
- restricted Boltzmann machine, physical computation, quantum annealing, sampling-based inference, hardware constraints, low-precision arithmetic, network topology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
