The "Wake-Sleep" Algorithm for Unsupervised Neural Networks
The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
An unsupervised learning algorithm for a multilayer network of stochastic neurons is described. Bottom-up "recognition" connections convert the input into representations in successive hidden layers, and top-down "generative" connections reconstruct the representation in one layer from the representation in the layer above. In the "wake" phase, neurons are driven by recognition connections, and generative connections are adapted to increase the probability that they would reconstruct the correct activity vector in the layer below. In the "sleep" phase, neurons are driven by generative connections, and recognition connections are adapted to increase the probability that they would produce the correct activity vector in the layer above.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A first meta-structure pops up when considering the classical Wake-Sleep algorithm [52], described above as a model for NREM sleep (that we extended by random perturbations in the replay).”
From How Adversarial REM Dreams May Facilitate Creativity, and Why We Become Aware of Them (2024) · cites this paper as [52]
In the citing paper
“In this algorithm, the FF recognition pathway is trained during the Sleep phase so that it inverts the FB generative pathway. The generative FB pathway is itself trained during the Wake phase so that it learns to reproduce the sensory activity from the internal representation generated by the FF pathway.”
From How Adversarial REM Dreams May Facilitate Creativity, and Why We Become Aware of Them (2024) · cites this paper as [52]
In the citing paper
“Bayesian inference and related theories have been proposed as a more appropriate theoretical framework for reasoning about top-down visual processing in the brain. 1,2,4,6,26,27”
From Hierarchical Bayesian inference in the visual cortex (2003) · cites this paper as [2]
In the citing paper
“In one common form of unsupervised learning, higher brain areas attempt to produce samples that are statistically similar to those actually seen in lower layers.”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Hinton et al., 1995
In the citing paper
“For example, the wake-sleep algorithm (Hinton et al., 1995) requires the sleep mode to sample potential data points whose distribution should then match the observed distribution.”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Hinton et al., 1995
Citations
Cites
Ballard, D. H., & Brown, C. M. (1982) · Cited by 5,303
A massively parallel architecture for a self-organizing neural pattern recognition machine
Carpenter, G. A., & Grossberg, S. (1987) · Computer Vision, Graphics, and Image Processing · doi:10.1016/s0734-189x(87)80014-2 · Cited by 2,525
Barlow, H. B. (1989) · Neural Computation · doi:10.1162/neco.1989.1.3.295 · Cited by 1,650
Stochastic Complexity in Statistical Inquiry
Rissanen, J. (1998) · WORLD SCIENTIFIC eBooks · doi:10.1142/0822 · Cited by 1,539
Forward models: supervised learning with a distal teacher
Jordan, M. I., & Rumelhart, D. E. (1992) · Cognitive Science · doi:10.1207/s15516709cog1603_1 · Cited by 1,451
Dayan, P., Hinton, G. E., Neal, R. M., & Zemel, R. S. (1995) · Neural Computation · doi:10.1162/neco.1995.7.5.889 · Cited by 1,207
Self-citationHasselmo, M. E., & Bower, J. M. (1993) · Trends in Neurosciences · doi:10.1016/0166-2236(93)90159-j · Cited by 427
A forward-inverse optics model of reciprocal connections between visual cortical areas
Kawato, M., Hayakawa, H., & Inui, T. (1993) · Network: Computation in Neural Systems · doi:10.1088/0954-898x_4_4_001 · Cited by 151
Grenander, U. (1978) · Springer eBooks · Cited by 89
Research works printed but not linked: 3
- Hinton, G. E., Advances in Neural Information Processing Systems 6: 3 (1994).
- Mumford, D., Large-Scale Theories of the Cortex: 125 (1994).
- Ullman, S., Large-Scale Theories of the Cortex: 257 (1994).
Cited by
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Self-citationAuto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Self-citation
Metadata
- Authors
- Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. M.
- Affiliations
- University of Toronto (CA)
- ORCID
- Dayan, Peter
- Venue
- Science
- Published
- 1995-05-26
- Volume
- 268
- Issue
- 5214
- Pages
- 1158-1161
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W1993845689
- Cited by
- 1,070
- References
- 9
- Citations by year
- 2026: 18 · 2025: 33 · 2024: 36 · 2023: 57 · 2022: 57 · 2021: 68 · 2020: 63 · 2019: 79 · 2018: 79 · 2017: 64 · 2016: 52 · 2015: 42 · 2014: 37 · 2013: 19 · 2012: 18
- FWCI
- 28.04
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- unsupervised learning, stochastic neurons, multilayer neural networks, top-down connections, representation learning, generative models, neural network adaptation
- MeSH terms
- Algorithms, Probability, Stochastic Processes, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The "Wake-Sleep" Algorithm for Unsupervised Neural Networks. https://app.cochat.ai/papers/10.1126/science.7761831
- The paper
- Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. M. (1995). The "Wake-Sleep" Algorithm for Unsupervised Neural Networks. Science. https://doi.org/10.1126/science.7761831
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.