The "Wake-Sleep" Algorithm for Unsupervised Neural Networks
The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
An unsupervised learning algorithm for a multilayer network of stochastic neurons is described. Bottom-up "recognition" connections convert the input into representations in successive hidden layers, and top-down "generative" connections reconstruct the representation in one layer from the representation in the layer above. In the "wake" phase, neurons are driven by recognition connections, and generative connections are adapted to increase the probability that they would reconstruct the correct activity vector in the layer below. In the "sleep" phase, neurons are driven by generative connections, and recognition connections are adapted to increase the probability that they would produce the correct activity vector in the layer above.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A first meta-structure pops up when considering the classical Wake-Sleep algorithm [52], described above as a model for NREM sleep (that we extended by random perturbations in the replay).”
From How Adversarial REM Dreams May Facilitate Creativity, and Why We Become Aware of Them (2024) · cites this paper as [52]
In the citing paper
“In this algorithm, the FF recognition pathway is trained during the Sleep phase so that it inverts the FB generative pathway. The generative FB pathway is itself trained during the Wake phase so that it learns to reproduce the sensory activity from the internal representation generated by the FF pathway.”
From How Adversarial REM Dreams May Facilitate Creativity, and Why We Become Aware of Them (2024) · cites this paper as [52]
In the citing paper
“Bayesian inference and related theories have been proposed as a more appropriate theoretical framework for reasoning about top-down visual processing in the brain. 1,2,4,6,26,27”
From Hierarchical Bayesian inference in the visual cortex (2003) · cites this paper as [2]
In the citing paper
“In one common form of unsupervised learning, higher brain areas attempt to produce samples that are statistically similar to those actually seen in lower layers.”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Hinton et al., 1995
In the citing paper
“For example, the wake-sleep algorithm (Hinton et al., 1995) requires the sleep mode to sample potential data points whose distribution should then match the observed distribution.”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Hinton et al., 1995
Citations
Cites
Ballard, D. H., & Brown, C. M. (1982) · Cited by 5,303
A massively parallel architecture for a self-organizing neural pattern recognition machine
Carpenter, G. A., & Grossberg, S. (1987) · Computer Vision, Graphics, and Image Processing · doi:10.1016/s0734-189x(87)80014-2 · Cited by 2,525
Barlow, H. B. (1989) · Neural Computation · doi:10.1162/neco.1989.1.3.295 · Cited by 1,650
Stochastic Complexity in Statistical Inquiry
Rissanen, J. (1998) · WORLD SCIENTIFIC eBooks · doi:10.1142/0822 · Cited by 1,539
Forward models: supervised learning with a distal teacher
Jordan, M. I., & Rumelhart, D. E. (1992) · Cognitive Science · doi:10.1207/s15516709cog1603_1 · Cited by 1,451
Dayan, P., Hinton, G. E., Neal, R. M., & Zemel, R. S. (1995) · Neural Computation · doi:10.1162/neco.1995.7.5.889 · Cited by 1,207
Self-citationHasselmo, M. E., & Bower, J. M. (1993) · Trends in Neurosciences · doi:10.1016/0166-2236(93)90159-j · Cited by 427
A forward-inverse optics model of reciprocal connections between visual cortical areas
Kawato, M., Hayakawa, H., & Inui, T. (1993) · Network: Computation in Neural Systems · doi:10.1088/0954-898x_4_4_001 · Cited by 151
Grenander, U. (1978) · Springer eBooks · Cited by 89
Research works printed but not linked: 3
- Hinton, G. E., Advances in Neural Information Processing Systems 6: 3 (1994).
- Mumford, D., Large-Scale Theories of the Cortex: 125 (1994).
- Ullman, S., Large-Scale Theories of the Cortex: 257 (1994).
Cited by
Items 101–200 of 1,002
Information theory, novelty and hippocampal responses: unpredicted or unpredictable?
Strange, B. A., Duggins, A., Penny, W., Dolan, R. J., & Friston, K. J. (2005) · Neural Networks · doi:10.1016/j.neunet.2004.12.004 · Cited by 264
Training of physical neural networks
Momeni, A., Rahmani, B., Scellier, B., Wright, L. G., McMahon, P. L., Wanjura, C. C., Li, Y., Skalli, A., Berloff, N. G., Onodera, T., et al. (2025) · Nature · doi:10.1038/s41586-025-09384-2 · Cited by 264
Attention-Gated Reinforcement Learning of Internal Representations for Classification
Roelfsema, P. R., & Ooyen, A. V. (2005) · Neural Computation · doi:10.1162/0899766054615699 · Cited by 260
Kim, K. G. (2016) · Healthcare Informatics Research · doi:10.4258/hir.2016.22.4.351 · Cited by 257
Gen: a general-purpose probabilistic programming system with programmable inference
Cusumano-Towner, M., Saad, F. A., Lew, A. K., & Mansinghka, V. K. (2019) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/3314221.3314642 · Cited by 245
Unsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05192 · Cited by 242
Interactions Between Membrane Conductances Underlying Thalamocortical Slow-Wave Oscillations
DESTEXHE, A., & SEJNOWSKI, T. J. (2003) · Physiological Reviews · doi:10.1152/physrev.00012.2003 · Cited by 241
Evaluating the Energy Efficiency of Deep Convolutional Neural Networks on CPUs and GPUs
Li, D., Chen, X., Becchi, M., & Zong, Z. (2016) · 2016 IEEE International Conferences on Big Data and Cloud Computing (BDCloud), Social Computing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom) (BDCloud-SocialCom-SustainCom) · doi:10.1109/bdcloud-socialcom-sustaincom.2016.76 · Cited by 236
Learning to See Physics via Visual De-animation
Wu, J., Lu, E., Kohli, P., Freeman, B., & Tenenbaum, J. B. (2017) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 233
Attend, Infer, Repeat: Fast Scene Understanding with Generative Models
Eslami, S. M. A., Heess, N., Weber, T., Tassa, Y., Szepesvári, D., Kavukcuoglu, K., & Hinton, G. E. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.08575 · Cited by 232
Self-citationOvernight alchemy: sleep-dependent memory evolution
Walker, M. P., & Stickgold, R. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2762-c1 · Cited by 229
Active inference on discrete state-spaces: A synthesis
Da Costa, L., Parr, T., Sajid, N., Veselic, S., Neacsu, V., & Friston, K. (2020) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2020.102447 · Cited by 226
“…cortex, which leads to synaptic plasticity dynamics in the prefrontal cortex, where the synaptic weights encode beliefs about A . is reminiscent of the synaptic mechanisms of sleep (e.g., REM sleep [176, 177]), reflection and associated machine learning algorithms (e.g., wake-sleep [178]).”
BackgroundInternal models in biological control
McNamee, D., & Wolpert, D. M. (2019) · Annual Review of Control, Robotics, and Autonomous Systems · doi:10.1146/annurev-control-060117-105206 · Cited by 225
“…brain encodes a prior p(z) reflecting its beliefs regarding the state z before any sensory information has been received, as well as a probabilistic internal model describing the dependency of sensory signals y on the latent state z known as a generative model in computational neuroscience [12].”
BackgroundHierarchical Neural Networks for Image Interpretation
Behnke, S. (2003) · Lecture Notes in Computer Science · doi:10.1007/b11963 · Cited by 225
How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation
Bengio, Y. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1407.7906 · Cited by 103
Heeger, D. J. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1619788114 · Cited by 220
Prediction and memory: A predictive coding account
Barron, H. C., Auksztulewicz, R., & Friston, K. (2020) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2020.101821 · Cited by 220
A Survey on Deep Learning for Neuroimaging-Based Brain Disorder Analysis
Zhang, L., Wang, M., Liu, M., & Zhang, D. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00779 · Cited by 220
Generative models for discovering sparse distributed representations
Hinton, G. E., & Ghahramani, Z. (1997) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.1997.0101 · Cited by 220
Self-citationThe sensitization and differentiation of dimensions during category learning.
Goldstone, R. L., & Steyvers, M. (2001) · Journal of Experimental Psychology: General · doi:10.1037//0096-3445.130.1.116 · Cited by 219
Priesemann, V., Valderrama, M., Wibral, M., & Le Van Quyen, M. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002985 · Cited by 218
Sleep and synaptic down-selection
Tononi, G., & Cirelli, C. (2020) · European Journal of Neuroscience · doi:10.1111/ejn.14335 · Cited by 217
Local distance preservation in the GP-LVM through back constraints
Lawrence, N. D., & Quiñonero-Candela, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143909 · Cited by 216
A Theory of Generative ConvNet
Xie, J., Lu, Y., Zhu, S. C., & Wu, Y. N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.03264 · Cited by 146
Neural Variational Inference for Text Processing
Miao, Y., Yu, L., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06038 · Cited by 210
StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners
Tian, Y., Fan, L., Isola, P., Chang, H., & Krishnan, D. (2023) · neural information processing systems · doi:10.52202/075280-2098 · Cited by 210
Comprehensive Exploration of Synthetic Data Generation: A SurveyPreprint
Bauer, A., Trapp, S., Stenger, M., Leppich, R., Kounev, S., Leznik, M., Chard, K., & Foster, I. T. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.02524 · Cited by 206
Transformation-invariant clustering using the EM algorithm
Frey, B. J., & Jojic, N. (2003) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2003.1159942 · Cited by 205
Self-citationLearning on Large-scale Text-attributed Graphs via Variational InferencePreprint
Zhao, J., Qu, M., Li, C., Yan, H., Liu, Q., Li, R., Xie, X., & Tang, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.14709 · Cited by 201
Picture: A probabilistic programming language for scene perception
Kulkarni, T. D., Kohli, P., Tenenbaum, J. B., & Mansinghka, V. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299068 · Cited by 133
Virtual reality and consciousness inference in dreaming
Hobson, J. A., Hong, C. C.-H., & Friston, K. J. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.01133 · Cited by 197
“Indeed, statistical schemes that operate along these lines can be found in powerful machine learning algorithms such as the wake-sleep algorithm (Hinton et al., 1995).”
BackgroundA review of some techniques for inclusion of domain-knowledge into deep neural networks
Dash, T., Chitlangia, S., Ahuja, A., & Srinivasan, A. (2022) · Scientific Reports · doi:10.1038/s41598-021-04590-0 · Cited by 196
“Higher-level domain-concepts are then abduced as compositions of these primitives via a neurally-guided search procedure based on a version of the Bayesian “wake-sleep” algorithm [ 33 ] .”
Deep Learning of Representations: Looking Forward
Bengio, Y. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1305.0445 · Cited by 99
A Brief Introduction to Machine Learning for Engineers
Simeone, O. (2018) · Foundations and Trends® in Signal Processing · doi:10.1561/2000000102 · Cited by 154
Getting ahead: forward models and their place in cognitive architecture
Pickering, M. J., & Clark, A. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.05.006 · Cited by 190
Inference in the age of big data: Future perspectives on neuroscience
Bzdok, D., & Yeo, B. T. T. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.04.061 · Cited by 189
Sentience and the Origins of Consciousness: From Cartesian Duality to Markovian Monism
Friston, K. J., Wiese, W., & Hobson, J. A. (2020) · Entropy · doi:10.3390/e22050516 · Cited by 186
“This complexity minimisation itself has formal links with both machine learning [128] and universal computation [88,134,135]—and physiology in the form of synaptic homoeostasis [133,136].”
BackgroundHan, K., Wen, H., Shi, J., Lu, K.-H., Zhang, Y., Fu, D., & Liu, Z. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.05.039 · Cited by 186
Dendritic solutions to the credit assignment problem
Richards, B. A., & Lillicrap, T. P. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2018.08.003 · Cited by 150
Predictive Processing and the Representation Wars
Williams, D. (2018) · Minds and Machines · doi:10.1007/s11023-017-9441-6 · Cited by 134
Deep Learning in Intrusion Detection Systems
Karataş, G., Demir, Ö., & Şahingöz, Ö. K. (2018) · International Congress on Big Data, Deep Learning and Fighting Cyber Terrorism (IBIGDELFT) · doi:10.1109/ibigdelft.2018.8625278 · Cited by 181
Neural Variational Inference and Learning in Belief Networks
Mnih, A., & Gregor, K. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.0030 · Cited by 176
Dynamical Variational Autoencoders: A Comprehensive Review
Girin, L., Leglaive, S., Bie, X., Diard, J., Hueber, T., & Alameda-Pineda, X. (2022) · Foundations and Trends in Machine Learning · doi:10.1561/2200000089 · Cited by 176
Rieser, C. J., & Bostian, C. W. (2004) · VTechWorks (Virginia Tech) · Cited by 174
A comparison of algorithms for inference and learning in probabilistic graphical models
Frey, B. J., & Jojic, N. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.169 · Cited by 173
Self-citationInvariant Visual Object and Face Recognition: Neural and Computational Bases, and a Model, VisNet
Rolls, E. T. (2012) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2012.00035 · Cited by 172
Classical Statistics and Statistical Learning in Imaging Neuroscience
Bzdok, D. (2017) · Frontiers in Neuroscience · doi:10.3389/fnins.2017.00543 · Cited by 170
Quantum variational autoencoder
Khoshaman, A., Vinci, W., Denis, B., Andriyash, E., Sadeghi, H., & Amin, M. H. (2018) · Quantum Science and Technology · doi:10.1088/2058-9565/aada1f · Cited by 167
A Tour of Unsupervised Deep Learning for Medical Image Analysis
Raza, K., & Singh, N. K. (2021) · Current Medical Imaging Formerly Current Medical Imaging Reviews · doi:10.2174/1573405617666210127154257 · Cited by 166
Derin Öğrenme Yöntemleri Ve Uygulamaları Hakkında Bir İnceleme
Şeker, A., Diri, B., & Balık, H. H. (2017) · Gazi Journal of Engineering Sciences · Cited by 165
An optimal estimation approach to visual perception and learning
Rao, R. P. N. (1999) · Vision Research · doi:10.1016/s0042-6989(98)00279-x · Cited by 164
Ethics and responsible AI deployment
Radanliev, P., Santos, O., Brandon-Jones, A., & Joinson, A. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1377011 · Cited by 163
The secret life of predictive brains: what’s spontaneous activity for?
Pezzulo, G., Zorzi, M., & Corbetta, M. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2021.05.007 · Cited by 163
Sleep: off-line memory reprocessing
Stickgold, R. (1998) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(98)01258-3 · Cited by 162
Deep generative models: Survey
Oussidi, A., & Elhassouny, A. (2018) · 2018 International Conference on Intelligent Systems and Computer Vision (ISCV) · doi:10.1109/isacv.2018.8354080 · Cited by 161
Web Phishing Detection Using a Deep Learning Framework
Yi, P., Guan, Y., Zou, F., Yao, Y., Wang, W., & Zhu, T. (2018) · Wireless Communications and Mobile Computing · doi:10.1155/2018/4678746 · Cited by 161
A Hierarchical Model of Binocular Rivalry
Dayan, P. (1998) · Neural Computation · doi:10.1162/089976698300017377 · Cited by 143
Self-citationVaziri, S., Diedrichsen, J., & Shadmehr, R. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.4747-05.2006 · Cited by 159
Dynamic representations and generative models of brain function
Friston, K. J., & Price, C. J. (2001) · Brain Research Bulletin · doi:10.1016/s0361-9230(00)00436-6 · Cited by 159
A multi-layer sparse coding network learns contour coding from natural images
Hoyer, P. O., & Hyvärinen, A. (2002) · Vision Research · doi:10.1016/s0042-6989(02)00017-2 · Cited by 158
A Bayesian framework for concept learning
Tenenbaum, J. B. (1999) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 158
Clark, A. (2012) · Mind · doi:10.1093/mind/fzs106 · Cited by 133
DeepAM: a heterogeneous deep learning framework for intelligent malware detection
Ye, Y., Chen, L., Hou, S., Hardy, W., & Li, X. (2017) · Knowledge and Information Systems · doi:10.1007/s10115-017-1058-9 · Cited by 155
Neurogenesis deep learning: Extending deep networks to accommodate new classes
Draelos, T. J., Miner, N. E., Lamb, C. C., Cox, J. A., Vineyard, C. M., Carlson, K. D., Severa, W. M., James, C. D., & Aimone, J. B. (2017) · 2017 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7965898 · Cited by 83
DGCC: data-driven granular cognitive computing
Wang, G. (2017) · Granular Computing · doi:10.1007/s41066-017-0048-3 · Cited by 151
Two Distributed-State Models For Generating High-Dimensional Time Series
Taylor, G. W., Hinton, G. E., & Roweis, S. T. (2011) · doi:10.5555/1953048.2021035 · Cited by 115
Self-citationPatterns across multiple memories are identified over time
Richards, B. A., Xia, F., Santoro, A., Husse, J., Woodin, M. A., Josselyn, S. A., & Frankland, P. W. (2014) · Nature Neuroscience · doi:10.1038/nn.3736 · Cited by 150
Replicator Neural Networks for Universal Optimal Source Coding
Hecht-Nielsen, R. (1995) · Science · doi:10.1126/science.269.5232.1860 · Cited by 133
Reweighted Wake-SleepPreprint
Bornschein, J., & Bengio, Y. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.2751 · Cited by 150
Machine Learning for Spatiotemporal Sequence Forecasting: A SurveyPreprint
Shi, X., & Yeung, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.06865 · Cited by 150
Above and beyond the concrete: The diverse representational substrates of the predictive brain
Gilead, M., Trope, Y., & Liberman, N. (2020) · Behavioral and Brain Sciences · doi:10.1017/s0140525x19002000 · Cited by 133
Wu, J., Tenenbaum, J. B., & Kohli, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.744 · Cited by 104
Deep Learning the Ising Model Near CriticalityPreprint
Morningstar, A., & Melko, R. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.04622 · Cited by 140
Can connectionism save constructivism?
Marcus, G. F. (1998) · Cognition · doi:10.1016/s0010-0277(98)00018-3 · Cited by 119
Varieties of Helmholtz Machine
Dayan, P., & Hinton, G. E. (1996) · Neural Networks · doi:10.1016/s0893-6080(96)00009-3 · Cited by 136
Self-citationInference Compilation and Universal Probabilistic Programming
Le, T., Baydin, A. G., & Wood, F. (2016) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.48550/arxiv.1610.09900 · Cited by 135
Deep Gaussian Processes and Variational Propagation of Uncertainty
Damianou, A. (2015) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York) · doi:10.13140/rg.2.1.1458.8885 · Cited by 134
Static malware detection and attribution in android byte-code through an end-to-end deep system
Amin, M., Tanveer, T. A., Tehseen, M., Khan, M., Khan, F. A., & Anwar, S. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.07.070 · Cited by 133
Hinton, G. (2014) · Cognitive Science · doi:10.1111/cogs.12049 · Cited by 106
Self-citationLatent variable models in the era of industrial big data: Extension and beyond
Kong, X., Jiang, X., Zhang, B., Yuan, J., & Ge, Z. (2022) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2022.09.005 · Cited by 121
World model learning and inference
Friston, K., Moran, R. J., Nagai, Y., Taniguchi, T., Gomi, H., & Tenenbaum, J. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.09.011 · Cited by 90
Two‐Phase Computational Model Training Long‐Term Memories in the Entorhinal‐Hippocampal Region
LÖRINCZ, A., & BUZSÁKI, G. (2000) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2000.tb06721.x · Cited by 120
Alom, M. Z., Sidike, P., Hasan, M., Taha, T. M., & Asari, V. K. (2018) · Computational Intelligence and Neuroscience · doi:10.1155/2018/6747098 · Cited by 127
Learning Stochastic Feedforward Neural Networks
Tang, Y., & Salakhutdinov, R. (2013) · neural information processing systems · Cited by 126
DreamCoder: growing generalizable, interpretable knowledge with wake–sleep Bayesian program learning
Ellis, K., Wong, L., Nye, M., Sablé-Meyer, M., Cary, L., Anaya Pozo, L., Hewitt, L., Solar-Lezama, A., & Tenenbaum, J. B. (2023) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2022.0050 · Cited by 125
A Neocortical Delta Rhythm Facilitates Reciprocal Interlaminar Interactions via Nested Theta Rhythms
Carracedo, L. M., Kjeldsen, H., Cunnington, L., Jenkins, A., Schofield, I., Cunningham, M. O., Davies, C. H., Traub, R. D., & Whittington, M. A. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.0735-13.2013 · Cited by 113
Switching between internal and external modes: A multiscale learning principle
Honey, C. J., Newman, E. L., & Schapiro, A. C. (2017) · Network Neuroscience · doi:10.1162/netn_a_00024 · Cited by 118
MuProp: Unbiased Backpropagation for Stochastic Neural NetworksPreprint
Gu, S., Levine, S., Sutskever, I., & Mnih, A. (2015) · Apollo (University of Cambridge) · doi:10.48550/arxiv.1511.05176 · Cited by 123
Meta-Learning Probabilistic Inference For Prediction
Gordon, J., Bronskill, J., Bauer, M., Nowozin, S., & Turner, R. E. (2018) · Apollo - University of Cambridge Repository · doi:10.17863/cam.47572 · Cited by 121
Face Image Analysis by Unsupervised Learning
Bartlett, M. S. (2001) · Springer eBooks · doi:10.1007/978-1-4615-1637-8 · Cited by 121
The Generalized Reparameterization GradientPreprint
Ruiz, F. J. R., Titsias, M. K., & Blei, D. M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.02287 · Cited by 121
Ghodrati, M., Khaligh‐Razavi, S., & Lehky, S. R. (2017) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2017.06.002 · Cited by 120
Deep neural network models of sensory systems: windows onto the role of task constraints
Kell, A. J. E., & McDermott, J. H. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.02.003 · Cited by 120
Toward an integration of deep learning and neuroscience
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119
On Unifying Deep Generative ModelsPreprint
Hu, Z., Yang, Z., Salakhutdinov, R., & Xing, E. P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.00550 · Cited by 119
Thalamocortical Assemblies: Sleep spindles, slow waves and epileptic discharges
Destexhe, A., & Sejknowski, T. (2023) · doi:10.1093/oso/9780198864998.001.0001 · Cited by 118
Improving Variational Encoder-Decoders in Dialogue Generation
Shen, X., Su, H., Niu, S., & Demberg, V. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11960 · Cited by 116
A Model of Grid Cell Development through Spatial Exploration and Spike Time-Dependent Plasticity
Widloski, J., & Fiete, I. (2014) · Neuron · doi:10.1016/j.neuron.2014.06.018 · Cited by 115
Dynamical Variational Autoencoders: A Comprehensive Review
Girin, L., Leglaive, S., Bie, X., Diard, J., Hueber, T., & Alameda-Pineda, X. (2021) · doi:10.1561/9781680839135 · Cited by 115
Benedetti, M., Realpe-Gómez, J., & Perdomo-Ortiz, A. (2018) · Quantum Science and Technology · doi:10.1088/2058-9565/aabd98 · Cited by 94
Metadata
- Authors
- Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. M.
- Affiliations
- University of Toronto (CA)
- ORCID
- Dayan, Peter
- Venue
- Science
- Published
- 1995-05-26
- Volume
- 268
- Issue
- 5214
- Pages
- 1158-1161
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W1993845689
- Cited by
- 1,070
- References
- 9
- Citations by year
- 2026: 18 · 2025: 33 · 2024: 36 · 2023: 57 · 2022: 57 · 2021: 68 · 2020: 63 · 2019: 79 · 2018: 79 · 2017: 64 · 2016: 52 · 2015: 42 · 2014: 37 · 2013: 19 · 2012: 18
- FWCI
- 28.04
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- unsupervised learning, stochastic neurons, multilayer neural networks, top-down connections, representation learning, generative models, neural network adaptation
- MeSH terms
- Algorithms, Probability, Stochastic Processes, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.