The "Wake-Sleep" Algorithm for Unsupervised Neural Networks
The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
An unsupervised learning algorithm for a multilayer network of stochastic neurons is described. Bottom-up "recognition" connections convert the input into representations in successive hidden layers, and top-down "generative" connections reconstruct the representation in one layer from the representation in the layer above. In the "wake" phase, neurons are driven by recognition connections, and generative connections are adapted to increase the probability that they would reconstruct the correct activity vector in the layer below. In the "sleep" phase, neurons are driven by generative connections, and recognition connections are adapted to increase the probability that they would produce the correct activity vector in the layer above.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A first meta-structure pops up when considering the classical Wake-Sleep algorithm [52], described above as a model for NREM sleep (that we extended by random perturbations in the replay).”
From How Adversarial REM Dreams May Facilitate Creativity, and Why We Become Aware of Them (2024) · cites this paper as [52]
In the citing paper
“In this algorithm, the FF recognition pathway is trained during the Sleep phase so that it inverts the FB generative pathway. The generative FB pathway is itself trained during the Wake phase so that it learns to reproduce the sensory activity from the internal representation generated by the FF pathway.”
From How Adversarial REM Dreams May Facilitate Creativity, and Why We Become Aware of Them (2024) · cites this paper as [52]
In the citing paper
“Bayesian inference and related theories have been proposed as a more appropriate theoretical framework for reasoning about top-down visual processing in the brain. 1,2,4,6,26,27”
From Hierarchical Bayesian inference in the visual cortex (2003) · cites this paper as [2]
In the citing paper
“In one common form of unsupervised learning, higher brain areas attempt to produce samples that are statistically similar to those actually seen in lower layers.”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Hinton et al., 1995
In the citing paper
“For example, the wake-sleep algorithm (Hinton et al., 1995) requires the sleep mode to sample potential data points whose distribution should then match the observed distribution.”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Hinton et al., 1995
Citations
Cites
Ballard, D. H., & Brown, C. M. (1982) · Cited by 5,303
A massively parallel architecture for a self-organizing neural pattern recognition machine
Carpenter, G. A., & Grossberg, S. (1987) · Computer Vision, Graphics, and Image Processing · doi:10.1016/s0734-189x(87)80014-2 · Cited by 2,525
Barlow, H. B. (1989) · Neural Computation · doi:10.1162/neco.1989.1.3.295 · Cited by 1,650
Stochastic Complexity in Statistical Inquiry
Rissanen, J. (1998) · WORLD SCIENTIFIC eBooks · doi:10.1142/0822 · Cited by 1,539
Forward models: supervised learning with a distal teacher
Jordan, M. I., & Rumelhart, D. E. (1992) · Cognitive Science · doi:10.1207/s15516709cog1603_1 · Cited by 1,451
Dayan, P., Hinton, G. E., Neal, R. M., & Zemel, R. S. (1995) · Neural Computation · doi:10.1162/neco.1995.7.5.889 · Cited by 1,207
Self-citationHasselmo, M. E., & Bower, J. M. (1993) · Trends in Neurosciences · doi:10.1016/0166-2236(93)90159-j · Cited by 427
A forward-inverse optics model of reciprocal connections between visual cortical areas
Kawato, M., Hayakawa, H., & Inui, T. (1993) · Network: Computation in Neural Systems · doi:10.1088/0954-898x_4_4_001 · Cited by 151
Grenander, U. (1978) · Springer eBooks · Cited by 89
Research works printed but not linked: 3
- Hinton, G. E., Advances in Neural Information Processing Systems 6: 3 (1994).
- Mumford, D., Large-Scale Theories of the Cortex: 125 (1994).
- Ullman, S., Large-Scale Theories of the Cortex: 257 (1994).
Cited by
Items 201–300 of 1,002
Fixation behavior in observation and imitation of human movement
Matarić, M. J., & Pomplun, M. (1998) · Cognitive Brain Research · doi:10.1016/s0926-6410(98)00025-1 · Cited by 115
Learning Data Manipulation for Augmentation and WeightingPreprint
Hu, Z., Tan, B., Salakhutdinov, R., Mitchell, T. M., & Xing, E. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12795 · Cited by 115
Efficient inverse graphics in biological face processing
Yildirim, I., Belledonne, M., Freiwald, W., & Tenenbaum, J. (2020) · Science Advances · doi:10.1126/sciadv.aax5979 · Cited by 88
Learning Joint Top-Down and Bottom-up Processes for 3D Visual Inference
Sminchisescu, C., Kanaujia, A., & Metaxas, D. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.169 · Cited by 97
Recognizing Handwritten Digits Using Mixtures of Linear Models
Hinton, G. E., Revow, M. D., & Dayan, P. (1994) · UCL Discovery (University College London) · Cited by 113
Self-citationInference Networks for Sequential Monte Carlo in Graphical Models
Paige, B., & Wood, F. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.06701 · Cited by 46
Discovering multiscale dynamical features with hierarchical Echo State Networks
Jaeger, H. (2007) · Cited by 112
Deep learning: from speech recognition to language and multimodal processing
Deng, L. (2016) · APSIPA Transactions on Signal and Information Processing · doi:10.1017/atsip.2015.22 · Cited by 110
Handwritten Bangla Digit Recognition Using Deep LearningPreprint
Alom, M. Z., Sidike, P., Taha, T. M., & Asari, V. K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.02680 · Cited by 110
Active inference as a theory of sentient behavior
Pezzulo, G., Parr, T., & Friston, K. (2024) · Biological Psychology · doi:10.1016/j.biopsycho.2023.108741 · Cited by 109
DroidDelver: An Android Malware Detection System Using Deep Belief Network Based on API Call Blocks
Hou, S., Saas, A., Ye, Y., & Chen, L. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-47121-1_5 · Cited by 109
Doubly Reparameterized Gradient Estimators for Monte Carlo ObjectivesPreprint
Tucker, G., Lawson, D., Gu, S., & Maddison, C. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.04152 · Cited by 109
Learning Deep Sigmoid Belief Networks with Data Augmentation
Gan, Z., Henao, R., Carlson, D., & Carin, L. (2015) · International Conference on Artificial Intelligence and Statistics · Cited by 108
Cortical Region Interactions and the Functional Role of Apical Dendrites
Spratling, M. W. (2002) · Behavioral and Cognitive Neuroscience Reviews · doi:10.1177/1534582302001003003 · Cited by 90
Handwritten Hangul recognition using deep convolutional neural networks
Kim, I., & Xie, X. S. (2014) · International Journal on Document Analysis and Recognition (IJDAR) · doi:10.1007/s10032-014-0229-4 · Cited by 107
Sleep, off-line processing, and vocal learning
Margoliash, D., & Schmidt, M. F. (2010) · Brain and Language · doi:10.1016/j.bandl.2009.09.005 · Cited by 105
Learning to represent visual input
Hinton, G. E. (2010) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2009.0200 · Cited by 91
Self-citationNeurons learn by predicting future activity
Luczak, A., McNaughton, B. L., & Kubo, Y. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-021-00430-y · Cited by 105
Relevance Realization and the Emerging Framework in Cognitive Science
Vervaeke, J., Lillicrap, T., & Richards, B. A. (2009) · Journal of Logic and Computation · doi:10.1093/logcom/exp067 · Cited by 104
Image Denoising using Deep Learning: Convolutional Neural Network
Ghose, S., Singh, N., & Singh, P. (2020) · International Conference on Cloud Computing, Data Science & Engineering (Confluence) · doi:10.1109/confluence47617.2020.9057895 · Cited by 104
DreamCoder: bootstrapping inductive program synthesis with wake-sleep library learning
Ellis, K., Wong, C., Nye, M., Sablé-Meyer, M., Morales, L., Hewitt, L., Cary, L., Solar-Lezama, A., & Tenenbaum, J. B. (2021) · Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation · doi:10.1145/3453483.3454080 · Cited by 103
“DreamCoder is inspired by the original wake-sleep algorithm of Hinton, Dayan and colleagues [23] (hence the name DreamCoder).”
BackgroundMethodCognitive Biology: Dealing with Information from Bacteria to Minds
Auletta, G. (2011) · doi:10.1093/acprof:oso/9780199608485.001.0001 · Cited by 103
Face image analysis by unsupervised learning and redundancy reduction
Bartlett, M. S., & MacLeod, D. I. A. (1998) · Cited by 103
A Model for Encoding Multiple Object Motions and Self-Motion in Area MST of Primate Visual Cortex
Zemel, R. S., & B, T. J. S. A. (1998) · Journal of Neuroscience · doi:10.1523/jneurosci.18-01-00531.1998 · Cited by 103
The neural coding framework for learning generative models
Ororbia, A., & Kifer, D. (2022) · Nature Communications · doi:10.1038/s41467-022-29632-7 · Cited by 57
Illusory Motion Reproduced by Deep Neural Networks Trained for Prediction
Watanabe, E., Kitaoka, A., Sakamoto, K., Yasugi, M., & Tanaka, K. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00345 · Cited by 101
Stochastic Spectral Descent for Restricted Boltzmann Machines
Carlson, D., Cevher, V., & Carin, L. (2015) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 99
On the principles of Parsimony and Self-consistency for the emergence of intelligence
Ma, Y., Tsao, D. Y., & Shum, H. (2022) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2200297 · Cited by 99
Learning How to Propagate Messages in Graph Neural Networks
Xiao, T., Chen, Z., Wang, D., & Wang, S. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467451 · Cited by 98
Variational Learning in Nonlinear Gaussian Belief Networks
Frey, B. J., & Hinton, G. E. (1999) · Neural Computation · doi:10.1162/089976699300016872 · Cited by 75
Self-citationAn Introduction to Neural Networks and Deep Learning
Suk, H. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-810408-8.00002-x · Cited by 98
Tea With Milk? A Hierarchical Generative Framework of Sequential Event Comprehension
Kuperberg, G. R. (2020) · Topics in Cognitive Science · doi:10.1111/tops.12518 · Cited by 96
A Non-Parametric Multi-Scale Statistical Model for Natural Images
Bonet, J. S. D., & Viola, P. A. (1997) · neural information processing systems · Cited by 96
Are computational models of any use to psychiatry?
Huys, Q. J. M., Moutoussis, M., & Williams, J. (2011) · Neural Networks · doi:10.1016/j.neunet.2011.03.001 · Cited by 94
Sleep-like unsupervised replay reduces catastrophic forgetting in artificial neural networks
Tadros, T., Krishnan, G. P., Ramyaa, R., & Bazhenov, M. (2022) · Nature Communications · doi:10.1038/s41467-022-34938-7 · Cited by 94
Goal-directed control and its antipodes
Dayan, P. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.03.004 · Cited by 93
“…(‘dreamt’) by the generative model can be used during times (notably sleeping, grooming and eating for rats, Diba & Buzsáki, 2007; Foster & Wilson, 2006; Lee & Wilson, 2002; Louie &Wilson, 2001; Skaggs &Mcnaughton, 1996) that active control is suppressed (Hinton et al., 1995; Sutton, 1991).”
Self-citationNeural Adaptive Sequential Monte Carlo
Gu, S., Ghahramani, Z., & Turner, R. E. (2015) · Apollo (University of Cambridge) · doi:10.17863/cam.17371 · Cited by 93
Stochastic inference with spiking neurons in the high-conductance state
Petrovici, M. A., Bill, J., Bytschok, I., Schemmel, J., & Meier, K. (2016) · Physical review. E · doi:10.1103/physreve.94.042312 · Cited by 92
Feature Extraction Through LOCOCODE
Hochreiter, S., & Schmidhuber, J. (1999) · Neural Computation · doi:10.1162/089976699300016629 · Cited by 83
A Review of Text Style Transfer Using Deep Learning
Toshevska, M., & Gievska, S. (2021) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3115992 · Cited by 90
Methodological challenges for understanding cognitive development in infants
Aslin, R. Ν., & Fiser, J. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.01.003 · Cited by 90
Approximating Posterior Distributions in Belief Networks Using Mixtures
Bishop, C., Lawrence, N. D., Jaakkola, T., & Jordan, M. I. (1997) · neural information processing systems · Cited by 89
A cognitive process occurring during sleep is revealed by rapid eye movements
Senzai, Y., & Scanziani, M. (2022) · Science · doi:10.1126/science.abp8852 · Cited by 88
Neural Decoding with Hierarchical Generative Models
van Gerven, M. A. J., de Lange, F. P., & Heskes, T. (2010) · Neural Computation · doi:10.1162/neco_a_00047 · Cited by 88
Simple substrates for complex cognition
Dayan, P. (2008) · Frontiers in Neuroscience · doi:10.3389/neuro.01.031.2008 · Cited by 87
Self-citationExpecting the World: Perception, Prediction, and the Origins of Human Knowledge
Clark, A. (2013) · The Journal of Philosophy · doi:10.5840/jphil2013110913 · Cited by 87
An Overview of Deep Generative Models
Xu, J., Li, H., & Zhou, S. (2014) · IETE Technical Review · doi:10.1080/02564602.2014.987328 · Cited by 87
Unsupervised Learning of Overlapping Image Components Using Divisive Input Modulation
Spratling, M. W., De Meyer, K., & Kompass, R. (2009) · Computational Intelligence and Neuroscience · doi:10.1155/2009/381457 · Cited by 78
Zhang, T., Cheng, X., Jia, S., Poo, M., Zeng, Y., & Xu, B. (2021) · Science Advances · doi:10.1126/sciadv.abh0146 · Cited by 85
Deep Amortized Inference for Probabilistic ProgramsPreprint
Ritchie, D., Horsfall, P., & Goodman, N. D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.05735 · Cited by 85
Cognitive Intelligence: Deep Learning, Thinking, and Reasoning by Brain-Inspired Systems
Wang, Y., Widrow, B. C., Zadeh, L. A., Howard, N., Wood, S. L., Bhavsar, V. C., Budin, G., Chan, C. W., Fiorini, R. A., Gavrilova, M. L., et al. (2016) · International Journal of Cognitive Informatics and Natural Intelligence · doi:10.4018/ijcini.2016100101 · Cited by 84
Local expectation gradients for black box variational inference
Titsias, M. K., & Lázaro-Gredilla, M. (2015) · neural information processing systems · Cited by 84
Deep Generative Models with Learnable Knowledge ConstraintsPreprint
Hu, Z., Yang, Z., Salakhutdinov, R., Liang, X., Qin, L., Dong, H., & Xing, E. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.09764 · Cited by 84
Memory as a Computational Resource
Dasgupta, I., & Gershman, S. J. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.12.008 · Cited by 68
Learning Image Components for Object Recognition
Spratling, M. (2006) · Research Portal (King's College London) · doi:10.5555/1248547.1248575 · Cited by 83
Nie, S., Zheng, M., & Ji, Q. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2763440 · Cited by 83
Divergence Triangle for Joint Training of Generator Model, Energy-Based Model, and Inferential Model
Han, T., Nijkamp, E., Fang, X., Hill, M., Zhu, S. C., & Wu, Y. N. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00887 · Cited by 83
Deep Temporal Sigmoid Belief Networks for Sequence ModelingPreprint
Gan, Z., Li, C., Henao, R., Carlson, D., & Carin, L. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.07087 · Cited by 83
Perception science in the age of deep neural networks.
VanRullen, R. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00142 · Cited by 82
“…do exist for artificial neural networks, but they often give rise to a feature space that is insufficiently powerful and needs to be complemented by supervised fine-tuning in order to allow, for example, for accurate object recognition (Hinton et al., 1995, 2006; Hinton and Salakhutdinov, 2006).”
BackgroundControllable Text Generation.Preprint
Hu, Z., Yang, Z., Liang, X., Salakhutdinov, R., & Xing, E. P. (2017) · arXiv (Cornell University) · Cited by 82
Deco, G., & Schürmann, B. (2000) · Vision Research · doi:10.1016/s0042-6989(00)00140-1 · Cited by 81
Variational Implicit ProcessesPreprint
Ma, C., Li, Y., & Hernández-Lobato, J. M. (2018) · arXiv (Cornell University) · doi:10.17863/cam.42186 · Cited by 81
Generating meaning: active inference and the scope and limits of passive AI
Pezzulo, G., Parr, T., Cisek, P., Clark, A., & Friston, K. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2023.10.002 · Cited by 80
Brain activity is not only for thinking
Laumann, T. O., & Snyder, A. Z. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.04.002 · Cited by 79
Geman, S., Potter, D. F., & Chi, Z. (2002) · Quarterly of Applied Mathematics · doi:10.1090/qam/1939008 · Cited by 79
Deep learning for single-molecule science
Albrecht, T., Slabaugh, G., Alonso, E., & Al-Arif, S. M. R. (2017) · Nanotechnology · doi:10.1088/1361-6528/aa8334 · Cited by 72
BYY harmony learning, independent state space, and generalized APT financial analyses
Xu, L. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.935094 · Cited by 78
Deep Learning in the EEG Diagnosis of Alzheimer’s Disease
Zhao, Y., & He, L. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16628-5_25 · Cited by 76
On the challenges of learning with inference networks on sparse, high-dimensional dataPreprint
Krishnan, R. G., Liang, D., & Hoffman, M. D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.06085 · Cited by 76
Charles Bonnet Syndrome: Evidence for a Generative Model in the Cortex?
Reichert, D. P., Seriès, P., & Storkey, A. J. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003134 · Cited by 61
A path towards quantum advantage in training deep generative models with quantum annealers
Winci, W., Buffoni, L., Sadeghi, H., Khoshaman, A., Andriyash, E., & Amin, M. H. (2020) · Machine Learning: Science and Technology · doi:10.1088/2632-2153/aba220 · Cited by 54
Apical drive-A cellular mechanism of dreaming?
Aru, J., Siclari, F., Phillips, W. A., & Storm, J. F. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.09.018 · Cited by 74
Remembrance of Inferences Past
Dasgupta, I., Schulz, E., Goodman, N. D., & Gershman, S. J. (2017) · Cognition · doi:10.1016/j.cognition.2018.04.017 · Cited by 73
Maximal Causes for Non-linear Component Extraction
Lücke, J., & Sahani, M. (2008) · Journal of Machine Learning Research · doi:10.5555/1390681.1390722 · Cited by 73
Does the Wake-sleep Algorithm Produce Good Density Estimators?
Frey, B. J., Hinton, G. E., & Dayan, P. (1995) · UCL Discovery (University College London) · Cited by 73
Self-citationLearning Latent Space Energy-Based Prior ModelPreprint
Pang, B., Han, T., Nijkamp, E., Zhu, S. C., & Wu, Y. N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.08205 · Cited by 73
Spratling, M. W. (2012) · Neural Computation · doi:10.1162/neco_a_00222 · Cited by 67
Huys, Q. J. M., Zemel, R. S., Natarajan, R., & Dayan, P. (2007) · Neural Computation · doi:10.1162/neco.2007.19.2.404 · Cited by 67
Self-citationThe overfitted brain: Dreams evolved to assist generalization
Hoel, E. (2021) · Patterns · doi:10.1016/j.patter.2021.100244 · Cited by 70
“It is worth noting that the proposal of a "wake/sleep" specific algorithm for unsupervised learning of generative models based on feedback from stochastic stimulation goes back 25 years (Hinton et al., 1995).”
BackgroundFast depth image denoising and enhancement using a deep convolutional network
Zhang, X., & Wu, R. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472127 · Cited by 70
Research on exchange rate forecasting based on deep belief network
Zheng, J., Fu, X., & Zhang, G. (2017) · Neural Computing and Applications · doi:10.1007/s00521-017-3039-z · Cited by 70
Deep Neural Networks for Automatic Android Malware Detection
Hou, S., Saas, A., Chen, L., Ye, Y., & Bourlai, T. (2017) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1145/3110025.3116211 · Cited by 70
What kind of a graphical model is the brain
Hinton, G. E. (2005) · International Joint Conference on Artificial Intelligence · Cited by 69
Self-citationComparative Study of Deep Learning Models for Network Intrusion Detection
Lee, B., Amaresh, S., Green, C., & Engels, D. W. (2018) · SMU Data Science Review · Cited by 69
Learning Dual Retrieval Module for Semi-supervised Relation Extraction
Lin, H., Yan, J., Qu, M., & Ren, X. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313573 · Cited by 69
DualGraph: Improving Semi-supervised Graph Classification via Dual Contrastive Learning
Luo, X., Ju, W., Qu, M., Chen, C., Deng, M., Hua, X., & Zhang, M. (2022) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde53745.2022.00057 · Cited by 68
Bilinearity, rules, and prefrontal cortex
Dayan, P. (2007) · Frontiers in Computational Neuroscience · doi:10.3389/neuro.10.001.2007 · Cited by 65
Self-citationHasselmo, M. E., & Cekic, M. M. (1996) · Behavioural Brain Research · doi:10.1016/0166-4328(96)00010-1 · Cited by 68
Silences, spikes and bursts: Three‐part knot of the neural code
Friedenberger, Z., Harkin, E. F., Tóth, K., & Naud, R. (2023) · The Journal of Physiology · doi:10.1113/jp281510 · Cited by 67
Intelligent systems approaches to product sound quality evaluations – A review
Pietila, G., & Lim, T. C. (2012) · Applied Acoustics · doi:10.1016/j.apacoust.2012.04.012 · Cited by 68
A memristive deep belief neural network based on silicon synapses
Wang, W., Danial, L., Li, Y., Herbelin, E., Pikhay, E., Roizin, Y., Hoffer, B., Wang, Z., & Kvatinsky, S. (2022) · Nature Electronics · doi:10.1038/s41928-022-00878-9 · Cited by 66
Credit Assignment in Neural Networks through Deep Feedback Control
Meulemans, A., Farinha, M. T., Ordóñez, J. G., Aceituno, P. V., Sacramento, J., & Grewe, B. F. (2021) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000529286 · Cited by 66
Learning FRAME Models Using CNN Filters
Lu, Y., Zhu, S. C., & Wu, Y. N. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10238 · Cited by 66
Variational Bayesian Phylogenetic Inference
Zhang, C., & Matsen, F. A., IV (2018) · International Conference on Learning Representations · Cited by 65
Learning the Structure of Similarity
Tenenbaum, J. B. (1995) · neural information processing systems · Cited by 65
A neural network walks into a lab: towards using deep nets as models for human behavior
Ma, W. J., & Peters, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.02181 · Cited by 39
Hinton, G. E., Ghahramani, Z., & Teh, Y. W. (1999) · UCL Discovery (University College London) · Cited by 64
Self-citationInferring a Continuous Distribution of Atom Coordinates from Cryo-EM Images using VAEs
Rosenbaum, D., Garnelo, M., Zieliński, M., Beattie, C., Clancy, E., Huber, A., Kohli, P., Senior, A., Jumper, J., Doersch, C., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.14108 · Cited by 28
The predictive nature of spontaneous brain activity across scales and species
Dimakou, A., Pezzulo, G., Zangrossi, A., & Corbetta, M. (2025) · Neuron · doi:10.1016/j.neuron.2025.02.009 · Cited by 63
The Predictive Forward-Forward AlgorithmPreprint
Ororbia, A. G., & Mali, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.01452 · Cited by 63
Metadata
- Authors
- Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. M.
- Affiliations
- University of Toronto (CA)
- ORCID
- Dayan, Peter
- Venue
- Science
- Published
- 1995-05-26
- Volume
- 268
- Issue
- 5214
- Pages
- 1158-1161
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W1993845689
- Cited by
- 1,070
- References
- 9
- Citations by year
- 2026: 18 · 2025: 33 · 2024: 36 · 2023: 57 · 2022: 57 · 2021: 68 · 2020: 63 · 2019: 79 · 2018: 79 · 2017: 64 · 2016: 52 · 2015: 42 · 2014: 37 · 2013: 19 · 2012: 18
- FWCI
- 28.04
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
- Keywords
- unsupervised learning, stochastic neurons, multilayer neural networks, top-down connections, representation learning, generative models, neural network adaptation
- MeSH terms
- Algorithms, Probability, Stochastic Processes, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.