Learning with hidden variables
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Learning and inferring features that generate sensory input is a task continuously performed by cortex. In recent years, novel algorithms and learning rules have been proposed that allow neural network models to learn such features from natural images, written text, audio signals, etc. These networks usually involve deep architectures with many layers of hidden neurons. Here we review recent advancements in this area emphasizing, amongst other things, the processing of dynamical inputs by networks with hidden nodes and the role of single neuron models. These points and the questions they arise can provide conceptual advancements in understanding of learning in the cortex and the relationship between machine learning approaches to learning with hidden nodes and those in cortical circuits.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“Recent advances in optimization [53], gated (multiplicative) units in the architecture [54, 55] careful initialization [56, 57] and tricks such as gradient clipping [58] have led to impressive results in modeling speech, handwriting and language [59, 60, 61, 28, 30, 29, 62, 63].”
BackgroundGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Neural networks and physical systems with emergent collective computational abilities.
Hopfield, J. J. (1982) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.79.8.2554 · Cited by 19,270 · cited 1×
“In the 80s, following the work of John Hopfield [42], Daniel Amit [43] and others, statistical physicists showed interest in the studying of neural networks with binary neurons.”
BackgroundThe "Wake-Sleep" Algorithm for Unsupervised Neural Networks
Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. M. (1995) · Science · doi:10.1126/science.7761831 · Cited by 1,070
Shape and arrangement of columns in cat's striate cortex
Hubel, D. H., & Wiesel, T. N. (1963) · The Journal of Physiology · doi:10.1113/jphysiol.1963.sp007079 · Cited by 834 · cited 1×
“is performed in a hierarchical manner. Although hierarchical models of information processing leading to behaviour had already been suggested in the 50s [1], the experimental work of Hubel and Wiesel [2] and the discovery of simple and complex cells inspired much activity in studying hierarchical models of cortical pr…”
BackgroundIdentifying and attacking the saddle point problem in high-dimensional non-convex optimization
Dauphin, Y., Pascanu, R., Gulcehre, C., Cho, K., Ganguli, S., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.2572 · Cited by 827
CNN Features off-the-shelf: an Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · arXiv · doi:10.48550/arxiv.1403.6382 · Cited by 607
Cited by
Deep Learning in Drug Discovery
Gawehn, E., Hiss, J. A., & Schneider, G. (2016) · Molecular Informatics · doi:10.1002/minf.201501008 · Cited by 723
Inverse statistical problems: from the inverse Ising problem to data science
Nguyen, H. C., Zecchina, R., & Berg, J. (2017) · Advances in Physics · doi:10.1080/00018732.2017.1341604 · Cited by 258
Toward an integration of deep learning and neuroscience
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119
Onciul, R., Tataru, C.-I., Dumitru, A. V., Crivoi, C., Serban, M., Covache-Busuioc, R.-A., Radoi, M. P., & Toader, C. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14020550 · Cited by 113
“Perhaps no discovery in neuroscience has influenced AI more profoundly than the brain’s hierarchical processing of sensory information [127]. brain’s hierarchical processing of sensory information [127].”
Hybrid Network Model for “Deep Learning” of Chemical Data: Application to Antimicrobial Peptides
Schneider, P., Müller, A. T., Gabernet, G., Button, A. L., Posselt, G., Wessler, S., Hiss, J. A., & Schneider, G. (2017) · Molecular Informatics · doi:10.1002/minf.201600011 · Cited by 52
Metadata
- Authors
- Roudi, Y., & Taylor, G.
- Venue
- Current Opinion in Neurobiology
- Publisher
- Elsevier BV
- Published
- 2015-12
- Volume
- 35
- Pages
- 110-118
- ISSN
- 0959-4388 (ISSN-L)
- OpenAlex
- W1707522557
- Cited by
- 23
- References
- 9
- FWCI
- 1.37
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.