Unsupervised Discovery of Nonlinear Structure Using Contrastive Backpropagation
The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
We describe a way of modeling high‐dimensional data vectors by using an unsupervised, nonlinear, multilayer neural network in which the activity of each neuron‐like unit makes an additive contribution to a global energy score that indicates how surprised the network is by the data vector. The connection weights that determine how the activity of each unit depends on the activities in earlier layers are learned by minimizing the energy assigned to data vectors that are actually observed and maximizing the energy assigned to “confabulations” that are generated by perturbing an observed data vector in a direction that decreases its energy under the current model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Self-citationEmergence of simple-cell receptive field properties by learning a sparse code for natural images
Olshausen, B. A., & Field, D. J. (1996) · Nature · doi:10.1038/381607a0 · Cited by 5,694 · cited 1×
“structure in natural images ( Olshausen and Field, 1996; Bell and Sejnowski, 1997).”
BackgroundTraining Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 1×
“If we start the process at an observed data-vector and just run it for a few steps, we can generate a “confabulation” that works very well for adjusting the weights (Hinton, 2002).”
MethodSelf-citationBackpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879 · cited 1×
“through time ( Werbos, 1990 ) can then be used to obtain the derivatives of the energy”
MethodThe “independent components” of natural scenes are edge filters
Bell, A. J., & Sejnowski, T. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00121-1 · Cited by 2,201
Independent component filters of natural images compared with simple cells in primary visual cortex
van Hateren, J. H., & van der Schaaf, A. (1998) · Proceedings of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rspb.1998.0303 · Cited by 1,106
Cited by
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Learning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
A Practical Guide to Training Restricted Boltzmann Machines
Hinton, G. E. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_32 · Cited by 2,852
“Their most important use is as learning modules that are composed to form deep belief nets (Hinton et al., 2006a).”
BackgroundMethodSelf-citationSpiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition
Cao, Y., Chen, Y., & Khosla, D. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0788-3 · Cited by 917
Classification for High Resolution Remote Sensing Imagery Using a Fully Convolutional Network
Fu, G., Liu, C., Zhou, R., Sun, T., & Zhang, Q. (2017) · Remote Sensing · doi:10.3390/rs9050498 · Cited by 340
Metadata
- Authors
- Hinton, G., Osindero, S., Welling, M., & Teh, Y.
- Venue
- Cognitive Science
- Publisher
- Wiley
- Published
- 2006-07
- Volume
- 30
- Issue
- 4
- Pages
- 725-731
- ISSN
- 0364-0213 (ISSN-L)
- 1551-6709
- OpenAlex
- W2108581046
- Cited by
- 132
- References
- 6
- FWCI
- 1.91
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Unsupervised Discovery of Nonlinear Structure Using Contrastive Backpropagation. https://app.cochat.ai/papers/10.1207/s15516709cog0000_76
- The paper
- Hinton, G., Osindero, S., Welling, M., & Teh, Y. (2006). Unsupervised Discovery of Nonlinear Structure Using Contrastive Backpropagation. Cognitive Science. https://doi.org/10.1207/s15516709cog0000_76
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.