Large-scale deep unsupervised learning using graphics processors
Abstract
The promise of unsupervised learning methods lies in their potential to use vast amounts of unlabeled data to learn complex, highly nonlinear models with millions of free parameters. We consider two well-known unsupervised learning models, deep belief networks (DBNs) and sparse coding, that have recently been applied to a flurry of machine learning applications (Hinton & Salakhutdinov, 2006; Raina et al., 2007). Unfortunately, current learning algorithms for both models are too slow for large-scale applications, forcing researchers to focus on smaller-scale models, or to use fewer training examples.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
CNNs were trained on GPUs at least as early as 2006.
“CNNs were trained on GPUs at least as early as 2006 (Chellapilla et al., 2006), as were some other models that were large at the time (such as the 100M parameter Deep Belief Network by Raina et al. (2009a) and the neural networks of D. C. Cire¸ san et al. (2011)).”
From Compute Trends Across Three Eras of Machine Learning (2022) · cites this paper as Raina et al. (2009a)
Massively parallel processing architectures with distributed memory architectures, such as GPUs and specialized chips for NN inference, have shown to be very energy efficient and capable of impressive computational throughput when fully utilized by concurrent algorithms.
“massively parallel processing architectures with distributed memory architectures, such as graphic processing units (GPUs) but also increasingly specialized chips for NN inference (e.g., [17]), have shown to be very energy efficient and capable of impressive computational throughput when fully utilized by concurrent algorithms [18].”
From An Introduction to Deep Learning for the Physical Layer (2017) · cites this paper as [18]
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
References 32 · Cited by 742
- Authors
- Raina, R., Madhavan, A., & Ng, A. Y.
- Affiliations
- Stanford University (US)
- ORCID
- Ng, Andrew Y.
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 873-880
- OpenAlex
- W2120432001
- Cited by
- 742
- References
- 32
- Citations by year
- 2026: 22 · 2025: 52 · 2024: 29 · 2023: 38 · 2022: 47 · 2021: 60 · 2020: 81 · 2019: 74 · 2018: 67 · 2017: 56 · 2016: 46 · 2015: 45 · 2014: 40 · 2013: 28 · 2012: 17
- FWCI
- 10.99
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Funders
- Defense Advanced Research Projects Agency (FA8750-05-2-0249)
- Office of Naval Research (MURI N000140710747)
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- unsupervised learning, deep belief network, sparse coding, graphics processors, GPU acceleration, large-scale machine learning, unlabeled data, training efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
