The DOI matches its Crossref record; 217 of the 217 references it cites were found in a registry or an index.
Learning Deep Architectures for AI
Abstract
Can machine learning deliver AI? Theoretical results, inspiration from the brain and cognition, as well as machine learning experiments suggest that in order to learn the kind of complicated functions that can represent high-level abstractions (e.g. in vision, language, and other AI-level tasks), one would need deep architectures. Deep architectures are composed of multiple levels of non-linear operations, such as in neural nets with many hidden layers, graphical models with many levels of latent variables, or in complicated propositional formulae re-using many sub-formulae. Each level of the architecture represents features at a different level of abstraction, defined as a composition of lower-level features. Searching the parameter space of deep architectures is a difficult task, but new algorithms have been discovered and a new sub-area has emerged in the machine learning community since 2006, following these discoveries. Learning algorithms such as those for Deep Belief Networks and other related unsupervised learning algorithms have recently been proposed to train deep architectures, yielding exciting results and beating the state-of-the-art in certain areas. Learning Deep Architectures for AI discusses the motivations for and principles of learning algorithms for deep architectures. By analyzing and comparing recent results with different learning algorithms for deep architectures, explanations for their success are proposed and discussed, highlighting challenges and suggesting avenues for future explorations in this area.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–217 of 217
A Two-Layer ICA-Like Model Estimated by Score Matching
Köster, U., & Hyvärinen, A. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74695-9_82 · Cited by 32
Efficient training of large neural networks for language modeling
Schwenk, H. (2005) · IEEE International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2004.1381158 · Cited by 32
Learning long-term dependencies is not as difficult with NARX recurrent neural networks
Lin, T.-N., Horne, B. G., Tiňo, P., et al. (1995) · University Libraries (University of Maryland) · Cited by 30
Global data analysis and the fragmentation problem in decision tree induction
Vilalta, R., Blix, G., & Rendell, L. A. (1997) · Lecture notes in computer science · doi:10.1007/3-540-62858-4_95 · Cited by 29
Smoothing Techniques for Macromolecular Global Optimization
Morè, J. J., & Zhijun, W. (1996) · Springer eBooks · doi:10.1007/978-1-4899-0289-4_21 · Cited by 22
Advances in Neural Information Processing Systems 20 (NIPS 2007)
Opper, M., & Sanguinetti, G. (2008) · Cited by 21
Training connectionist models for the structured language model
Xu, P., Emami, A., & Jelinek, F. (2003) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1119355.1119376 · Cited by 20
A Theoretical Analysis of Robust Coding over Noisy Overcomplete Channels
Doi, E., Balcan, D. C., & Lewicki, M. S. (2005) · Cited by 19
A Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images
Ranzato, M., & LeCun, Y. (2007) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2007.4377108 · Cited by 17
Paccanaro, A., & Hinton, G. E. (2000) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2000.857906 · Cited by 17
Memoires associatives distribuees: Une comparaison (Distributed associative memories: A comparison)
Gallinari, P., LeCun, Y., Thiria, S., & Soulié, F. F. (1987) · Cited by 15
How to Make a Low-Dimensional Representation Suitable for Diverse TasksPreprint
Intrator, N., & Edelman, S. (1996) · CogPrints (University of Southampton) · Cited by 13
Learning despite concept variation by finding structure in attribute-based data
Pérez, E., & Rendell, L. A. (1996) · Cited by 12
Descartes' Rule of Signs for Radial Basis Function Neural Networks
Schmitt, M. (2002) · Neural Computation · doi:10.1162/089976602760805386 · Cited by 12
Parallel continuation-based global optimization for molecular conformation and protein folding
Coleman, T. F., & Wu, Z. (1994) · University of North Texas Digital Library (University of North Texas) · Cited by 10
Complexity Lower Bounds for Approximation Algebraic Computation Trees
Cucker, F., & Grigoriev, D. (1999) · Journal of Complexity · doi:10.1006/jcom.1999.0519 · Cited by 7
(1996)
Cited by
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Self-citationGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
(2008) · Journal of Machine Learning Research · Cited by 33,051
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Metadata
- Authors
- Bengio, Y.
- Venue
- now publishers, Inc. eBooks
- Publisher
- now Publishers Inc
- Published
- 2009
- OpenAlex
- W2072128103
- Cited by
- 4,941
- References
- 217
- FWCI
- 28.65
- Type
- book
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
