The DOI matches its Crossref record; 217 of the 217 references it cites were found in a registry or an index.
Learning Deep Architectures for AI
Abstract
Can machine learning deliver AI? Theoretical results, inspiration from the brain and cognition, as well as machine learning experiments suggest that in order to learn the kind of complicated functions that can represent high-level abstractions (e.g. in vision, language, and other AI-level tasks), one would need deep architectures. Deep architectures are composed of multiple levels of non-linear operations, such as in neural nets with many hidden layers, graphical models with many levels of latent variables, or in complicated propositional formulae re-using many sub-formulae. Each level of the architecture represents features at a different level of abstraction, defined as a composition of lower-level features. Searching the parameter space of deep architectures is a difficult task, but new algorithms have been discovered and a new sub-area has emerged in the machine learning community since 2006, following these discoveries. Learning algorithms such as those for Deep Belief Networks and other related unsupervised learning algorithms have recently been proposed to train deep architectures, yielding exciting results and beating the state-of-the-art in certain areas. Learning Deep Architectures for AI discusses the motivations for and principles of learning algorithms for deep architectures. By analyzing and comparing recent results with different learning algorithms for deep architectures, explanations for their success are proposed and discussed, highlighting challenges and suggesting avenues for future explorations in this area.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Self-citationOptimization by Simulated Annealing
Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983) · Science · doi:10.1126/science.220.4598.671 · Cited by 45,134
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
(2008) · Journal of Machine Learning Research · Cited by 33,051
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,522
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 20,825
A logical calculus of the ideas immanent in nervous activity
McCulloch, W. S., & Pitts, W. (1943) · The Bulletin of Mathematical Biophysics · doi:10.1007/bf02478259 · Cited by 18,566
Stochastic relaxation, gibbs distributions, and the bayesian restoration of images.
Geman, S., & Geman, D. (1984) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1984.4767596 · Cited by 17,949
Donoho, D. L. (2004) · Cited by 16,703
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Nonlinear Dimensionality Reduction by Locally Linear Embedding
Roweis, S. T., & Saul, L. K. (2000) · Science · doi:10.1126/science.290.5500.2323 · Cited by 14,965
Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex
Hubel, D. H., & Wiesel, T. N. (1962) · The Journal of Physiology · doi:10.1113/jphysiol.1962.sp006837 · Cited by 13,680
A Global Geometric Framework for Nonlinear Dimensionality Reduction
Tenenbaum, J. B., Silva, V. D., & Langford, J. C. (2000) · Science · doi:10.1126/science.290.5500.2319 · Cited by 13,585
Indexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
A training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
The origins of intelligence in children.
Piaget, J. (1952) · doi:10.1037/11494-000 · Cited by 10,687
Analysis of a complex of statistical variables into principal components.
Hotelling, H. (1933) · Journal of Educational Psychology · doi:10.1037/h0071325 · Cited by 9,633
An Information-Maximization Approach to Blind Separation and Blind Deconvolution
Bell, A. J., & Sejnowski, T. J. (1995) · Neural Computation · doi:10.1162/neco.1995.7.6.1129 · Cited by 8,986
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542
Self-citationNonlinear Component Analysis as a Kernel Eigenvalue Problem
Schölkopf, B., Smola, A., & Müller, K.-R. (1998) · Neural Computation · doi:10.1162/089976698300017467 · Cited by 8,084
Wolpert, D. H. (1992) · Neural Networks · doi:10.1016/s0893-6080(05)80023-1 · Cited by 7,951
Independent Component Analysis
(2004) · Independent Component Analysis · doi:10.7551/mitpress/3717.003.0014 · Cited by 7,903
Extracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationDecoding by Linear Programming
Candes, E. J., & Tao, T. (2005) · IEEE Transactions on Information Theory · doi:10.1109/tit.2005.858979 · Cited by 7,121
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Advances in kernel methods: support vector learning
Schölkopf, B., Burges, C. J. C., & Smola, A. J. (1999) · doi:10.5555/299094 · Cited by 5,789
Fukushima, K. (1980) · Biological Cybernetics · doi:10.1007/bf00344251 · Cited by 5,759
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366
Self-citationDimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.100 · Cited by 5,336
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058
Ho, T. K. (2002) · Proceedings of 3rd International Conference on Document Analysis and Recognition · doi:10.1109/icdar.1995.598994 · Cited by 5,054
McClelland, J. L., & Rumelhart, D. E. (1981) · Psychological Review · doi:10.1037/0033-295x.88.5.375 · Cited by 4,889
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879
Greedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationAn introduction to hidden Markov models
Rabiner, L., & Juang, B. (1986) · IEEE ASSP Magazine · doi:10.1109/massp.1986.1165342 · Cited by 4,775
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276
Duane, S., Kennedy, A. D., Pendleton, B. J., & Roweth, D. (1987) · Physics Letters B · doi:10.1016/0370-2693(87)91197-x · Cited by 3,872
Learning with Local and Global Consistency
Zhou, D., Bousquet, O., Lal, T. N., Weston, J., & Schölkopf, B. (2003) · MPG.PuRe (Max Planck Society) · Cited by 3,864
An Introduction to Kolmogorov Complexity and Its Applications
Li, M., & Vitányi, P. (2019) · Texts in Computer Science · doi:10.1007/978-3-030-11298-1 · Cited by 3,681
Sparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642
An Energy Budget for Signaling in the Grey Matter of the Brain
Attwell, D., & Laughlin, S. B. (2001) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1097/00004647-200110000-00001 · Cited by 3,543
Semi-supervised learning using Gaussian fields and harmonic functions
Zhu, X., Zoubin Ghahramani, A., & Lafferty, J. (2003) · Cited by 3,509
A Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Self-citationA learning algorithm for Boltzmann machines
Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985) · Cognitive Science · doi:10.1207/s15516709cog0901_7 · Cited by 3,402
Best practices for convolutional neural networks applied to visual document analysis
Simard, P. Y., Steinkraus, D., & Platt, J. C. (2005) · Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings. · doi:10.1109/icdar.2003.1227801 · Cited by 2,741
Parallel & distributed processing
Slusallek, P., Shirley, P., Mark, W., Stoll, G., & Wald, I. (2005) · ACM SIGGRAPH 2005 Courses on - SIGGRAPH '05 · doi:10.1145/1198555.1198750 · Cited by 2,568
An introduction to MCMC for machine learning.
Andrieu, C., de Freitas, N., Doucet, A., & Jordan, M. I. (2003) · Machine Learning · doi:10.1023/a:1020281327116 · Cited by 2,451
(1998) · Springer eBooks · doi:10.1007/978-94-011-5014-9 · Cited by 2,418
Lee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553453 · Cited by 2,412
Efficient sparse coding algorithms
Lee, H., Battle, A., Raina, R., & Ng, A. Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0105 · Cited by 2,394
Neural Networks: Tricks of the Trade: Second Edition
(2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8 · Cited by 2,116
Three approaches to the quantitative definition of information*
Kolmogorov, A. N. (1968) · International Journal of Computer Mathematics · doi:10.1080/00207166808803030 · Cited by 2,031
Learning and development in neural networks: the importance of starting small
Elman, J. L. (1993) · Cognition · doi:10.1016/0010-0277(93)90058-4 · Cited by 1,988
(2001) · Cited by 1,932
Salakhutdinov, R., & Hinton, G. E. (2009) · Cited by 1,890
Restricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
A formal theory of inductive inference. Part II
Solomonoff, R. J. (1964) · Information and Control · doi:10.1016/s0019-9958(64)90131-7 · Cited by 1,758
(2003) · Cited by 1,733
An Introduction to Mathematical Logic
(2005) · Discrete Algorithmic Mathematics · doi:10.1201/b13782-17 · Cited by 1,732
Information processing in dynamical systems: foundations of harmony theory
Smolensky, P. (1986) · The MIT Press eBooks · Cited by 1,671
Stochastic Complexity in Statistical Inquiry
Rissanen, J. (1998) · WORLD SCIENTIFIC eBooks · doi:10.1142/0822 · Cited by 1,539
Self-taught learning: transfer learning from unlabeled data
Raina, R., Battle, A., Lee, H., Packer, B., & Ng, A. Y. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273592 · Cited by 1,510
Hierarchical Bayesian inference in the visual cortex
Lee, T. S., & Mumford, D. (2003) · Journal of the Optical Society of America A · doi:10.1364/josaa.20.001434 · Cited by 1,473
Slow Feature Analysis: Unsupervised Learning of Invariances
Wiskott, L., & Sejnowski, T. J. (2002) · Neural Computation · doi:10.1162/089976602317318938 · Cited by 1,350
Auto-association by multilayer perceptrons and singular value decomposition
Bourlard, H., & Kamp, Y. (1988) · Biological Cybernetics · doi:10.1007/bf00332918 · Cited by 1,345
LeCun, Y., Bottou, L., Orr, G., & Müller, K. (1998) · neural information processing systems · Cited by 1,333
Learning methods for generic object recognition with invariance to pose and lighting
LeCun, Y., Huang, F. J., & Bottou, L. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315150 · Cited by 1,324
Active Learning with Statistical Models
Cohn, D. A., Ghahramani, Z., & Jordan, M. I. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.295 · Cited by 1,317
Exploiting Generative Models in Discriminative Classifiers
Jaakkola, T., & Haussler, D. H. (1998) · Cited by 1,271
Numerical Continuation Methods
Allgower, E. L., & Georg, K. (1990) · Springer Series in Computational Mathematics · doi:10.1007/978-3-642-61257-2 · Cited by 1,269
Gaussian Processes for Regression
Williams, C. K. I., & Rasmussen, C. E. (1995) · Aston Publications Explorer (Aston University) · Cited by 1,246
Salakhutdinov, R., & Hinton, G. (2009) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2008.11.006 · Cited by 1,246
Dayan, P., Hinton, G. E., Neal, R. M., & Zemel, R. S. (1995) · Neural Computation · doi:10.1162/neco.1995.7.5.889 · Cited by 1,207
(1993) · Cited by 1,173
Input space versus feature space in kernel-based methods
Scholkopf, B., Mika, S., Burges, C. J. C., Knirsch, P., Muller, K.-R., Ratsch, G., & Smola, A. J. (1999) · IEEE Transactions on Neural Networks · doi:10.1109/72.788641 · Cited by 1,164
The cost of cortical computation.
Lennie, P. (2003) · Current Biology · doi:10.1016/s0960-9822(03)00135-0 · Cited by 1,134
Learning Overcomplete Representations
Lewicki, M. S., & Sejnowski, T. J. (2000) · Neural Computation · doi:10.1162/089976600300015826 · Cited by 1,129
Efficient Learning of Sparse Representations with an Energy-Based Model
Ranzato, M., Poultney, C., Chopra, S., & LeCun, Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0147 · Cited by 1,106
Exploring Strategies for Training Deep Neural Networks
Larochelle, H., Bengio, Y., Louradour, J., & Lamblin, P. (2009) · doi:10.1145/1577069.1577070 · Cited by 1,095
Self-citationUnsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition
Ranzato, M., Huang, F. J., Boureau, Y.-L., & LeCun, Y. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383157 · Cited by 1,086
The "Wake-Sleep" Algorithm for Unsupervised Neural Networks
Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. M. (1995) · Science · doi:10.1126/science.7761831 · Cited by 1,070
Learning and relearning in Boltzmann machines
Hinton, G. E., & Sejnowski, T. J. (2001) · Graphical Models · doi:10.7551/mitpress/3349.003.0005 · Cited by 1,037
An empirical evaluation of deep architectures on problems with many factors of variation
Larochelle, H., Erhan, D., Courville, A., Bergstra, J., & Bengio, Y. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273556 · Cited by 992
Self-citationNumerical Continuation Methods--An Introduction.
Watson, L. T., Allgower, E. L., & Georg, K. (1992) · Mathematics of Computation · doi:10.2307/2153001 · Cited by 991
A combined SVM and LDA approach for classification
Xiong, T., & Cherkassky, V. (2006) · Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005. · doi:10.1109/ijcnn.2005.1556089 · Cited by 980
Supervised Dictionary Learning
Mairal, J., Bach, F., Ponce, J., Sapiro, G., & Zisserman, A. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0809.3083 · Cited by 938
(2007) · Cited by 937
An Information Measure for Classification
Wallace, C. S., & Boulton, D. M. (1968) · The Computer Journal · doi:10.1093/comjnl/11.2.185 · Cited by 933
Parallel models of associative memory
Hinton, G. E., & Anderson, J. A. (1981) · CERN Document Server (European Organization for Nuclear Research) · Cited by 927
The Complexity of Boolean Functions
Wegener, I. (1987) · Cited by 882
Recursive distributed representations
Pollack, J. B. (1990) · Artificial Intelligence · doi:10.1016/0004-3702(90)90005-k · Cited by 852
(2007) · Cited by 849
A Tutorial on Energy-Based Learning
LeCun, Y., Chopra, S., Hadsell, R. T., Ranzato, A., & Huang, F. J. (2006) · Cited by 842
Explorations in parallel distributed processing
McClelland, J. L. (1988) · Cited by 841
A Scalable Hierarchical Distributed Language Model
Mnih, A., & Hinton, G. E. (2008) · Cited by 838
Training restricted Boltzmann machines using approximations to the likelihood gradient
Tieleman, T. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390290 · Cited by 837
Cristianini, N., Kandola, J., Elisseeff, A., & Shawe-Taylor, J. (2006) · Studies in Fuzziness and Soft Computing · doi:10.1007/3-540-33486-6_8 · Cited by 837
Learning distributed representations of concepts.
Hinton, G. E. (1989) · eScholarship (California Digital Library) · Cited by 827
Representational Power of Restricted Boltzmann Machines and Deep Belief Networks
Le Roux, N., & Bengio, Y. (2008) · Neural Computation · doi:10.1162/neco.2008.04-07-510 · Cited by 821
Self-citationPractical issues in temporal difference learning
Tesauro, G. (1992) · Machine Learning · doi:10.1007/bf00992697 · Cited by 787
(2005) · doi:10.5555/1046920.1088696 · Cited by 756
Sparse Feature Learning for Deep Belief Networks
Ranzato, M., Boureau, Y.-L., & LeCun, Y. (2007) · neural information processing systems · Cited by 745
Segmentation using eigenvectors: a unifying view
Weiss, Y. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790354 · Cited by 729
Small codes and large image databases for recognition
Torralba, A., Fergus, R., & Weiss, Y. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587633 · Cited by 727
Explorations in Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises
Kawamoto, A. H., McClelland, J. L., & Rumelhart, D. E. (1989) · The American Journal of Psychology · doi:10.2307/1423065 · Cited by 726
Modeling Human Motion Using Binary Latent Variables
Taylor, G. W., Hinton, G. E., & Roweis, S. T. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0173 · Cited by 689
Classification using discriminative restricted Boltzmann machines
Larochelle, H., & Bengio, Y. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390224 · Cited by 655
Self-citationOn Contrastive Divergence Learning.
Carreira-Perpiñán, M. Á., & Hinton, G. E. (2005) · Cited by 630
Almost optimal lower bounds for small depth circuits
Hastad, J. (1986) · Proceedings of the eighteenth annual ACM symposium on Theory of computing - STOC '86 · doi:10.1145/12130.12132 · Cited by 620
The Entire Regularization Path for the Support Vector Machine
Hastie, T. J., Rosset, S., Tibshirani, R. J., & Zhu, J. (2004) · Cited by 607
Unsupervised Learning of Models for Recognition
Weber, M., Welling, M., & Perona, P. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45054-8_2 · Cited by 594
Gasarch, W. (1997) · ACM SIGACT News · doi:10.1145/262301.1040343 · Cited by 583
Fundamentals of statistical exponential families with applications in statistical decision theory
Brown, L. D. (1986) · SPIE eBooks · doi:10.1214/lnms/1215466757 · Cited by 573
Three new graphical models for statistical language modelling
Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273577 · Cited by 563
Separating the polynomial-time hierarchy by oracles
Yao, A. C.-C. (1985) · 26th Annual Symposium on Foundations of Computer Science (sfcs 1985) · doi:10.1109/sfcs.1985.49 · Cited by 548
Regularization and Semi-supervised Learning on Large Graphs
Belkin, M., Matveeva, I., & Niyogi, P. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-27819-1_43 · Cited by 536
(2007) · The MIT Press eBooks · doi:10.7551/mitpress/7496.001.0001 · Cited by 536
(2007) · The MIT Press eBooks · doi:10.7551/mitpress/7443.001.0001 · Cited by 525
Universal Artificial Intellegence
Hutter, M. (2005) · Texts in Theoretical Computer Science An EATCS Series · doi:10.1007/b138233 · Cited by 523
(1995) · Cited by 520
Connectionist learning of belief networks
Neal, R. M. (1992) · Artificial Intelligence · doi:10.1016/0004-3702(92)90065-6 · Cited by 513
A Bayesian/Information Theoretic Model of Learning to Learn via Multiple Task Sampling
Baxter, J. (1997) · Machine Learning · doi:10.1023/a:1007327622663 · Cited by 462
(1992) · neural information processing systems · Cited by 460
A survey of kernels for structured data
Gärtner, T. (2003) · ACM SIGKDD Explorations Newsletter · doi:10.1145/959242.959248 · Cited by 452
Learning a Nonlinear Embedding by Preserving Class Neighbourhood Structure
Salakhutdinov, R., & Hinton, G. E. (2007) · Cited by 440
Exponential Family Harmoniums with an Application to Information Retrieval
Welling, M., Rosen‐Zvi, M., & Hinton, G. E. (2004) · Cited by 423
A Hierarchical Model of Shape and Appearance for Human Action Classification
Niebles, J. C., & Fei-Fei, L. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383132 · Cited by 421
On the quantitative analysis of deep belief networks
Salakhutdinov, R., & Murray, I. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390266 · Cited by 411
Brand, M. (2002) · Cited by 396
Learning the Kernel Matrix with Semi-Definite Programming
Lanckriet, G. R. G., Christianini, N., Bartlett, P. L., Ghaoui, L. E., & Jordan, M. I. (2002) · QUT ePrints (Queensland University of Technology) · doi:10.5555/1005332.1005334 · Cited by 386
Fundamentals of Statistical Exponential Families
Brown, L. D. (1987) · Cited by 386
Deep learning via semi-supervised embedding
Weston, J., Ratle, F., & Collobert, R. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390303 · Cited by 380
Object Class Recognition and Localization Using Sparse Features with Limited Receptive Fields
Mutch, J., & Lowe, D. G. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0118-0 · Cited by 373
Factored conditional restricted Boltzmann Machines for modeling motion style
Taylor, G. W., & Hinton, G. E. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553505 · Cited by 358
The Difficulty of Training Deep Architectures and the Effect of Unsupervised Pre-Training
Erhan, D., Manzagol, P.-A., Bengio, Y., Samy Bengio, A., & Vincent, P. (2009) · International Conference on Artificial Intelligence and Statistics · Cited by 346
Self-citationLearning Eigenfunctions Links Spectral Embedding and Kernel PCA
Bengio, Y., Delalleau, O., Roux, N. L., Paiement, J.-F., Vincent, P., & Ouimet, M. (2004) · Neural Computation · doi:10.1162/0899766041732396 · Cited by 345
Self-citationHinton, G. E. (1999) · 9th International Conference on Artificial Neural Networks: ICANN '99 · doi:10.1049/cp:19991075 · Cited by 341
Principled Hybrids of Generative and Discriminative Models
Lasserre, J. A., Bishop, C. M., & Minka, T. P. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.227 · Cited by 335
Deep learning from temporal coherence in video
Mobahi, H., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553469 · Cited by 328
Mean Field Theory for Sigmoid Belief Networks
Saul, L. K., Jaakkola, T., & Jordan, M. I. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.251 · Cited by 327
To recognize shapes, first learn to generate images
Hinton, G. E. (2007) · Progress in Brain Research · doi:10.1016/s0079-6123(06)65034-6 · Cited by 309
Links between perceptrons, MLPs and SVMs
Collobert, R., & Bengio, S. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015415 · Cited by 302
Using fast weights to improve persistent contrastive divergence
Tieleman, T., & Hinton, G. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553506 · Cited by 274
Unsupervised learning of distributions on binary vectors using two layer networks
Freund, Y., & Haussler, D. H. (1991) · Cited by 267
On the power of small-depth threshold circuits
H�stad, J., & Goldmann, M. (1991) · Computational Complexity · doi:10.1007/bf01272517 · Cited by 247
Flexible shaping: How learning in small steps helps
Krueger, K. A., & Dayan, P. (2009) · Cognition · doi:10.1016/j.cognition.2008.11.014 · Cited by 234
A quantitative theory of immediate visual recognition
Serre, T., Kreiman, G., Kouh, M., Cadieu, C. F., Knoblich, U., & Poggio, T. (2007) · Progress in brain research · doi:10.1016/s0079-6123(06)65004-8 · Cited by 222
Generative models for discovering sparse distributed representations
Hinton, G. E., & Ghahramani, Z. (1997) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.1997.0101 · Cited by 220
Fast Inference in Sparse Coding Algorithms with Applications to Object RecognitionPreprint
Kavukcuoglu, K., Ranzato, M., & LeCun, Y. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1010.3467 · Cited by 220
Semi-supervised learning of compact document representations with deep networks
Ranzato, M. A., & Szummer, M. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390256 · Cited by 217
Global Continuation for Distance Geometry Problems
Morè, J. J., & Wu, Z. (1997) · SIAM Journal on Optimization · doi:10.1137/s1052623495283024 · Cited by 214
Learning Internal Representations
Baxter, J. (1995) · Cited by 211
Justifying and Generalizing Contrastive Divergence
Bengio, Y., & Delalleau, O. (2008) · Neural Computation · doi:10.1162/neco.2008.11-07-647 · Cited by 209
Self-citationThe Curse of Highly Variable Functions for Local Kernel Machines
Bengio, Y., Delalleau, O., & Roux, N. L. (2005) · Cited by 193
Self-citationLearning Multilevel Distributed Representations for High-Dimensional Sequences
Sutskever, I., & Hinton, G. E. (2007) · Cited by 193
Natural language processing with modular pdp networks and distributed lexicon
MIIKKULAINEN, R., & DYER, M. (1991) · Cognitive Science · doi:10.1016/0364-0213(91)80002-m · Cited by 183
Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes
Hinton, G. E., & Salakhutdinov, R. (2007) · neural information processing systems · Cited by 177
Unsupervised Learning of Image Transformations
Memisevic, R., & Hinton, G. E. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383036 · Cited by 176
Some extensions of score matching
Hyvärinen, A. (2007) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2006.09.003 · Cited by 174
Efficient Non-Parametric Function Induction in Semi-Supervised LearningPreprint
Bengio, Y., Delalleau, O., & Roux, N. L. (2004) · RePEc: Research Papers in Economics · Cited by 170
Self-citationShift-invariant sparse coding for audio classification
Grosse, R., Raina, R., Kwong, H., & Ng, A. Y. (2007) · Uncertainty in Artificial Intelligence · Cited by 168
Nonlinear Autoassociation Is Not Equivalent to PCA
Japkowicz, N., Hanson, S. J., & Gluck, M. A. (2000) · Neural Computation · doi:10.1162/089976600300015691 · Cited by 167
Rational Kernels: Theory and Algorithms
Cortes, C., Haffner, P., & Mohri, M. (2004) · doi:10.5555/1005332.1016793 · Cited by 165
Describing Visual Scenes Using Transformed Objects and Parts
Sudderth, E. B., Torralba, A., Freeman, W. T., & Willsky, A. S. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0069-5 · Cited by 164
Ahmed, A. A., Yu, K., Xu, W. H., Gong, Y., & Xing, E. P. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88690-7_6 · Cited by 154
Bengio, Y., Roux, N. L., Vincent, P., Delalleau, O., & Marcotte, P. (2005) · Cited by 153
Self-citationUsing Manifold Stucture for Partially Labeled Classification
Belkin, M., & Niyogi, P. (2002) · neural information processing systems · Cited by 151
A Context-Sensitive Generalization of ICA
Pearlmutter, B. A., & Parra, L. C. (1996) · Maynooth University ePrints and eTheses Archive (Maynooth University) · Cited by 145
A DAY OF GREAT ILLUMINATION: B. F. SKINNER'S DISCOVERY OF SHAPING
Peterson, G. B. (2004) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2004.82-317 · Cited by 143
How to invent a lexicon: the development of shared symbols in interaction
Hutchins, E. L., & Hazlehurst, B. L. (2006) · Routledge eBooks · doi:10.4324/9780203993699-17 · Cited by 139
(2000) · Applied Physics Letters · doi:10.1162/jmlr.2003.4.7-8.1235 · Cited by 139
Unsupervised Discovery of Nonlinear Structure Using Contrastive Backpropagation
Hinton, G. E., Osindero, S., Welling, M., & Teh, Y. (2006) · Cognitive Science · doi:10.1207/s15516709cog0000_76 · Cited by 133
Utgoff, P. E., & Stracuzzi, D. J. (2002) · Neural Computation · doi:10.1162/08997660260293319 · Cited by 129
Connectionist language modeling for large vocabulary continuous speech recognition
Schwenk, H., & Gauvain, J. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2002.5743830 · Cited by 125
Loss Functions for Discriminative Training of Energy-Based Models
LeCun, Y., & Huang, F. J. (2005) · Cited by 123
An asymptotic analysis of generative, discriminative, and pseudolikelihood estimators
Liang, P., & Jordan, M. I. (2008) · International Conference on Machine Learning · doi:10.1145/1390156.1390230 · Cited by 119
A Unified Energy-Based Framework for Unsupervised Learning
Ranzato, M., Boureau, Y.-L., Chopra, S., & LeCun, Y. (2007) · International Conference on Artificial Intelligence and Statistics · Cited by 105
Modeling image patches with a directed hierarchy of Markov random fields
Osindero, S., & Hinton, G. E. (2007) · Cited by 103
Computational complexity of neural networks: a survey
Orponen, P. (1994) · Cited by 102
Is postnatal neocortical maturation hierarchical?
Guillery, R. W. (2005) · Trends in Neurosciences · doi:10.1016/j.tins.2005.08.006 · Cited by 101
Deep belief net learning in a long-range vision system for autonomous off-road driving
Hadsell, R., Erkan, A., Sermanet, P., Scoffier, M., Muller, U., & LeCun, Y. (2008) · 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2008.4651217 · Cited by 101
Bagnell, J. A., & Bradley, D. M. (2008) · neural information processing systems · Cited by 92
Decision trees do not generalize to new variations
Bengio, Y., Delalleau, O., & Simard, C. (2010) · Computational Intelligence · doi:10.1111/j.1467-8640.2010.00366.x · Cited by 89
Learning Continuous Attractors in Recurrent Networks
Seung, H. S. (1997) · Cited by 76
Learning Nonlinear Overcomplete Representations for Efficient Coding
Lewicki, M. S., & B, T. J. S. A. (1997) · Cited by 72
Non-Local Manifold Tangent Learning
Bengio, Y., & Monperrus, M. (2004) · Cited by 65
Self-citationEvaluating probabilities under high-dimensional latent variable models
Murray, I., & Salakhutdinov, R. (2008) · ERA · Cited by 61
Slow, Decorrelated Features for Pretraining Complex Cell-like Networks
Bengio, Y., & Bergstra, J. (2009) · neural information processing systems · Cited by 61
Self-citationA Nonparametric Bayesian Approach to Modeling Overlapping Clusters.
Heller, K., & Zoubin Ghahramani, A. (2007) · Cited by 56
Hyvärinen, A. (2007) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2007.895819 · Cited by 51
Nonlocal Estimation of Manifold Structure
Bengio, Y., Monperrus, M., & Larochelle, H. (2006) · Neural Computation · doi:10.1162/neco.2006.18.10.2509 · Cited by 45
Self-citationConstituent Parsing with Incremental Sigmoid Belief Networks
Titov, I., & Henderson, J. (2007) · Cited by 43
Establishing Good Benchmarks and Baselines for Face Recognition
Pinto, N., DiCarlo, J. J., & Cox, D. (2008) · HAL · Cited by 42
Welling, M., Zemel, R. S., & Hinton, G. E. (2002) · Cited by 36
Cited by
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Self-citationGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
(2008) · Journal of Machine Learning Research · Cited by 33,051
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Metadata
- Authors
- Bengio, Y.
- Venue
- now publishers, Inc. eBooks
- Publisher
- now Publishers Inc
- Published
- 2009
- OpenAlex
- W2072128103
- Cited by
- 4,941
- References
- 217
- FWCI
- 28.65
- Type
- book
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
