The DOI matches its Crossref record; 52 of the 56 references it cites were found in a registry or an index.
Convolutional networks and applications in vision
Open accessAbstract
Intelligent tasks, such as visual perception, auditory perception, and language understanding require the construction of good internal representations of the world (or "features")? which must be invariant to irrelevant variations of the input while, preserving relevant information. A major question for Machine Learning is how to learn such good features automatically. Convolutional Networks (ConvNets) are a biologically-inspired trainable architecture that can learn invariant features. Each stage in a ConvNets is composed of a filter bank, some nonlinearities, and feature pooling layers. With multiple stages, a ConvNet can learn multi-level hierarchies of features. While ConvNets have been successfully deployed in many commercial applications from OCR to video surveillance, they require large amounts of labeled training samples. We describe new unsupervised learning algorithms, and new non-linear stages that allow ConvNets to be trained with very few labeled samples. Applications to visual object recognition and vision navigation for off-road mobile robots are described.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Self-citationDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · cited 3×
“Most common feature extraction systems like SIFT [40], HoG [41] are composed of filterbanks (oriented edge detectors at multiple scales) followed by non-linearities (winner take all) and pooling (histogramming).”
MethodHistograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584 · cited 2×
“Most common feature extraction systems like SIFT [40], HoG [41] are composed of filterbanks (oriented edge detectors at multiple scales) followed by non-linearities (winner take all) and pooling (histogramming).”
MethodReducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · cited 1×
“The innovation in [5], [1] was to simplify the architecture and to use the back-propagation algorithm to train the entire system in a supervised fashion.”
MethodSelf-citationBeyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916 · cited 4×
“A Pyramid Match Kernel (PMK) SVM [39] classifer can also be seen as another layer of feature extraction since it performs a K-means based feature extraction followed by local histogramming.”
MethodInfluentialGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841 · cited 1×
“Recent research in deep learning [30], [31], [32] has shown that unsupervised learning can be used to train each stage one after the other using only unlabeled data, reducing the requirement for labeled samples significantly.”
Background(1989) · neural information processing systems · Cited by 3,736
Self-citationSparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642
Best practices for convolutional neural networks applied to visual document analysis
Simard, P. Y., Steinkraus, D., & Platt, J. C. (2005) · Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings. · doi:10.1109/icdar.2003.1227801 · Cited by 2,741
Lee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553453 · Cited by 2,412 · cited 1×
“In [34], good accuracy was obtained on the”
MethodWhat is the best multi-stage architecture for object recognition?
Jarrett, K., Kavukcuoglu, K., Ranzato, M. A., & LeCun, Y. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459469 · Cited by 2,153 · cited 2×
“In [33], using abs and normalization non-linearities,”
BackgroundSelf-citationA Tutorial on Energy-Based Learning
LeCun, Y., Chopra, S., Hadsell, R. T., Ranzato, A., & Huang, F. J. (2006) · Cited by 1,931
Self-citationLeCun, Y., Bottou, L., Orr, G., & Müller, K. (1998) · neural information processing systems · Cited by 1,333
Self-citationLeCun, Y., Bottou, L., Orr, G. B., & Müller, K. R. (1998) · Lecture Notes in Computer Science · doi:10.1007/3-540-49430-8_2 · Cited by 919
Self-citation(2008) · Cited by 838
Object Recognition with Features Inspired by Visual Cortex
Serre, T., Wolf, L., & Poggio, T. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.254 · Cited by 821 · cited 3×
“Stunning results were obtained at MIT for reconstructing neuronal circuits from an stack of brain slice images a few nanometer thick.”
MethodSparse Feature Learning for Deep Belief Networks
Ranzato, M., Boureau, Y.-L., & LeCun, Y. (2007) · neural information processing systems · Cited by 744
Self-citationFukushima, K., & Miyake, S. (1982) · Pattern Recognition · doi:10.1016/0031-3203(82)90024-3 · Cited by 742 · cited 1×
“The first such model to be simulated on a computer was Fukushima’s Neocognitron [9], which used a layer-wise, unsupervised competitive learning algorithm for the filter banks, and a separately-trained supervised linear classifier for the”
MethodConvolutional face finder: a neural architecture for fast and robust face detection
Garcia, C., & Delakis, M. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.97 · Cited by 547 · cited 2×
“France Télécom has deployed ConvNetbased face detection systems for video-conference and other systems [15].”
MethodWhy is Real-World Visual Object Recognition Hard?
Pinto, N., Cox, D. D., & DiCarlo, J. J. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0040027 · Cited by 537 · cited 2×
“[4] have identified the most appropriate non-linearities and normaliza-”
Background(2006) · Cited by 490
Multiclass Object Recognition with Sparse, Localized Features
Mutch, J., & Lowe, D. G. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.200 · Cited by 453 · cited 1×
“Poggio’s lab at MIT [28], [29], which uses hard-wired Gabor filters in the first stage, and a simple”
Method(2005) · neural information processing systems · Cited by 446
Self-citationDeep learning via semi-supervised embedding
Weston, J., Ratle, F., & Collobert, R. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390303 · Cited by 380 · cited 1×
“Several works at NEC have also shown that using auxiliary tasks [35], [36] helps regularizing the system and produces excellent performance.”
BackgroundCNP: An FPGA-based processor for Convolutional Networks
Farabet, C., Poulet, C., Han, J. Y., & LeCun, Y. (2009) · 2009 International Conference on Field Programmable Logic and Applications · doi:10.1109/fpl.2009.5272559 · Cited by 367 · cited 1×
“group [51] achieves 10GOPS for 7x7 kernels, with an architecture that implements entire ConvNets, including pre/postprocessing, and is entirely programmable.”
BackgroundSelf-citationSynergistic Face Detection and Pose Estimation with Energy-Based Models
Osadchy, M., LeCun, Y., & Miller, M. L. (2007) · Journal of Machine Learning Research · doi:10.5555/1248659.1248700 · Cited by 348
Self-citationSerrano-Gotarredona, R., Oster, M., Lichtsteiner, P., Linares-Barranco, A., Paz-Vicente, R., Gomez-Rodriguez, F., Camunas-Mesa, L., Berner, R., Rivas-Perez, M., Delbruck, T., et al. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2009.2023653 · Cited by 325 · cited 1×
“CAVIAR is the leading such project, with a performance of 12G connections/sec [49].”
BackgroundLearning long‐range vision for autonomous off‐road driving
Hadsell, R., Sermanet, P., Ben, J., Erkan, A., Scoffier, M., Kavukcuoglu, K., Muller, U., & LeCun, Y. (2009) · Journal of Field Robotics · doi:10.1002/rob.20276 · Cited by 314 · cited 2×
“on vision-based navigation for off-road robots used ConvNets for long-range obstacle detection [22], [23].”
MethodSelf-citationOriginal approach for the localisation of objects in images
Vaillant, R. (1994) · IEE Proceedings - Vision, Image, and Signal Processing · doi:10.1049/ip-vis:19941301 · Cited by 280 · cited 1×
“Supervised ConvNets have also been used for object detection in images, including faces with record accuracy and real-time performance [14], [15], [16], [17], Google recently deployed a ConvNet to detect”
MethodLearning invariant features through topographic filter maps
Kavukcuoglu, K., Ranzato, M., Fergus, R., & LeCun, Y. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206545 · Cited by 274 · cited 1×
“the pooling also pools similar feature at the same location, in addition to the same feature at nearby locations [7].”
BackgroundSelf-citationThe Perception of Multiple Objects: A Connectionist Approach
Mozer, M. C. (1991) · Cited by 266
Toward automatic phenotyping of developing embryos from videos
Ning, F., Delhomme, D., LeCun, Y., Piano, F., Bottou, L., & Barbano, P. E. (2005) · IEEE Transactions on Image Processing · doi:10.1109/tip.2005.852470 · Cited by 261 · cited 1×
“Interesting new applications include image restoration [24] and image segmentation, particularly for biological images [25].”
BackgroundSelf-citationSupervised Learning of Image Restoration with Convolutional Networks
Jain, V., Murray, J. F., Roth, F., Turaga, S., Zhigulin, V., Briggman, K. L., Helmstaedter, M. N., Denk, W., & Seung, H. S. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408909 · Cited by 217
Synergistic Face Detection and Pose Estimation with Energy-Based Models
Osadchy, M., Le Cun, Y., & Miller, M. L. (2006) · Lecture Notes in Computer Science · doi:10.1007/11957959_10 · Cited by 209
Self-citationThe Perception of Multiple Objects: A Connectionist Approach
Mozer, M. C. (1991) · Medical Entomology and Zoology · Cited by 208
Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition
Kavukcuoglu, K., Ranzato, M., & LeCun, Y. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1010.3467 · Cited by 183 · cited 2×
“The unsupervised method we propose, to learn the filter coefficients in the filter bank layers, is called Predictive Sparse Decomposition (PSD) [37].”
MethodSelf-citationAhmed, A. A., Yu, K., Xu, W. H., Gong, Y., & Xing, E. P. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88690-7_6 · Cited by 179 · cited 1×
“Several works at NEC have also shown that using auxiliary tasks [35], [36] helps regularizing the system and produces excellent performance.”
BackgroundNonlinear image representation using divisive normalization
Lyu, S., & Simoncelli, E. P. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587821 · Cited by 153 · cited 1×
“layer is inspired by visual neuroscience models [3], [4].”
BackgroundAn analog neural network processor with programmable topology
Boser, B. E., Sackinger, E., Bromley, J., Le Cun, Y., & Jackel, L. D. (1991) · IEEE Journal of Solid-State Circuits · doi:10.1109/4.104196 · Cited by 142 · cited 1×
“operations per second [46], [47], with 4 bit resolution on the states and 6 bits on the weights.”
BackgroundSelf-citationApplication of the ANNA neural network chip to high-speed character recognition
Säckinger, E., Boser, B. E., Bromley, J. M., LeCun, Y., & Jackel, L. D. (1992) · IEEE Transactions on Neural Networks · doi:10.1109/72.129422 · Cited by 113 · cited 1×
“operations per second [46], [47], with 4 bit resolution on the states and 6 bits on the weights.”
BackgroundSelf-citationA Convolutional Neural Network Hand Tracker
Nowlan, S. J., & Platt, J. C. (1994) · Cited by 90
Face Detection Using GPU-Based Convolutional Neural Networks
Nasse, F., Thurau, C., & Fink, G. A. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-03767-2_10 · Cited by 71 · cited 1×
“Supervised ConvNets have also been used for object detection in images, including faces with record accuracy and real-time performance [14], [15], [16], [17], Google recently deployed a ConvNet to detect”
MethodAn FPGA-based stream processor for embedded real-time vision with Convolutional Networks
Farabet, C., Poulet, C., & LeCun, Y. (2009) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2009.5457611 · Cited by 65 · cited 1×
“An actual face detection application was demonstrated on this system, achieving 10fps on VGA images [52].”
MethodSelf-citationTEXT DETECTION WITH CONVOLUTIONAL NEURAL NETWORKS
Delakis, M., & García, C. (2008) · International Conference on Computer Vision Theory and Applications · doi:10.5220/0001079902900294 · Cited by 60 · cited 1×
“Other experimental detection applications include hands/gesture [19], logos and text [20].”
BackgroundVIP: an FPGA-based processor for image processing and neural networks
Cloutier, J., Cosatto, E., Pigeon, S., Boyer, F. R., & Simard, P. Y. (2002) · International Conference on Microelectronics for Neural Networks and Fuzzy Systems (MicroNeuro) · doi:10.1109/mnnfs.1996.493811 · Cited by 52 · cited 1×
“FPGA implementations of ConvNets appeared in the mid-90’s with [50], which used low-accuracy arithmetic to”
MethodTwo-Tier Approach for Arabic Offline Handwriting Recognition
Abdulkader, A. (2006) · HAL · Cited by 29
EBLearn: Open-Source Energy-Based Learning in C++
Serman, P., Kavukcuoglu, K., & LeCun, Y. (2009) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai.2009.28 · Cited by 27 · cited 1×
“The second one is EBlearn, a C++ machine learning library with class hierarchy to the Lush implementation [44].”
BackgroundSelf-citationOptimally combining a cascade of classifiers
Chellapilla, K., Shilman, M., & Simard, P. (2006) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.643669 · Cited by 20 · cited 1×
“This motivated Microsoft to deploy ConvNets in a number of OCR and handwriting recognition systems [6], [10], [11] including for Arabic [12]”
MethodA New Radical Based Approach to Offline Handwritten East-Asian Character Recognition
Chellapilla, K., & Simard, P. (2006) · HAL · Cited by 19
Projection-Field-Type VLSI Convolutional Neural Networks Using Merged/Mixed Analog-Digital Approach
Nomura, O., & Morie, T. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-69158-7_111 · Cited by 9 · cited 1×
“More recently, a group from the Canon corporation developed a prototype ConvNet chip for low-power intelligent cameras [48].”
BackgroundUsing Learned Features from 3D Data for Robot Navigation
Happold, M., & Ollis, M. (2007) · Studies in computational intelligence · doi:10.1007/978-3-540-73424-6_8 · Cited by 5 · cited 1×
“on vision-based navigation for off-road robots used ConvNets for long-range obstacle detection [22], [23].”
Method
Research works printed but not linked: 4
- lecun, 0, Handwritten digit recognition with a backpropagation network, NIPS'89
- frome, 0, Large-scale privacy protection in street-level imagery, ICCV'09
- collobert, 2008, Torch, Workshop on Machine Learning Open Source Software NIPS
- lecun, 2002, The Lush Reference Manual
Cited by
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
“Traditionally, the neighborhoods summarized by adjacent pooling units do not overlap (e.g., [17, 11, 4]).”
DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks
Moosavi-Dezfooli, S.-M., Fawzi, A., & Frossard, P. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.282 · Cited by 5,317
“However, these same architectures have been shown to be unstable to small, well sought, perturbations of the images.”
BackgroundOn Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215
“a form of structured sparsity is orthogonal to the sparsity that occurs from network pruning as described in Section VII-B2. Accordingly, a convolutional neural network (CNN) is a popular form of DNN [35]. 1) Convolutional Neural Networks (CNNs): CNNs are composed of multiple convolutional layers (CONV), as shown in F…”
BackgroundGeometric Deep Learning: Going beyond Euclidean data
Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A., & Vandergheynst, P. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2693418 · Cited by 3,661
“and computer vision [5], [6], [7], [8], [9], [10], [11] (the reader is referred to [12], [13] for many additional examples of successful applications of deep learning).”
BackgroundSelf-citation
Metadata
- Authors
- LeCun, Y., Kavukcuoglu, K., & Farabet, C.
- Affiliations
- Courant Institute of Mathematical Sciences (US)
- ORCID
- LeCun, Yann
- Venue
- Proceedings of 2010 IEEE International Symposium on Circuits and Systems
- Publisher
- IEEE
- Published
- 2010-05
- Pages
- 253-256
- ISSN
- 0271-4302 (ISSN-L)
- OpenAlex
- W2015861736
- Semantic Scholar
- c43025c429b1fbf6f1379f61801a1b40834d62e7
- Cited by
- 2,213
- References
- 56
- Citations by year
- 2026: 30 · 2025: 112 · 2024: 168 · 2023: 201 · 2022: 223 · 2021: 242 · 2020: 255 · 2019: 240 · 2018: 206 · 2017: 183 · 2016: 142 · 2015: 94 · 2014: 59 · 2013: 21 · 2012: 21
- FWCI
- 15.63
- Type
- conference paper
- Language
- en
- Open access
- green
- hal.science
- Licence
- other-oa
- Locations
- IEEE International Symposium on Circuits and Systems proceedings (publishedVersion)
- HAL (submittedVersion, other-oa)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- convolutional neural networks, unsupervised learning, representation learning, hierarchical feature representation, filter banks, feature pooling, nonlinear activation functions, visual object recognition, visual navigation, mobile robots, few labeled samples
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.