What is the best multi-stage architecture for object recognition?
Open accessThe DOI matches its Crossref record; 56 of the 57 references it cites were found in a registry or an index.
Abstract
In many recent object recognition systems, feature extraction stages are generally composed of a filter bank, a non-linear transformation, and some sort of feature pooling layer. Most systems use only one stage of feature extraction in which the filters are hard-wired, or two stages where the filters in one or both stages are learned in supervised or unsupervised mode. This paper addresses three questions: 1. How does the non-linearities that follow the filter banks influence the recognition accuracy? 2. does learning the filter banks in an unsupervised or supervised manner improve the performance over random filters or hardwired filters? 3. Is there any advantage to using an architecture with two stages of feature extraction, rather than one? We show that using non-linearities that include rectification and local contrast normalization is the single most important ingredient for good accuracy on object recognition benchmarks. We show that two stages of feature extraction yield better accuracy than one. Most surprisingly, we show that a two-stage system with random filters can yield almost 63% recognition rate on Caltech-101, provided that the proper non-linearities and pooling layers are used. Finally, we show that with supervised refinement, the system achieves state-of-the-art performance on NORB dataset (5.6%) and unsupervised pre-training followed by supervised refinement produces good accuracy on Caltech-101 (> 65%), and the lowest known error rate on the undistorted, unprocessed MNIST dataset (0.53%).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 6×
“Results are presented on the well-known Caltech-101 dataset [7], on the NORB object dataset [15], and on the MNIST dataset of handwritten digits [14].”
BackgroundMethodInfluentialSelf-citationDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · cited 10×
“SIFT uses a rectification followed by winner-take-all operation over orientation , which is an extreme form of normalization.”
MethodInfluentialHistograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584 · cited 3×
“Particular embodiments of the single-stage systems use SIFT features [19, 13], HoG [6], Geometric Blur [5], and models inspired by the architecture of the mammalian primary visual cortex [24], to mention a few.”
MethodReducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Harrison's Principles of Internal Medicine.
(1988) · Annals of Internal Medicine · doi:10.7326/0003-4819-108-3-499_2 · Cited by 12,514
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916 · cited 12×
“The performance of our two-stage system is similar to that of the best single-stage systems based on SIFT and PMK-SVM, perhaps because the PM Kernel is conceptually similar to our feature extractions stage.”
BackgroundMethodInfluentialGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Sparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642
Linear spatial pyramid matching using sparse coding for image classification
Yang, J., Yu, K., Gong, Y., & Huang, T. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206757 · Cited by 2,828
Fei-Fei, L., Fergus, R., & Perona, P. (2005) · 2004 Conference on Computer Vision and Pattern Recognition Workshop · doi:10.1109/cvpr.2004.383 · Cited by 2,502
Lee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553453 · Cited by 2,412
Efficient sparse coding algorithms
Lee, H., Battle, A., Raina, R., & Ng, A. Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0105 · Cited by 2,394
Learning methods for generic object recognition with invariance to pose and lighting
LeCun, Y., Huang, F. J., & Bottou, L. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315150 · Cited by 1,324
Self-citationGradientBased Learning Applied to Document Recognition
(2009) · Intelligent Signal Processing · doi:10.1109/9780470544976.ch9 · Cited by 1,135 · cited 5×
“Results are presented on the well-known Caltech-101 dataset [7], on the NORB object dataset [15], and on the MNIST dataset of handwritten digits [14].”
BackgroundMethodInfluentialSVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition
Zhang, H., Berg, A. C., Maire, M., & Malik, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.301 · Cited by 1,114
Efficient Learning of Sparse Representations with an Energy-Based Model
Ranzato, M., Poultney, C., Chopra, S., & LeCun, Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0147 · Cited by 1,106
Self-citationUnsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition
Ranzato, M., Huang, F. J., Boureau, Y.-L., & LeCun, Y. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383157 · Cited by 1,086 · cited 7×
“Finally, one can train the filters in an unsupervised fashion by following the so-called “deep belief network” strategy [8, 4, 26, 9, 25, 17]: the filters are trained so that representations at one stage can be reconstructed from the representation at the next stage under sparsity constraints [25, 11] or using the so-…”
BackgroundMethodInfluentialSelf-citation(2007) · Cited by 849
Shape Matching and Object Recognition Using Low Distortion Correspondences
Berg, A. C., Berg, T. L., & Malik, J. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.320 · Cited by 845 · cited 2×
“Particular embodiments of the single-stage systems use SIFT features [19, 13], HoG [6], Geometric Blur [5], and models inspired by the architecture of the mammalian primary visual cortex [24], to mention a few.”
MethodTiollais, P., & Buendia, M.-A. (1991) · Scientific American · doi:10.1038/scientificamerican0491-116 · Cited by 830
Object Recognition with Features Inspired by Visual Cortex
Serre, T., Wolf, L., & Poggio, T. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.254 · Cited by 821 · cited 8×
“FCSG −PM This is also a typical building block of convolutional networks, as well as the basis of the HMAX and other architectures [28, 25], which alternate tanh-squashed filter banks with max-pooling layers.”
BackgroundMethodInfluentialDiscriminative learned dictionaries for local image analysis
Mairal, J., Bach, F., Ponce, J., Sapiro, G., & Zisserman, A. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587652 · Cited by 726 · cited 1×
“The main problem with the unsupervised approach is that the filters are learned independently of the task, although a few authors have proposed methods that combine unsupervised and supervised criteria to alleviate the problem [21, 27, 4].”
BackgroundCOVER STORY Guidelines for infection control in dental health care settings—2003
KOHN, W. G., HARTE, J. A., MALVITZ, D. M., COLLINS, A. S., CLEVELAND, J. L., & EKLUND, K. J. (2004) · The Journal of the American Dental Association · doi:10.14219/jada.archive.2004.0019 · Cited by 686
(2001) · PsycEXTRA Dataset · doi:10.1037/e548302006-001 · Cited by 564
Why is Real-World Visual Object Recognition Hard?
Pinto, N., Cox, D. D., & DiCarlo, J. J. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0040027 · Cited by 537 · cited 9×
“A common choice for the filter bank of the first stage is Gabor Wavelets [28, 22, 24].”
BackgroundMethodInfluentialLearning The Discriminative Power-Invariance Trade-Off
Varma, M., & Ray, D. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408875 · Cited by 520 · cited 2×
“While the best results on Caltech-101 have been obtained by combining a large number of different feature families [ 29 ], the present study concerns systems with a single feature family, hence results will be compared with other work in which a single feature family is used.”
MethodResultChronic hepatitis B virus infection in Asian countries
Merican, I., Guan, R., Amarapuka, D., Alexander, M., Chutaputti, A., Chien, R., Hasnian, S., Leung, N., Lesmana, L., Phiet, P., et al. (2000) · Journal of Gastroenterology and Hepatology · doi:10.1046/j.1440-1746.2000.0150121356.x · Cited by 505
Linear spatial pyramid matching using sparse coding for image classification
Yang, J., Yu, K., Gong, Y., & Huang, T. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvprw.2009.5206757 · Cited by 496
Multiclass Object Recognition with Sparse, Localized Features
Mutch, J., & Lowe, D. G. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.200 · Cited by 453 · cited 7×
“A common choice for the filter bank of the first stage is Gabor Wavelets [28, 22, 24].”
BackgroundMethodInfluentialHepatitis B: Evolving Epidemiology and Implications for Control
Margolis, H., Alter, M., & Hadler, S. (1991) · Seminars in Liver Disease · doi:10.1055/s-2008-1040427 · Cited by 430
Hepatitis B epidemiology in Asia, the Middle East and Africa
André, F. (2000) · Vaccine · doi:10.1016/s0264-410x(99)00456-9 · Cited by 348
Epidemiology of hepatitis B in Europe and worldwide
Alter, M. J. (2003) · Journal of Hepatology · doi:10.1016/s0168-8278(03)00141-7 · Cited by 338
THE EPIDEMIOLOGY OF VIRAL HEPATITIS IN THE UNITED STATES
Alter, M. J., & Mast, E. E. (1994) · Gastroenterology Clinics of North America · doi:10.1016/s0889-8553(21)00146-1 · Cited by 324
Large-scale Learning with SVM and Convolutional for Generic Object Categorization
Huang, F. J., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.164 · Cited by 308 · cited 4×
“FCSG −PA This is the basic building block of traditional convolutional networks, alternating tanh-squashed filter banks with average down-sampling layers [14, 10].”
BackgroundMethodInfluentialSelf-citationLearning invariant features through topographic filter maps
Kavukcuoglu, K., Ranzato, M., Fergus, R., & LeCun, Y. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206545 · Cited by 274
Self-citationUnsupervised Learning: Foundations of Neural Computation
(1999) · The MIT Press eBooks · doi:10.7551/mitpress/7011.001.0001 · Cited by 272
Unsupervised learning : foundations of neural computation
Hinton, G. E., & B, T. J. S. A. (1999) · Cited by 266
Strategies to prevent and control hepatitis B and C virus infections: a global perspective
Mast, E. E., Alter, M. J., & Margolis, H. S. (1999) · Vaccine · doi:10.1016/s0264-410x(98)00415-0 · Cited by 259
Semi-supervised learning of compact document representations with deep networks
Ranzato, M. A., & Szummer, M. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390256 · Cited by 217 · cited 1×
“The main problem with the unsupervised approach is that the filters are learned independently of the task, although a few authors have proposed methods that combine unsupervised and supervised criteria to alleviate the problem [21, 27, 4].”
BackgroundSelf-citationEASL INTERNATIONAL CONSENSUS CONFERENCE ON HEPATITIS B
Jury, E. (2003) · Cited by 214
Natural History and Clinical Consequences of Hepatitis B Virus Infection
Pan, C. Q., & Zhang, J. X. (2005) · International Journal of Medical Sciences · doi:10.7150/ijms.2.36 · Cited by 197
Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition
Kavukcuoglu, K., Ranzato, M., & LeCun, Y. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1010.3467 · Cited by 183
Self-citationAhmed, A. A., Yu, K., Xu, W. H., Gong, Y., & Xing, E. P. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88690-7_6 · Cited by 179
K-SVD and its non-negative variant for dictionary design
Aharon, M., Elad, M., & Bruckstein, A. M. (2005) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.613878 · Cited by 167 · cited 1×
“A system with two such stages trained in unsupervised mode, and the classifier (only) trained in supervised mode, as in [25], is denoted UU.”
MethodNonlinear image representation using divisive normalization
Lyu, S., & Simoncelli, E. P. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587821 · Cited by 146 · cited 2×
“The local contrast normalization layer is inspired by computational neuroscience models [24, 20].”
MethodMerat, S., Malekzadeh, R., Hashemi, R., & Khatibian, M. (2000) · Cited by 128
Epidemiology of hepatitis B in the Islamic Republic of Iran
Zali, M. R., Mohammad, K., Farhadi, A., Masjedi, M. R., Zargar, A., & Nowroozi, A. (2021) · Eastern Mediterranean Health Journal · doi:10.26719/1996.2.2.290 · Cited by 102
Ghavanini, A. A., & Sabri, M. R. (2000) · Eastern Mediterranean Health Journal · doi:10.26719/2000.6.5-6.1114 · Cited by 100
Epidemiology of viral hepatitis among Iranian population--a viral marker study.
Farzadegan, H., Shamszad, M., & Noori-Arya, K. (1980) · PubMed · Cited by 75
Seroprevalence of hepatitis B in Nahavand, Islamic Republic of Iran.
Alizadeh, A. H. M., Ranjbar, M., Ansari, S., Mirarab, A., Alavian, S. M., Mohammad, K., Adibi, P., Sadri, G., Keramat, F., Ardalan, A., et al. (2006) · PubMed · Cited by 74
Preliminary report of hepatitis B virus genotype prevalence in Iran.
Alavian, S. M., Keyvani, H., Rezai, M., Ashayeri, N., & Sadeghi, H. M. (2006) · Europe PMC (PubMed Central) · doi:10.3748/wjg.v12.i32.5211 · Cited by 71
Imamura, M., & Chayama, K. (2017) · Nihon Naika Gakkai Zasshi · doi:10.2169/naika.106.428 · Cited by 39
Lankarani, K. B., Firouzi, M. S., Nabipour, I., Fatahi, F., Mohammad, S. Y., & Malekzadeh, R. (1999) · Cited by 29
CAUSES OF CIRRHOSIS IN A SERIES OF PATIENTS AT A UNIVERSITY HOSPITAL IN TEHRAN
Azimi, K., Sarafi, M., Alavian, S. M., Alavi, M.-S., Golestan, S. H., Mikaeili, J., & Malekzadeh, R. (2002) · Cited by 14
Wong, E. S. (2004) · 42nd Annual Meeting · Cited by 13
Cited by
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Self-citationSegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
“For more details of these operations, see (Krizhevsky et al., 2012) and (Jarrett et al., 2009).”
Background
Metadata
- Authors
- Jarrett, K., Kavukcuoglu, K., Ranzato, M. A., & LeCun, Y.
- Affiliations
- Courant Institute of Mathematical Sciences (US)
- ORCID
- LeCun, Yann
- Venue
- 2009 IEEE 12th International Conference on Computer Vision
- Publisher
- IEEE
- Published
- 2009-09
- Pages
- 2146-2153
- OpenAlex
- W2546302380
- Semantic Scholar
- 1f88427d7aa8225e47f946ac41a0667d7b69ac52
- Cited by
- 2,153
- References
- 57
- Citations by year
- 2026: 20 · 2025: 48 · 2024: 82 · 2023: 114 · 2022: 124 · 2021: 194 · 2020: 207 · 2019: 205 · 2018: 211 · 2017: 210 · 2016: 198 · 2015: 162 · 2014: 137 · 2013: 86 · 2012: 69
- FWCI
- 42.18
- Type
- conference paper
- Language
- en
- Open access
- green
- hal.science
- Licence
- cc-by-nc
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- HAL (submittedVersion, cc-by-nc)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- object recognition, filter banks, rectification, feature pooling, unsupervised pre-training, Caltech-101, MNIST, image classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.