The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Why is Real-World Visual Object Recognition Hard?
Open accessAbstract
Progress in understanding the brain mechanisms underlying vision requires the construction of computational models that not only emulate the brain's anatomy and physiology, but ultimately match its performance on visual tasks. In recent years, "natural" images have become popular in the study of vision and have been used to show apparently impressive progress in building such models. Here, we challenge the use of uncontrolled "natural" images in guiding that progress. In particular, we show that a simple V1-like model--a neuroscientist's "null" model, which should perform poorly at real-world visual object recognition tasks--outperforms state-of-the-art object recognition systems (biologically inspired and otherwise) on a standard, ostensibly natural image recognition test. As a counterpoint, we designed a "simpler" recognition test to better span the real-world variation in object pose, position, and scale, and we show that this test correctly exposes the inadequacy of the V1-like model. Taken together, these results demonstrate that tests based on uncontrolled natural images can be seriously misleading, potentially guiding progress in the wrong direction. Instead, we reexamine what it means for images to be natural and argue for a renewed focus on the core problem of object recognition--real-world image variation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex
Hubel, D. H., & Wiesel, T. N. (1962) · The Journal of Physiology · doi:10.1113/jphysiol.1962.sp006837 · Cited by 13,680
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916
Receptive fields and functional architecture of monkey striate cortex
Hubel, D. H., & Wiesel, T. N. (1968) · The Journal of Physiology · doi:10.1113/jphysiol.1968.sp008455 · Cited by 6,568
LabelMe: A Database and Web-Based Tool for Image Annotation
Russell, B. C., Torralba, A., Murphy, K. P., & Freeman, W. T. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0090-8 · Cited by 4,210
Fei-Fei, L., Fergus, R., & Perona, P. (2005) · 2004 Conference on Computer Vision and Pattern Recognition Workshop · doi:10.1109/cvpr.2004.383 · Cited by 2,502
Natural Image Statistics and Neural Representation
Simoncelli, E. P., & Olshausen, B. A. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.1193 · Cited by 2,460
(2007) · CaltechAUTHORS (California Institute of Technology) · Cited by 2,274
The “independent components” of natural scenes are edge filters
Bell, A. J., & Sejnowski, T. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00121-1 · Cited by 2,201
Robust Object Recognition with Cortex-Like Mechanisms
Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., & Poggio, T. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.56 · Cited by 1,667
Jones, J. P., & Palmer, L. A. (1987) · Journal of Neurophysiology · doi:10.1152/jn.1987.58.6.1233 · Cited by 1,658 · cited 1×
“The responses of these cells to visual stimuli are welldescribed by a spatial linear filter, resembling a Gabor wavelet [29– 31], with a nonlinear output function (threshold and saturation) and some local normalization (roughly analogous to ‘‘contrast gain control’’).”
BackgroundLearning methods for generic object recognition with invariance to pose and lighting
LeCun, Y., Huang, F. J., & Bottou, L. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315150 · Cited by 1,324
SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition
Zhang, H., Berg, A. C., Maire, M., & Malik, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.301 · Cited by 1,114
Untangling invariant object recognition
DiCarlo, J. J., & Cox, D. D. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.06.010 · Cited by 959
Self-citationFast Readout of Object Identity from Macaque Inferior Temporal Cortex
Hung, C. P., Kreiman, G., Poggio, T., & DiCarlo, J. J. (2005) · Science · doi:10.1126/science.1117593 · Cited by 854
Self-citationUnsupervised Learning of Models for Recognition
Weber, M., Welling, M., & Perona, P. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45054-8_2 · Cited by 594
Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity
Masquelier, T., & Thorpe, S. J. (2007) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0030031 · Cited by 548
Learning The Discriminative Power-Invariance Trade-Off
Varma, M., & Ray, D. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408875 · Cited by 520
Peekaboom: a game for locating objects in images
von Ahn, L., Liu, R., & Blum, M. (2006) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124782 · Cited by 509
Multiclass Object Recognition with Sparse, Localized Features
Mutch, J., & Lowe, D. G. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.200 · Cited by 453
A natural approach to studying vision
Felsen, G., & Dan, Y. (2005) · Nature Neuroscience · doi:10.1038/nn1608 · Cited by 285
Rust, N. C., & Movshon, J. A. (2005) · Nature Neuroscience · doi:10.1038/nn1606 · Cited by 240
Dataset Issues in Object Recognition
Ponce, J., Berg, T. L., Everingham, M., Forsyth, D. A., Hebert, M., Lazebnik, S., Marszalek, M., Schmid, C., Russell, B. C., Torralba, A., et al. (2006) · Lecture Notes in Computer Science · doi:10.1007/11957959_2 · Cited by 204
Using Dependent Regions for Object Categorization in a Generative Framework
Wang, G., Zhang, Y., & Fei-Fei, L. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.324 · Cited by 141
Gallant, J. L., Connor, C. E., & Essen, D. C. V. (1998) · Neuroreport · doi:10.1097/00001756-199805110-00075 · Cited by 137
Streetscenes: towards scene understanding in still images
Poggio, T., & Bileschi, S. (2006) · Cited by 71
Arathorn, D. W. (2002) · Cited by 61
How do visual neurons respond in the real world?
Reinagel, P. (2001) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(00)00231-2 · Cited by 61
The duration of the attentional blink in natural scenes depends on stimulus category
Einhäuser, W., Koch, C., & Makeig, S. (2007) · Vision Research · doi:10.1016/j.visres.2006.12.007 · Cited by 53
Gallant, J. L., Connor, C. E., & Essen, D. C. V. (1998) · Neuroreport · doi:10.1097/00001756-199801050-00017 · Cited by 32
Pyramid Match Kernels: Discriminative Classification with Sets of Image Features (version 2)
Grauman, K. L., & Darrell, T. (2006) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 20
Cited by
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524
Convolutional networks and applications in vision
LeCun, Y., Kavukcuoglu, K., & Farabet, C. (2010) · Proceedings of 2010 IEEE International Symposium on Circuits and Systems · doi:10.1109/iscas.2010.5537907 · Cited by 2,213
What is the best multi-stage architecture for object recognition?
Jarrett, K., Kavukcuoglu, K., Ranzato, M. A., & LeCun, Y. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459469 · Cited by 2,153
Metadata
- Authors
- Pinto, N., Cox, D. D., & DiCarlo, J. J.
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2008-01-25
- Volume
- 4
- Issue
- 1
- Pages
- e27
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2026942141
- Cited by
- 537
- References
- 32
- FWCI
- 17.69
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.