Deep Networks Can Resemble Human Feed-forward Vision in Invariant Object Recognition
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Deep convolutional neural networks (DCNNs) have attracted much attention recently, and have shown to be able to recognize thousands of object categories in natural image databases. Their architecture is somewhat similar to that of the human visual system: both use restricted receptive fields, and a hierarchy of layers which progressively extract more and more abstracted features. Yet it is unknown whether DCNNs match human performance at the task of view-invariant object recognition, whether they make similar errors and use similar representations for this task, and whether the answers depend on the magnitude of the viewpoint variations. To investigate these issues, we benchmarked eight state-of-the-art DCNNs, the HMAX model, and a baseline shallow model and compared their results to those of humans with backward masking. Unlike in all previous DCNN studies, we carefully controlled the magnitude of the viewpoint variations to demonstrate that shallow nets can outperform deep nets and humans when variations are weak. When facing larger variations, however, more layers were needed to match human performance and error distributions, and to have representations that are consistent with human behavior. A very deep net with 18 layers even outperformed humans at the highest variation level, using the most human-like representations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Fukushima, K. (1980) · Biological Cybernetics · doi:10.1007/bf00344251 · Cited by 5,759 · cited 1×
“Early models were only comprised of a few layers [8, 9, 10, 11, 12], while the new generation, called “deep convolutional neural networks” (DCNNs) contain many layers (8 and above).”
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019 · cited 1×
“To our interest, recent studies also confirmed the abilities of DCNNs in object recognition problems (e.g. refs 15, 27 and 61).”
Speed of processing in the human visual system
Thorpe, S., Fize, D., & Marlot, C. (1996) · Nature · doi:10.1038/381520a0 · Cited by 3,538
OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6229 · Cited by 2,680
Lee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553453 · Cited by 2,412
The distinct modes of vision offered by feedforward and recurrent processing
Lamme, V. A. F., & Roelfsema, P. R. (2000) · Trends in Neurosciences · doi:10.1016/s0166-2236(00)01657-x · Cited by 2,338
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924
How does the brain solve visual object recognition?
DiCarlo, J. J., Zoccolan, D., & Rust, N. C. (2012) · Neuron · doi:10.1016/j.neuron.2012.01.010 · Cited by 1,790 · cited 3×
“It is believed that the primate visual system solves the task through hierarchical processing along the ventral stream of the visual cortex [1].”
Building high-level features using large scale unsupervised learning
Le, Q. V. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639343 · Cited by 1,695 · cited 1×
“DCNNs have been shown to be a novel and powerful approach to tackle this problem [15, 54, 55, 16, 56, 57, 58, 27, 59, 60].”
Going Deeper with Convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2014) · arXiv · doi:10.48550/arxiv.1409.4842 · Cited by 1,461
Top-down influences on visual processing
Gilbert, C. D., & Li, W. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3476 · Cited by 1,224 · cited 1×
“In addition to object recognition, feedback connections plays a pivotal role in other visual processes such as figure-ground segregation [68, 69], spatial and feature-based attention [70], and perceptual learning [71].”
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204
The role of context in object recognition
Oliva, A., & Torralba, A. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.09.009 · Cited by 1,063
Guclu, U., & van Gerven, M. A. J. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.5023-14.2015 · Cited by 961
Fast Readout of Object Identity from Macaque Inferior Temporal Cortex
Hung, C. P., Kreiman, G., Poggio, T., & DiCarlo, J. J. (2005) · Science · doi:10.1126/science.1117593 · Cited by 854
Cadieu, C. F., Hong, H., Yamins, D. L. K., Pinto, N., Ardila, D., Solomon, E. A., Majaj, N. J., & DiCarlo, J. J. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003963 · Cited by 824 · cited 9×
“[27] compared the responses of monkey IT and V4 neurons with the penultimate layer of three DCNNs, but they did not tested, for example, how V4 responses are correlated to other layers.”
InfluentialFreiwald, W. A., & Tsao, D. Y. (2010) · Science · doi:10.1126/science.1194908 · Cited by 678
Masking Interrupts Figure-Ground Signals in V1
Lamme, V. A. F., Zipser, K., & Spekreijse, H. (2002) · Journal of Cognitive Neuroscience · doi:10.1162/089892902320474490 · Cited by 285
Figure—Ground Segregation in a Recurrent Network Architecture
Roelfsema, P. R., Lamme, V. A. F., Spekreijse, H., & Bosch, H. (2002) · Journal of Cognitive Neuroscience · doi:10.1162/08989290260045756 · Cited by 254 · cited 2×
“In addition to object recognition, feedback connections plays a pivotal role in other visual processes such as figure-ground segregation 68,69 , spatial and feature-based attention 70 , and perceptual learning 71 .”
Cited by
Deep learning in spiking neural networks
Tavanaei, A., Ghodrati, M., Kheradpisheh, S. R., Masquelier, T., & Maida, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2018.12.002 · Cited by 1,386
Self-citationSTDP-based spiking deep convolutional neural networks for object recognition
Kheradpisheh, S. R., Ganjtabesh, M., Thorpe, S. J., & Masquelier, T. (2018) · Neural Networks · doi:10.1016/j.neunet.2017.12.005 · Cited by 768
Self-citationConvolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
Lindsay, G. W. (2021) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01544 · Cited by 557
Rajalingham, R., Issa, E. B., Bashivan, P., Kar, K., Schmidt, K., & DiCarlo, J. J. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.0388-18.2018 · Cited by 358
Deep Learning: The Good, the Bad, and the Ugly
Serre, T. (2019) · Annual Review of Vision Science · doi:10.1146/annurev-vision-091718-014951 · Cited by 267
Metadata
- Authors
- Kheradpisheh, S. R., Ghodrati, M., Ganjtabesh, M., & Masquelier, T.
- Venue
- Scientific Reports
- Published
- 2016-09-07
- Volume
- 6
- Issue
- 1
- Pages
- 32672-32672
- ISSN
- 2045-2322 (ISSN-L)
- OpenAlex
- W1821584645
- Cited by
- 134
- References
- 20
- FWCI
- 12.09
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.