Beyond core object recognition: Recurrent processes account for object recognition under occlusion
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Abstract Core object recognition, the ability to rapidly recognize objects despite variations in their appearance, is largely solved through the feedforward processing of visual information. Deep neural networks are shown to achieve human-level performance in these tasks, and explain the primate brain representation. On the other hand, object recognition under more challenging conditions (i.e. beyond the core recognition problem) is less characterized. One such example is object recognition under occlusion. It is unclear to what extent feedforward and recurrent processes contribute in object recognition under occlusion. Furthermore, we do not know whether the conventional deep neural networks, such as AlexNet, which were shown to be successful in solving core object recognition, can perform similarly well in problems that go beyond the core recognition. Here, we characterize neural dynamics of object recognition under occlusion, using magnetoencephalography (MEG), while participants were presented with images of objects with various levels of occlusion. We provide evidence from multivariate analysis of MEG data, behavioral data, and computational modelling, demonstrating an essential role for recurrent processes in object recognition under occlusion. Furthermore, the computational model with local recurrent connections, used here, suggests a mechanistic explanation of how the human brain might be solving this problem. Author Summary In recent years, deep-learning-based computer vision algorithms have been able to achieve human-level performance in several object recognition tasks. This has also contributed in our understanding of how our brain may be solving these recognition tasks. However, object recognition under more challenging conditions, such as occlusion, is less characterized. Temporal dynamics of object recognition under occlusion is largely unknown in the human brain. Furthermore, we do not know if the previously successful deep-learning algorithms can similarly achieve human-level performance in these more challenging object recognition tasks. By linking brain data with behavior, and computational modeling, we characterized temporal dynamics of object recognition under occlusion, and proposed a computational mechanism that explains both behavioral and the neural data in humans. This provides a plausible mechanistic explanation for how our brain might be solving object recognition under more challenging conditions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665 · cited 2×
“Specifically, residual learning allows for a much deeper neural network with hundreds [58] and even thousands [59] of the layers providing better performance.”
BackgroundNeocortex size as a constraint on group size in primates
Dunbar, R. I. M. (1992) · Journal of Human Evolution · doi:10.1016/0047-2484(92)90081-j · Cited by 2,542
The distinct modes of vision offered by feedforward and recurrent processing
Lamme, V. A. F., & Roelfsema, P. R. (2000) · Trends in Neurosciences · doi:10.1016/s0166-2236(00)01657-x · Cited by 2,338 · cited 3×
“macaque) studies [4] controlling for the fact that the macaque cortex is smaller, with a shorter neural distance and therefore faster transmission of visual information [37].”
BackgroundPerformance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924
A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients
Portilla, J., & Simoncelli, E. P. (2000) · International Journal of Computer Vision · doi:10.1023/a:1026553619983 · Cited by 1,806
How does the brain solve visual object recognition?
DiCarlo, J. J., Zoccolan, D., & Rust, N. C. (2012) · Neuron · doi:10.1016/j.neuron.2012.01.010 · Cited by 1,790
Top-down influences on visual processing
Gilbert, C. D., & Li, W. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3476 · Cited by 1,224
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204
Self-citationGuclu, U., & van Gerven, M. A. J. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.5023-14.2015 · Cited by 961
Cadieu, C. F., Hong, H., Yamins, D. L. K., Pinto, N., Ardila, D., Solomon, E. A., Majaj, N. J., & DiCarlo, J. J. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003963 · Cited by 824 · cited 1×
“We used a well-known CNN (AlexNet) [101] that is shown to account for the core object recognition [10, 12, 74, 75].”
MethodCichy, R. M., Khosla, A., Pantazis, D., Torralba, A., & Oliva, A. (2016) · Scientific Reports · doi:10.1038/srep27755 · Cited by 716 · cited 1×
“stimulus onset [38, 43, 45, 49, 64], largely associated with the feedforward path of visual information processing [4, 10–12].”
BackgroundTop-down signal from prefrontal cortex in executive control of memory retrieval
Tomita, H., Ohbayashi, M., Nakahara, K., Hasegawa, I., & Miyashita, Y. (1999) · Nature · doi:10.1038/44372 · Cited by 615
Masking Disrupts Reentrant Processing in Human Visual Cortex
Fahrenfort, J. J., Scholte, H. S., & Lamme, V. A. F. (2007) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2007.19.9.1488 · Cited by 344 · cited 2×
“A number of other studies, however, suggest that the masking mainly interferes with the top-down feedback processes [4, 48, 53, 55].”
BackgroundSeeking Categories in the Brain
Thorpe, S. J., & Fabre-Thorpe, M. (2001) · Science · doi:10.1126/science.1058249 · Cited by 300
Evoked brain responses are generated by feedback loops
Garrido, M. I., Kilner, J. M., Kiebel, S. J., & Friston, K. J. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0706274105 · Cited by 296
Masking Interrupts Figure-Ground Signals in V1
Lamme, V. A. F., Zipser, K., & Spekreijse, H. (2002) · Journal of Cognitive Neuroscience · doi:10.1162/089892902320474490 · Cited by 285
The time course of visual processing: Backward masking and natural scene categorisation
Bacon-Macé, N., Macé, M. J.-M., Fabre-Thorpe, M., & Thorpe, S. J. (2005) · Vision Research · doi:10.1016/j.visres.2005.01.004 · Cited by 236
Why is Brain Size so Important:Design Problems and Solutions as Neocortex Gets Biggeror Smaller
Kaas, J. H. (2000) · Brain and Mind · doi:10.1023/a:1010028405318 · Cited by 225
Brain Mechanisms Underlying the Brief Maintenance of Seen and Unseen Sensory Information
King, J.-R., Pescetelli, N., & Dehaene, S. (2016) · Neuron · doi:10.1016/j.neuron.2016.10.051 · Cited by 206
Klink, P. C., Dagnino, B., Gariel-Mathis, M.-A., & Roelfsema, P. R. (2017) · Neuron · doi:10.1016/j.neuron.2017.05.033 · Cited by 135
Ban, H., Yamamoto, H., Hanakawa, T., Urayama, S.-I., Aso, T., Fukuyama, H., & Ejima, Y. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.1455-12.2013 · Cited by 80 · cited 1×
“[35] used topographic mapping with simple geometric shapes (e.”
Method
Cited by
Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
Lindsay, G. W. (2021) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01544 · Cited by 557
Deep Neural Networks as Scientific Models
Cichy, R. M., & Kaiser, D. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.01.009 · Cited by 454
Recurrence is required to capture the representational dynamics of the human visual system
Kietzmann, T. C., Spoerer, C. J., Sörensen, L. K. A., Cichy, R. M., Hauk, O., & Kriegeskorte, N. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1905544116 · Cited by 406
VITO-Transformer: A Visual-Tactile Fusion Network for Object Recognition
Li, B., Bai, J., Qiu, S., Wang, H., & Guo, Y. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3326241 · Cited by 39
Emergence of Visual Center-Periphery Spatial Organization in Deep Convolutional Neural Networks
Mohsenzadeh, Y., Mullin, C., Lahner, B., & Oliva, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-61409-0 · Cited by 31
Self-citation
Metadata
- Authors
- Rajaei, K., Mohsenzadeh, Y., Ebrahimpour, R., & Khaligh-Razavi, S.-M.
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2019-05-15
- Volume
- 15
- Issue
- 5
- Pages
- e1007001
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2953263739
- Cited by
- 114
- References
- 26
- FWCI
- 6.49
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.