The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Deep convolutional networks do not classify based on global object shape
Open accessAbstract
Deep convolutional networks (DCNNs) are achieving previously unseen performance in object classification, raising questions about whether DCNNs operate similarly to human vision.In biological vision, shape is arguably the most important cue for recognition.We tested the role of shape information in DCNNs trained to recognize objects.In Experiment 1, we presented a trained DCNN with object silhouettes that preserved overall shape but were filled with surface texture taken from other objects.Shape cues appeared to play some role in the classification of artifacts, but little or none for animals.In Experiments 2-4, DCNNs showed no ability to classify glass figurines or outlines but correctly classified some silhouettes.Aspects of these results led us to hypothesize that DCNNs do not distinguish object's bounding contours from other edges, and that DCNNs access some local shape features, but not global shape.In Experiment 5, we tested this hypothesis with displays that preserved local features but disrupted global shape, and vice versa.With disrupted global shape, which reduced human accuracy to 28%, DCNNs gave the same classification labels as with ordinary shapes.Conversely, local contour changes eliminated accurate DCNN classification but caused no difficulty for human observers.These results provide evidence that DCNNs have access to some local shape information in the form of local edge relations, but they have no access to global object shapes. Author summary"Deep learning" systems-specifically, deep convolutional neural networks (DCNNs)have recently achieved near human levels of performance in object recognition tasks.It has been suggested that the processing in these systems may model or explain object perception abilities in biological vision.For humans, shape is the most important cue for recognizing objects.We tested whether deep convolutional neural networks trained to recognize objects make use of object shape.Our findings indicate that other cues, such as surface texture, play a larger role in deep network classification than in human recognition.Most crucially, we show that deep learning systems have no sensitivity to the overall shape of an object.Whereas deep learning systems can access some local shape features,
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
I.—COMPUTING MACHINERY AND INTELLIGENCE
TURING, A. M. (1950) · Mind · doi:10.1093/mind/lix.236.433 · Cited by 10,290
Computing Machinery and Intelligence (1950)
Turing, A. (2004) · The Essential Turing · doi:10.1093/oso/9780198250791.003.0017 · Cited by 6,118
Marr, D. (1982) · Cumulative Index of Computer Aided Architectural Design · Cited by 5,733
Recognition-by-components: A theory of human image understanding.
Biederman, I. (1987) · Psychological Review · doi:10.1037/0033-295x.94.2.115 · Cited by 5,621
Snodgrass, J. G., & Vanderwart, M. (1980) · Journal of Experimental Psychology Human Learning & Memory · doi:10.1037//0278-7393.6.2.174 · Cited by 5,615
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A., Yosinski, J., & Clune, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298640 · Cited by 3,258
Some informational aspects of visual perception.
Attneave, F. (1954) · Psychological Review · doi:10.1037/h0054663 · Cited by 2,890
(1999) · Cited by 2,243
Representation and recognition of the spatial organization of three-dimensional shapes
Marr, D., & Nishihara, H. K. (1978) · Proceedings of the Royal Society of London. Series B. Biological Sciences · doi:10.1098/rspb.1978.0020 · Cited by 2,158
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924
Laws of organization in perceptual forms.
Wertheimer, M. (1938) · A source book of Gestalt psychology. · doi:10.1037/11496-005 · Cited by 1,493
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204
Rossion, B., & Pourtois, G. (2004) · Perception · doi:10.1068/p5117 · Cited by 971
Guclu, U., & van Gerven, M. A. J. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.5023-14.2015 · Cited by 961
Vision Science: Photons to Phenomenology
Lindsey, D. T. (2000) · Optometry and Vision Science · doi:10.1097/00006324-200005000-00008 · Cited by 855
Cadieu, C. F., Hong, H., Yamins, D. L. K., Pinto, N., Ardila, D., Solomon, E. A., Majaj, N. J., & DiCarlo, J. J. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003963 · Cited by 824
Cichy, R. M., Khosla, A., Pantazis, D., Torralba, A., & Oliva, A. (2016) · Scientific Reports · doi:10.1038/srep27755 · Cited by 716
A theory of visual interpolation in object perception
Kellman, P. J., & Shipley, T. F. (1991) · Cognitive Psychology · doi:10.1016/0010-0285(91)90009-d · Cited by 710
Self-citationSurface versus edge-based determinants of visual recognition
Biederman, I., & Ju, G. (1988) · Cognitive Psychology · doi:10.1016/0010-0285(88)90024-2 · Cited by 657
Shape Representation in Area V4: Position-Specific Tuning for Boundary Conformation
Pasupathy, A., & Connor, C. E. (2001) · Journal of Neurophysiology · doi:10.1152/jn.2001.86.5.2505 · Cited by 483
Estimating Receptive Field Size from fMRI Data in Human Striate and Extrastriate Visual Cortex
Smith, A. T. (2001) · Cerebral Cortex · doi:10.1093/cercor/11.12.1182 · Cited by 357
Visual Turing test for computer vision systems
Geman, D., Geman, S., Hallonquist, N., & Younes, L. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1422953112 · Cited by 321
Pictorial Recognition as an Unlearned Ability: A Study of One Child's Performance
Hochberg, J., & Brooks, V. (1962) · The American Journal of Psychology · doi:10.2307/1420286 · Cited by 309
Deep Neural Networks as a Computational Model for Human Shape Sensitivity
Kubilius, J., Bracci, S., & Op de Beeck, H. P. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004896 · Cited by 295
Color diagnosticity in object recognition
Tanaka, J. W., & Presnell, L. M. (1999) · Perception & Psychophysics · doi:10.3758/bf03207619 · Cited by 282
Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet
Kümmerer, M., Theis, L., & Bethge, M. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.1045 · Cited by 231
Perceptual Learning, Cognition, and Expertise
(2013) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/b978-0-12-407237-4.00004-9 · Cited by 164
Cue dynamics underlying rapid detection of animals in natural scenes
Elder, J. H., & Velisavljevic, L. (2009) · Journal of Vision · doi:10.1167/9.7.7 · Cited by 158
What Makes an Object Memorable?
Dubey, R., Peterson, J. C., Khosla, A., Yang, M., & Ghanem, B. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.130 · Cited by 140
Outline shape is a mediator of object recognition that is particularly important for living things
Lloyd‐Jones, T. J., & Luckhurst, L. (2002) · Memory & Cognition · doi:10.3758/bf03194950 · Cited by 88
Abstract shape representation in human visual perception.
Baker, N., & Kellman, P. J. (2018) · Journal of Experimental Psychology General · doi:10.1037/xge0000409 · Cited by 78
Self-citationTuring Test as a Defining Feature of AI-Completeness
Yampolskiy, R. V. (2012) · Studies in computational intelligence · doi:10.1007/978-3-642-29694-9_1 · Cited by 74
Kellman, P. J., Garrigan, P., & Erlikhman, G. (2013) · Advances in computer vision and pattern recognition · doi:10.1007/978-1-4471-5195-1_18 · Cited by 9
Self-citationIthapu, V. K. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.216 · Cited by 7
Comparing the brainʼs representation of shape to that of a deep convolutional neural network
Pospisil, D. A., Pasupathy, A., & Bair, W. (2016) · International Conference on Bio-Inspired Models of Network, Information and Computing Systems (BIONETICS) · doi:10.4108/eai.3-12-2015.2262486 · Cited by 6
Cited by
Shortcut learning in deep neural networks
Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00257-z · Cited by 2,224
Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
Lindsay, G. W. (2021) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01544 · Cited by 557
Reducing Domain Gap by Reducing Style Bias
Nam, H., Lee, H., Park, J., Yoon, W., & Yoo, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00858 · Cited by 353
MPViT: Multi-Path Vision Transformer for Dense Prediction
Lee, Y., Kim, J., Willette, J., & Hwang, S. J. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00714 · Cited by 351
Deep Learning: The Good, the Bad, and the Ugly
Serre, T. (2019) · Annual Review of Vision Science · doi:10.1146/annurev-vision-091718-014951 · Cited by 267
Metadata
- Authors
- Baker, N., Lu, H., Erlikhman, G., & Kellman, P. J.
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2018-12-07
- Volume
- 14
- Issue
- 12
- Pages
- e1006613
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2903867357
- Cited by
- 345
- References
- 39
- FWCI
- 14.46
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.