The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Deep Neural Networks as a Computational Model for Human Shape Sensitivity
Open accessAbstract
Theories of object recognition agree that shape is of primordial importance, but there is no consensus about how shape might be represented, and so far attempts to implement a model of shape perception that would work with realistic stimuli have largely failed. Recent studies suggest that state-of-the-art convolutional 'deep' neural networks (DNNs) capture important aspects of human object perception. We hypothesized that these successes might be partially related to a human-like representation of object shape. Here we demonstrate that sensitivity for shape features, characteristic to human and primate vision, emerges in DNNs when trained for generic object recognition from natural photographs. We show that these models explain human shape judgments for several benchmark behavioral and neural stimulus sets on which earlier models mostly failed. In particular, although never explicitly trained for such stimuli, DNNs develop acute sensitivity to minute variations in shape and to non-accidental properties that have long been implicated to form the basis for object recognition. Even more strikingly, when tested with a challenging stimulus set in which shape and category membership are dissociated, the most complex model architectures capture human shape sensitivity as well as some aspects of the category structure that emerges from human judgments. As a whole, these results indicate that convolutional neural networks not only learn physically correct representations of object categories but also develop perceptually accurate representational spaces of shapes. An even more complete model of human object representations might be in sight by training deep architectures for multiple tasks, which is so characteristic in human development.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
(2017) · doi:10.1109/cvpr35066.2017 · Cited by 7,141
scikit-image: image processing in Python
(2014) · doi:10.7717/peerj.453 · Cited by 6,794
Recognition-by-components: A theory of human image understanding.
Biederman, I. (1987) · Psychological Review · doi:10.1037/0033-295x.94.2.115 · Cited by 5,621
Snodgrass, J. G., & Vanderwart, M. (1980) · Journal of Experimental Psychology Human Learning & Memory · doi:10.1037//0278-7393.6.2.174 · Cited by 5,615
Speed of processing in the human visual system
Thorpe, S., Fize, D., & Marlot, C. (1996) · Nature · doi:10.1038/381520a0 · Cited by 3,538
Representational similarity analysis-connecting the branches of systems neuroscience
Kriegeskorte, N. (2008) · Frontiers in Systems Neuroscience · doi:10.3389/neuro.06.004.2008 · Cited by 3,355
Hierarchical models of object recognition in cortex
Riesenhuber, M., & Poggio, T. (1999) · Nature Neuroscience · doi:10.1038/14819 · Cited by 3,309
NLTK: the Natural Language Toolkit
Loper, E., & Bird, S. (2002) · Proceedings of the ACL-02 Workshop on Effective tools and methodologies for teaching natural language processing and computational linguistics - · doi:10.3115/1118108.1118117 · Cited by 3,218
(2014) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 2,534
Invariant visual representation by single neurons in the human brain
Quiroga, R. Q., Reddy, L., Kreiman, G., Koch, C., & Fried, I. (2005) · Nature · doi:10.1038/nature03687 · Cited by 1,988
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924
Distortion invariant object recognition in the dynamic link architecture
Lades, M., Vorbruggen, J. C., Buhmann, J., Lange, J., von der Malsburg, C., Wurtz, R. P., & Konen, W. (1993) · IEEE Transactions on Computers · doi:10.1109/12.210173 · Cited by 1,805
Inferotemporal cortex and object vision
Tanaka, K. (1996) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.19.030196.000545 · Cited by 1,402
Representing shape with a spatial pyramid kernel
Bosch, A., Zisserman, A., & Munoz, X. (2007) · Proceedings of the 6th ACM international conference on Image and video retrieval · doi:10.1145/1282280.1282340 · Cited by 1,399
Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey
Kriegeskorte, N., Mur, M., Ruff, D. A., Kiani, R., Bodurka, J., Esteky, H., Tanaka, K., & Bandettini, P. A. (2008) · Neuron · doi:10.1016/j.neuron.2008.10.043 · Cited by 1,344
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204
Multi-scale Orderless Pooling of Deep Convolutional Activation Features
Gong, Y., Wang, L., Guo, R., & Lazebnik, S. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_26 · Cited by 1,030
Chalupa, L. M., Werner, J. S., & Barnstable, C. J. (2004) · Cited by 1,029
Perceptual organization and visual recognition
(1985) · Computer Vision, Graphics, and Image Processing · doi:10.1016/0734-189x(85)90041-6 · Cited by 1,022
Perceptual Organization and Visual Recognition
Lowe, D. G. (1985) · The Kluwer International Series in Engineering and Computer Science · doi:10.1007/978-1-4613-2551-2 · Cited by 982
Rossion, B., & Pourtois, G. (2004) · Perception · doi:10.1068/p5117 · Cited by 971
Guclu, U., & van Gerven, M. A. J. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.5023-14.2015 · Cited by 961
Untangling invariant object recognition
DiCarlo, J. J., & Cox, D. D. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.06.010 · Cited by 959
A Toolbox for Representational Similarity Analysis
Nili, H., Wingfield, C., Walther, A., Su, L., Marslen-Wilson, W., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003553 · Cited by 937
Dynamic binding in a neural network for shape recognition.
Hummel, J. E., & Biederman, I. (1992) · Psychological Review · doi:10.1037/0033-295x.99.3.480 · Cited by 924
A feedforward architecture accounts for rapid categorization
Serre, T., Oliva, A., & Poggio, T. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0700622104 · Cited by 918
NLTK: The Natural Language Toolkit
Bird, S. (2006) · Proceedings of the COLING/ACL on Interactive presentation sessions - · doi:10.3115/1225403.1225421 · Cited by 841
Rapid natural scene categorization in the near absence of attention
Li, F. F., VanRullen, R., Koch, C., & Perona, P. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.092277599 · Cited by 833
Biederman, I., & Gerhardstein, P. C. (1993) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037//0096-1523.19.6.1162 · Cited by 601
Do We Know What the Early Visual System Does?
Carandini, M., Demb, J. B., Mante, V., Tolhurst, D. J., Dan, Y., Olshausen, B. A., Gallant, J. L., & Rust, N. C. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.3726-05.2005 · Cited by 600
A century of Gestalt psychology in visual perception: II. Conceptual and theoretical foundations.
Wagemans, J., Feldman, J., Gepshtein, S., Kimchi, R., Pomerantz, J. R., van der Helm, P. A., & van Leeuwen, C. (2012) · Psychological Bulletin · doi:10.1037/a0029334 · Cited by 557
Shape Representation in Area V4: Position-Specific Tuning for Boundary Conformation
Pasupathy, A., & Connor, C. E. (2001) · Journal of Neurophysiology · doi:10.1152/jn.2001.86.5.2505 · Cited by 483
The Briefest of Glances: The Time Course of Natural Scene Understanding
Greene, M. R., & Oliva, A. (2009) · Psychological Science · doi:10.1111/j.1467-9280.2009.02316.x · Cited by 482
CORTICAL ALGORITHMS FOR PERCEPTUAL GROUPING
Roelfsema, P. R. (2006) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.112939 · Cited by 432
Tarr, M. J., & Bülthoff, H. (1995) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.21.6.1494 · Cited by 400
Op de Beeck, H. P., Torfs, K., & Wagemans, J. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.2511-08.2008 · Cited by 266
Self-citationDissociations and Associations between Shape and Category Representations in the Two Visual Pathways
Bracci, S., & Op de Beeck, H. (2016) · The Journal of Neuroscience · doi:10.1523/jneurosci.2314-15.2016 · Cited by 261
Self-citationDiscrimination Training Alters Object Representations in Human Extrastriate Cortex
Op de Beeck, H. P., Baker, C. I., DiCarlo, J. J., & Kanwisher, N. G. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.2481-06.2006 · Cited by 250
Self-citationThe Role of Attention in Figure-Ground Segregation in Areas V1 and V4 of the Visual Cortex
Poort, J., Raudies, F., Wannig, A., Lamme, V. A. F., Neumann, H., & Roelfsema, P. R. (2012) · Neuron · doi:10.1016/j.neuron.2012.04.032 · Cited by 228
Inverse MDS: Inferring Dissimilarity Structure from Multiple Item Arrangements
Kriegeskorte, N., & Mur, M. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00245 · Cited by 219
(2002) · Hyperactivity and Attention Disorders of Childhood · doi:10.1017/cbo9780511544767.007 · Cited by 172
Tuning for shape dimensions in macaque inferior temporal cortex
Kayaert, G., Biederman, I., de Beeck, H. O., & Vogels, R. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04202.x · Cited by 151
Shape Tuning in Macaque Inferior Temporal Cortex
Kayaert, G., Biederman, I., & Vogels, R. (2003) · Journal of Neuroscience · doi:10.1523/jneurosci.23-07-03016.2003 · Cited by 123
Inferior Temporal Neurons Show Greater Sensitivity to Nonaccidental than to Metric Shape Differences
Vogels, R., Biederman, I., Bar, M., & Lörincz, A. (2001) · Journal of Cognitive Neuroscience · doi:10.1162/08989290152001871 · Cited by 94
Nonaccidental Properties Underlie Human Categorization of Complex Natural Scenes
Walther, D. B., & Shen, D. (2014) · Psychological Science · doi:10.1177/0956797613512662 · Cited by 89
Identification of Everyday Objects on the Basis of Silhouette and Outline Versions
Wagemans, J. P., Winter, J. D., de Beeck, H. O., Ploeger, A., Beckers, T., & Vanroose, P. (2008) · Perception · doi:10.1068/p5825 · Cited by 88
Proceedings of the COLING/ACL on Interactive presentation sessions -
(2006) · doi:10.3115/1225403 · Cited by 87
A Layered View of Shape Perception
Green, E. (2015) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/axv042 · Cited by 56
The neural basis for shape preferences
Amir, O., Biederman, I., & Hayworth, K. J. (2011) · Vision Research · doi:10.1016/j.visres.2011.08.015 · Cited by 54
Deep convolutional filter banks for texture recognition and segmentationPreprint
Cimpoi, M., Maji, S., & Vedaldi, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.6836 · Cited by 51
Kim, J. G., & Biederman, I. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.08.066 · Cited by 38
Biederman, I., Yue, X., & Davidoff, J. B. (2009) · Psychological Science · doi:10.1111/j.1467-9280.2009.02465.x · Cited by 37
A conceptual framework of computations in mid-level vision
Kubilius, J., Wagemans, J. P., & de Beeck, H. O. (2014) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2014.00158 · Cited by 33
Self-citationSeeing things from a different angle: The pigeon's recognition of single geons rotated in depth.
Peissig, J. J., Young, M. E., Wasserman, E. A., & Biederman, I. (2000) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.26.2.115 · Cited by 30
An, X., Gong, H., Yin, J., Wang, X., Pan, Y., Zhang, X., Lu, Y., Yang, Y., Toth, Z. G., Schießl, I., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0106753 · Cited by 24
A framework for streamlining research workflow in neuroscience and psychology
Kubilius, J. (2014) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2013.00052 · Cited by 22
Self-citationParker, S., & Serre, T. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00115 · Cited by 4
Cited by
Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., & Brendel, W. (2018) · arXiv · doi:10.48550/arxiv.1811.12231 · Cited by 1,490
“1 Moreover, CNNs have been proposed as computational models of human shape perception by Kubilius et al. (2016), who conducted an impressive number of experiments comparing human and CNN shape representations and concluded that CNNs “implicitly learn representations of shape that reflect human shape…”
BackgroundConvolutional Neural Networks as a Model of the Visual System: Past, Present, and Future
Lindsay, G. W. (2021) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01544 · Cited by 557
Cognitive computational neuroscience
Kriegeskorte, N., & Douglas, P. K. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0210-5 · Cited by 535
Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?
Schrimpf, M., Kubilius, J., Hong, H., Majaj, N. J., Rajalingham, R., Issa, E. B., Kar, K., Bashivan, P., Prescott-Roy, J., Geiger, F., et al. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/407007 · Cited by 458
Self-citationDeep Neural Networks as Scientific Models
Cichy, R. M., & Kaiser, D. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.01.009 · Cited by 454
Metadata
- Authors
- Kubilius, J., Bracci, S., & Op de Beeck, H. P.
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-04-28
- Volume
- 12
- Issue
- 4
- Pages
- e1004896
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2343204383
- Cited by
- 295
- References
- 67
- FWCI
- 20.37
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.