Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation
Open accessThe DOI matches its Crossref record; 99 of the 99 references it cites were found in a registry or an index.
Abstract
Inferior temporal (IT) cortex in human and nonhuman primates serves visual object recognition. Computational object-vision models, although continually improving, do not yet reach human performance. It is unclear to what extent the internal representations of computational models can explain the IT representation. Here we investigate a wide range of computational model representations (37 in total), testing their categorization performance and their ability to account for the IT representational geometry. The models include well-known neuroscientific object-recognition models (e.g. HMAX, VisNet) along with several models from computer vision (e.g. SIFT, GIST, self-similarity features, and a deep convolutional neural network). We compared the representational dissimilarity matrices (RDMs) of the model representations with the RDMs obtained from human IT (measured with fMRI) and monkey IT (measured with cell recording) for the same set of stimuli (not used in training the models). Better performing models were more similar to IT in that they showed greater clustering of representational patterns by category. In addition, better performing models also more strongly resembled IT in terms of their within-category representational dissimilarities. Representational geometries were significantly correlated between IT and many of the models. However, the categorical clustering observed in IT was largely unexplained by the unsupervised models. The deep convolutional network, which was trained by supervision with over a million category-labeled images, reached the highest categorization performance and also best explained IT, although it did not fully explain the IT data. Combining the features of this model with appropriate weights and adding linear combinations that maximize the margin between animate and inanimate objects and between faces and other objects yielded a representation that fully explained our IT data. Overall, our results suggest that explaining IT requires computational features trained through supervised learning to emphasize the behaviorally important categorical divisions prominently reflected in IT.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





![Figure 6 RDMs of all layers of the strongly supervised deep convolutional network. RDMs for all layers of the deep convolutional network (Krizhevsky et al. 2012) ref [41] are shown for the set of the 96 images (L1: layer 1 to L7: layer 7). Kendall-τ A RDM correlations of the models with hIT and mIT are stated underneath each RDM. All correlations are statistically significant. For inferential comparisons to IT and other regions, see Figure 7 and Table 2, respectively.](https://figures.cochat.ai/paper-figures/cdd6af5b05dc419485a2fef207a50051/deep-supervised-but-not-unsupervised-models-may-explain-it-cortical-figure-6.jpg)






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Using this approach, known as neuroconnectionism, numerous studies have identified shared representational structure across brains and ANN models, as well as factors influencing this alignment [5–11]”
From The Umwelt Representation Hypothesis: rethinking Universality (2026) · cites this paper as [10]
The cited source says what the claim attributes to it.
In the citing paper
“Analogous approaches have proven successful in relating the structure of neural activity to computational functions in other domains, including vision 17–19, audition 20, olfaction 21,22, thermosensation 23, perceptual discrimination 24, facial perception 25, and navigation 26,27.”
From A virtual rodent predicts the structure of neural activity across behaviours (2024) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 2×
“This is a supervised convolutional neural network, trained with 1.2 million labelled images from ImageNet (1000 category labels) [52].”
MethodInfluentialDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Object recognition from local scale-invariant features
Lowe, D. G. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790410 · Cited by 16,472
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · cited 1×
“In recent years, the advent of efficient training algorithms for deep neural networks [40,71,72,41] has made it possible to learn from image data not just the final classification step, but also the internal representations.”
BackgroundVisualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Multiresolution gray-scale and rotation invariant texture classification with local binary patterns
Ojala, T., Pietikainen, M., & Maenpaa, T. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1017623 · Cited by 15,398
Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex
Hubel, D. H., & Wiesel, T. N. (1962) · The Journal of Physiology · doi:10.1113/jphysiol.1962.sp006837 · Cited by 13,680
Libsvm : A library for support vector machines
Ferrari, V. (2008) · Medical Entomology and Zoology · Cited by 10,122
Research works printed but not linked: 1
- Krizhevsky A., Sutskever I. and Hinton, G E. (2012) ImageNet Classification with Deep Convolutional Neural Networks. NIPS. Lake Tahoe, Nevada.
Cited by
Items 1,001–1,003 of 1,003
Basic Analysis of Electrophysiological Signals
de Beeck, H. O., & Nakatani, C. (2019) · Cambridge University Press eBooks · doi:10.1017/9781316847916.015
de Beeck, H. O., & Nakatani, C. (2019) · Cambridge University Press eBooks · doi:10.1017/9781316847916.011
Mattioni, S., Rezk, M., Battal, C., Bottini, R., Mendoza, K. E. C., Oosterhof, N. N., & Collignon, O. (2019) · eLife · doi:10.7554/elife.50732.sa2
Metadata
- Authors
- Khaligh-Razavi, S.-M., & Kriegeskorte, N.
- Affiliations
- MRC Cognition and Brain Sciences Unit (GB)
- Corresponding author
- Kriegeskorte, Nikolaus
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-11-06
- Volume
- 10
- Issue
- 11
- Pages
- e1003915
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC4222664
- OpenAlex
- W2040036684
- Cited by
- 1,204
- References
- 98
- Citations by year
- 2026: 75 · 2025: 100 · 2024: 110 · 2023: 127 · 2022: 112 · 2021: 124 · 2020: 125 · 2019: 151 · 2018: 97 · 2017: 87 · 2016: 75 · 2015: 20 · 2014: 1
- FWCI
- 34.02
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Cambridge Overseas Trust
- Medical Research Council (MC-A060- 5PR20)
- FP7 Ideas: European Research Council (261352)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- inferior temporal cortex, visual object recognition, representational geometry, representational dissimilarity matrices, supervised learning, unsupervised learning, deep convolutional neural networks, categorical clustering, HMAX, monkey visual cortex, human visual cortex
- MeSH terms
- Animals, Haplorhini, Humans, Models, Neurological, Temporal Lobe, Computational Biology, Support Vector Machine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.