The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?
Open accessAbstract
The internal representations of early deep artificial neural networks (ANNs) were found to be remarkably similar to the internal neural representations measured experimentally in the primate brain. Here we ask, as deep ANNs have continued to evolve, are they becoming more or less brain-like? ANNs that are most functionally similar to the brain will contain mechanisms that are most like those used by the brain. We therefore developed Brain-Score – a composite of multiple neural and behavioral benchmarks that score any ANN on how similar it is to the brain’s mechanisms for core object recognition – and we deployed it to evaluate a wide range of state-of-the-art deep ANNs. Using this scoring system, we here report that: (1) DenseNet-169, CORnet-S and ResNet-101 are the most brain-like ANNs. (2) There remains considerable variability in neural and behavioral responses that is not predicted by any ANN, suggesting that no ANN model has yet captured all the relevant mechanisms. (3) Extending prior work, we found that gains in ANN ImageNet performance led to gains on Brain-Score. However, correlation weakened at ≥ 70% top-1 ImageNet performance, suggesting that additional guidance from neuroscience is needed to make further advances in capturing brain mechanisms. (4) We uncovered smaller (i.e. less complex) ANNs that are more brain-like than many of the best-performing ImageNet models, which suggests the opportunity to simplify ANNs to better understand the ventral stream. The scoring system used here is far from complete. However, we propose that evaluating and tracking model-benchmark correspondences through a Brain-Score that is regularly updated with new brain data is an exciting opportunity: experimental benchmarks can be used to guide machine network evolution, and machine networks are mechanistic hypotheses of the brain’s network and thus drive next experiments. To facilitate both of these, we release Brain-Score.org : a platform that hosts the neural and behavioral benchmarks, where ANNs for visual processing can be submitted to receive a Brain-Score and their rank relative to other models, and where new experimental data can be naturally incorporated.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
(2017) · Cited by 11,625
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016) · arXiv · doi:10.48550/arxiv.1602.07360 · Cited by 6,201
Neural Architecture Search with Reinforcement Learning
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 3,974
(2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-24553-9 · Cited by 2,285
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924
Self-citationUsing goal-driven deep learning models to understand sensory cortex
Yamins, D. L. K., & DiCarlo, J. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4244 · Cited by 1,621
Self-citationA review of unsupervised feature learning and deep learning for time-series modeling
Längkvist, M., Karlsson, L., & Loutfi, A. (2014) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.01.008 · Cited by 1,243
Exploring the Limits of Weakly Supervised Pretraining
Mahajan, D., Girshick, R., Ramanathan, V., He, K., Paluri, M., Li, Y., Bharambe, A., & van der Maaten, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01216-8_12 · Cited by 1,224
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204
Guclu, U., & van Gerven, M. A. J. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.5023-14.2015 · Cited by 961
Cichy, R. M., Khosla, A., Pantazis, D., Torralba, A., & Oliva, A. (2016) · Scientific Reports · doi:10.1038/srep27755 · Cited by 716
Kell, A. J. E., Yamins, D. L. K., Shook, E. N., Norman-Haignere, S. V., & McDermott, J. H. (2018) · Neuron · doi:10.1016/j.neuron.2018.03.044 · Cited by 544
Self-citationFlexible timing by temporal scaling of cortical responses
Wang, J., Narain, D., Hosseini, E. A., & Jazayeri, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-017-0028-6 · Cited by 443
Generalisation in humans and deep neural networks
Geirhos, R., Temme, C. R. M., Rauber, J., Schütt, H. H., Bethge, M., & Wichmann, F. A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.08750 · Cited by 399
Rajalingham, R., Issa, E. B., Bashivan, P., Kar, K., Schmidt, K., & DiCarlo, J. J. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.0388-18.2018 · Cited by 358
Self-citationDeep convolutional models improve predictions of macaque V1 responses to natural images
Cadena, S. A., Denfield, G. H., Walker, E. Y., Gatys, L. A., Tolias, A. S., Bethge, M., & Ecker, A. S. (2019) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1006897 · Cited by 347
Supporting user-oriented analysis for multi-view domain-specific visual languages
Guerra, E., de Lara, J., Malizia, A., & Díaz, P. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.005 · Cited by 322
Günther, J., Pilarski, P. M., Helfrich, G., Shen, H., & Diepold, K. (2014) · Procedia Technology · doi:10.1016/j.protcy.2014.09.007 · Cited by 303
Deep Neural Networks as a Computational Model for Human Shape Sensitivity
Kubilius, J., Bracci, S., & Op de Beeck, H. P. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004896 · Cited by 295
Self-citationMajaj, N. J., Hong, H., Solomon, E. A., & DiCarlo, J. J. (2015) · The Journal of Neuroscience · doi:10.1523/jneurosci.5181-14.2015 · Cited by 204
Self-citationYamins, D. L. K., Hong, H., Cadieu, C. F., & DiCarlo, J. J. (2013) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 116
Self-citationPixels to Voxels: Modeling Visual Representation in the Human BrainPreprint
Agrawal, P., Stansbury, D. E., Malik, J., & Gallant, J. L. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1407.5104 · Cited by 109
Comparison of Object Recognition Behavior in Human and Monkey
Rajalingham, R., Schmidt, K., & DiCarlo, J. J. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.0573-15.2015 · Cited by 109
Self-citationRajalingham, R., & DiCarlo, J. J. (2019) · Neuron · doi:10.1016/j.neuron.2019.02.001 · Cited by 48
Self-citationToward Goal-Driven Neural Network Models for the Rodent Whisker-Trigeminal SystemPreprint
Zhuang, C., Kubilius, J., Hartmann, M. J. Z., & Yamins, D. L. K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.07555 · Cited by 24
Self-citationCan Deep Neural Networks Rival Human Ability to Generalize in Core Object Recognition?
Kubilius, J., Kar, K., Schmidt, K., & DiCarlo, J. J. (2018) · Conference on Cognitive Computational Neuroscience (CCN) · doi:10.32470/ccn.2018.1234-0 · Cited by 7
Self-citationNeural Population Control via Deep ANN Image Synthesis
Bashivan, P., Kar, K., & DiCarlo, J. J. (2018) · Conference on Cognitive Computational Neuroscience (CCN) · doi:10.32470/ccn.2018.1222-0 · Cited by 4
Self-citation
Cited by
A deep learning framework for neuroscience
Richards, B. A., Lillicrap, T. P., Beaudoin, P., Bengio, Y., Bogacz, R., Christensen, A., Clopath, C., Costa, R. P., de Berker, A., Ganguli, S., et al. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0520-2 · Cited by 1,027
Self-citationThe neural architecture of language: Integrative modeling converges on predictive processing
Schrimpf, M., Blank, I. A., Tuckute, G., Kauf, C., Hosseini, E. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2105646118 · Cited by 517
Self-citationKar, K., Kubilius, J., Schmidt, K., Issa, E. B., & DiCarlo, J. J. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0392-5 · Cited by 496
Self-citationNeural population control via deep image synthesis
Bashivan, P., Kar, K., & DiCarlo, J. J. (2019) · Science · doi:10.1126/science.aav9436 · Cited by 434
Self-citationIf deep learning is the answer, what is the question?
Saxe, A., Nelli, S., & Summerfield, C. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-00395-8 · Cited by 397
Metadata
- Authors
- Schrimpf, M., Kubilius, J., Hong, H., Majaj, N. J., Rajalingham, R., Issa, E. B., Kar, K., Bashivan, P., Prescott-Roy, J., Geiger, F., Schmidt, K., Yamins, D. L. K., & DiCarlo, J. J.
- Venue
- bioRxiv (Cold Spring Harbor Laboratory)
- Publisher
- openRxiv
- Published
- 2018-09-05
- OpenAlex
- W2892147425
- Cited by
- 458
- References
- 34
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.