Brain-like functional specialization emerges spontaneously in deep neural networks
Open accessPublished version of a preprint
The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
Abstract
The human brain contains multiple regions with distinct, often highly specialized functions, from recognizing faces to understanding language to thinking about what others are thinking. However, it remains unclear why the cortex exhibits this high degree of functional specialization in the first place. Here, we consider the case of face perception using artificial neural networks to test the hypothesis that functional segregation of face recognition in the brain reflects a computational optimization for the broader problem of visual recognition of faces and other visual categories. We find that networks trained on object recognition perform poorly on face recognition and vice versa and that networks optimized for both tasks spontaneously segregate themselves into separate systems for faces and objects. We then show functional segregation to varying degrees for other visual categories, revealing a widespread tendency for optimization (without built-in task-specific inductive biases) to lead to functional specialization in machines and, we conjecture, also brains.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Fig. 1. Distinct face and object representations in singly trained CNNs while a dual-task CNN performs well. (A) Three networks with VGG16 architecture (left) were optimized, one on face identity categorization (Face CNN in red), one on object categorization (Object CNN in orange), and one on both tasks simultaneously (dual-task CNN in gray). (B) Decoding accuracy of held-out face identities and held-out object categories using activation patterns extracted from the penultimate layer [i.e., FC2 in (A)] of the Face CNN and the Object CNN. The Face CNN outperforms the Object CNN in face decoding and vice versa for object decoding. Thus, the representations optimized for each task do not naturally support the other. The dashed gray line indicates chance level (1%). Error bars indicate SEM across classification folds. (C) A dual-task CNN optimized on both tasks performed and the separate networks (% top 1 accuracy on the test set). Error bars denote 95% confidence interval (CI) bootstrapped across classes and stimuli.](https://figures.cochat.ai/paper-figures/f7fd2e615ccf47089c9fb1193e6a0236/brain-like-functional-specialization-emerges-spontaneously-in-deep-neural-figure-1.jpg)





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Using this approach, known as neuroconnectionism, numerous studies have identified shared representational structure across brains and ANN models, as well as factors influencing this alignment [5–11]”
From The Umwelt Representation Hypothesis: rethinking Universality (2026) · cites this paper as [7]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Modular structure has been shown to spontaneously emerge in some end-to-end neural network systems in domains other than language [e.g., 203, 204]”
From Dissociating language and thought in large language models (2024) · cites this paper as [204]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 3×
“We used similar training parameters as suggested in the original VGG paper (20): stochastic gradient descent (SGD) with momentum with an initial learning rate of 10−3, a weight decay of 10−4, and momentum of 0.”
MethodImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1×
“MATERIALS AND METHODS Task-optimizing CNNs on face and object recognition To measure object categorization performance in a network trained on object categorization only (Object CNN), we trained a randomly initialized VGG16 network (20) on 423 manually sampled categories of the ILSVRC-2012 database (53).”
MethodRich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
The fusiform face area: a module in human extrastriate cortex specialized for face perception
Kanwisher, N., McDermott, J., & Chun, M. M. (1997) · The Journal of Neuroscience · doi:10.1523/jneurosci.17-11-04302.1997 · Cited by 7,738 · cited 1×
“Here, we test this third hypothesis for one of the best-established cases of functional specialization in the brain: the visual recognition of faces (5, 7, 8).”
BackgroundSelf-citationVGGFace2: A Dataset for Recognising Faces across Pose and Age
Cao, Q., Shen, L., Xie, W., Parkhi, O. M., & Zisserman, A. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00020 · Cited by 3,032
Food-101 – Mining Discriminative Components with Random Forests
Bossard, L., Guillaumin, M., & Van Gool, L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10599-4_29 · Cited by 2,072 · cited 2×
“To test whether we would find segregation for other natural tasks, we additionally trained two dual-task networks: (i) We trained a network on object categorization and food discrimination using the Food101 ( 57 ) dataset.”
MethodPerformance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924
Learning Face Representation from Scratch
Yi, D., Lei, Z., Liao, S., & Li, S. Z. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.7923 · Cited by 1,521
Understanding Neural Networks Through Deep Visualization
Yosinski, J., Clune, J., Nguyen, A. M., Fuchs, T. J., & Lipson, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06579 · Cited by 1,358 · cited 2×
“In addition, periodic (every 10 iterations) blurring and random jittering were applied to the pixel values (50) to enhance visualization,”
BackgroundMethodDo Better ImageNet Models Transfer Better?
Kornblith, S., Shlens, J., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00277 · Cited by 1,325 · cited 2×
“Prior work suggests that this may not necessarily be the case: CNNs optimized for generic object recognition transfer well to many other tasks (12–15), including fine-grained discrimination within a category (13).”
Background
Research works printed but not linked: 6
- S. Finger Origins Of Neuroscience: A History Of Explorations Into Brain Function (Oxford Univ. Press 2001).
- C. R. Gallistel The replacement of general-purpose learning models with adaptively specialized learning modules in The Cognitive Neurosciences M. S. Gazzaniga Ed. (MIT Press ed. 2 2000) pp. 1179–1191.
- B. Long G. Kachergis K. Agrawal M. C. Frank Detecting social information in a dense dataset of infants’ natural visual experience (PsyArXiv 2020); https://psyarxiv.com/z7tdg/.
- 10.1037/0033-295x.98.2.164 · doi:10.1037/0033-295x.98.2.164 · Paper page
- A. Nayebi N. C. L. Kong C. Zhuang J. L. Gardner A. M. Norcia D. L. K. Yamins Unsupervised models of mouse visual cortex. bioRxiv 448730 [Preprint] (2021); www.biorxiv.org/content/10.1101/2021.06.16.448730v1. · doi:10.1101/2021.06.16.448730v1
- A. Krizhevsky I. Sutskever G. E. Hinton in Advances in Neural Information Processing Systems (MIT Press 2012) pp. 1097–1105.
Cited by
Dissociating language and thought in large language models
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.011 · Cited by 345
“Modular structure has been shown to spontaneously emerge in some end-to-end neural network systems in domains other than language (e.g., Dobs et al., 2022; Yang et al., 2019), which suggests that emergent modularity may constitute an optimal solution to many complex tasks.”
Self-citationThe neuroconnectionist research programme
Doerig, A., Sommers, R. P., Seeliger, K., Richards, B., Ismael, J., Lindsay, G. W., Kording, K. P., Konkle, T., van Gerven, M. A. J., Kriegeskorte, N., et al. (2023) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00705-w · Cited by 233
Using artificial neural networks to ask ‘why’ questions of minds and brains
Kanwisher, N., Khosla, M., & Dobs, K. (2023) · Trends in Neurosciences · doi:10.1016/j.tins.2022.12.008 · Cited by 182
“However, segregation of the highly heterogeneous visual category of food in both networks [44] and brains [72] suggests that this account is incomplete at best.”
BackgroundMethodInfluentialSelf-citationSelective responses to faces, scenes, and bodies in the ventral visual pathway of infants
Kosakowski, H. L., Cohen, M. A., Takahashi, A., Keil, B., Kanwisher, N., & Saxe, R. (2022) · Current Biology · doi:10.1016/j.cub.2021.10.064 · Cited by 163
Self-citationKhosla, M., Murty, N. A. R., & Kanwisher, N. (2022) · Current Biology · doi:10.1016/j.cub.2022.08.009 · Cited by 135
Self-citation
Metadata
- Authors
- Dobs, K., Martinez, J., Kell, A. J. E., & Kanwisher, N.
- Affiliations
- McGovern Institute for Brain Research (US)
- Mortimer B. Zuckerman Mind Brain Behavior Institute (US)
- Corresponding author
- Dobs, Katharina
- Venue
- Science Advances
- Published
- 2022-03-18
- Volume
- 8
- Issue
- 11
- Pages
- eabl8913-eabl8913
- ISSN
- 2375-2548 (ISSN-L)
- PMC
- PMC8926347
- OpenAlex
- W4225899846
- Cited by
- 129
- References
- 52
- Citations by year
- 2026: 25 · 2025: 24 · 2024: 33 · 2023: 30 · 2022: 16
- FWCI
- 13.28
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc
- Locations
- Science Advances (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- Eunice Kennedy Shriver National Institute of Child Health and Human Development (DP1HD091947)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- functional specialization, functional segregation, face recognition (cognition), object recognition, deep neural network, computational optimization, inductive bias, brain organization, visual cortex
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.