The DOI matches its Crossref record; 38 of the 40 references it cites were found in a registry or an index.
Individual differences among deep neural network models
Open accessPublished version of a preprint
Abstract
Deep neural networks (DNNs) excel at visual recognition tasks and are increasingly used as a modeling framework for neural computations in the primate brain. Just like individual brains, each DNN has a unique connectivity and representational profile. Here, we investigate individual differences among DNN instances that arise from varying only the random initialization of the network weights. Using tools typically employed in systems neuroscience, we show that this minimal change in initial conditions prior to training leads to substantial differences in intermediate and higher-level network representations despite similar network-level classification performance. We locate the origins of the effects in an under-constrained alignment of category exemplars, rather than misaligned category centroids. These results call into question the common practice of using single networks to derive insights into neural information processing and rather suggest that computational neuroscientists working with DNNs may need to base their inferences on groups of multiple network instances.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures








![Fig. 9 Final-layer representational consistency (exemplar-based) across training epochs. a Comparing representational consistency across early epochs [1 to 10] (left) and throughout all training epochs [1 to 350 in steps of 50] (right). Lines parallel to the main diagonal indicate that network instances remain on their distinct representational trajectory compared to other networks. Average consistency shown across 45 network pairs, derived from 10 network instances. b Representational consistency, computed and averaged across all network pairs (45 pairs total) for each training epoch, demonstrates increasing individual differences with training (shown with 95% CI). c Test performance across training (average top-1 accuracy across 10 network instances with 95% CI). d Representational consistency and test performance exhibit a strong negative relationship (Pearson r = −0.91, p < 0.001; robust correlation) indicating that task training enhances individual differences (dots represent network training epochs, error bar indicates 95% CI).](https://figures.cochat.ai/paper-figures/b9fd7e77cdfe499c88471522d9b3f53a/individual-differences-among-deep-neural-network-models-figure-9.jpg)

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Moreover, systematic individual differences also arise across ANNs when varying network depth, dropout, and random initializations [73,79]”
From The Umwelt Representation Hypothesis: rethinking Universality (2026) · cites this paper as [79]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 2×
“To expand our experiments to network architectures commonly used to predict brain data 3,4,13 – 15 , we trained and tested 10 network instances of a recent version of AlexNet 16 on a large-scale object classi fi cation data set ILSVRC 2012 17 .”
Method(2014) · Cited by 33,419
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · cited 1×
“As a result, the networks only differ in the initial random weights, which are, however, sampled from the same distribution (He et al., 2015).”
BackgroundStriving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447 · cited 1×
“We approach this question by training multiple instances of the All-CNN-C network architecture 11 and a custom architecture (All-CNN-7) on an object classi fi cation task (CIFAR-10 12 ), followed by an in-depth analysis of resulting network-internal representations.”
MethodRepresentational similarity analysis-connecting the branches of systems neuroscience
Kriegeskorte, N. (2008) · Frontiers in Systems Neuroscience · doi:10.3389/neuro.06.004.2008 · Cited by 3,355
Self-citationReturn of the Devil in the Details: Delving Deep into Convolutional Nets
Chatfield, K., Simonyan, K., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.6 · Cited by 3,335
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204
Self-citation
Research works printed but not linked: 1
- Raghu, M., Gilmer, J., Yosinski, J. & Sohl-Dickstein, J. SVCCA: singular vector canonical correlation analysis for deep learning dynamics and interpretability. In 31st Conference on Neural Information Processing Systems 10 (2017).
Cited by
The neuroconnectionist research programme
Doerig, A., Sommers, R. P., Seeliger, K., Richards, B., Ismael, J., Lindsay, G. W., Kording, K. P., Konkle, T., van Gerven, M. A. J., Kriegeskorte, N., et al. (2023) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00705-w · Cited by 233
Self-citationFlesch, T., Juechems, K., Dumbalska, T., Saxe, A., & Summerfield, C. (2022) · Neuron · doi:10.1016/j.neuron.2022.01.005 · Cited by 249
“More generally, the diverse representational structure that can emerge in the rich and lazy regimes, and its variablemapping to the brain, may shed light on why emerging representation structure can be heterogenous in trained neural networks (Mehrer et al., 2020).”
Deep problems with neural network models of human vision
Bowers, J. S., Malhotra, G., Dujmović, M., Llera Montero, M., Tsvetkov, C., Biscione, V., Puebla, G., Adolfi, F., Hummel, J. E., Heaton, R. F., et al. (2023) · Behavioral and Brain Sciences · doi:10.1017/s0140525x22002813 · Cited by 204
Using artificial neural networks to ask ‘why’ questions of minds and brains
Kanwisher, N., Khosla, M., & Dobs, K. (2023) · Trends in Neurosciences · doi:10.1016/j.tins.2022.12.008 · Cited by 182
“Better yet, researchers should kick the tires on network hyperparameters (e.g., using different random initializations [58]) to test the robustness of the emergence of the particular phenomenon in question.”
BackgroundAn ecologically motivated image dataset for deep learning yields better models of human vision
Mehrer, J., Spoerer, C. J., Jones, E. C., Kriegeskorte, N., & Kietzmann, T. C. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2011417118 · Cited by 140
“To account for individual differences among DNNs (21), 10 network instances per architecture, each initialized with different random weights, were trained on each dataset (see Materials and Methods ).”
Self-citation
Metadata
- Authors
- Mehrer, J., Spoerer, C. J., Kriegeskorte, N., & Kietzmann, T. C.
- Affiliations
- MRC Cognition and Brain Sciences Unit (GB)
- Mortimer B. Zuckerman Mind Brain Behavior Institute (US)
- Radboud University Nijmegen (NL)
- Corresponding author
- Mehrer, Johannes
- Kietzmann, Tim C.
- Venue
- Nature Communications
- Published
- 2020-11-12
- Volume
- 11
- Issue
- 1
- Pages
- 5725-5725
- ISSN
- 2041-1723 (ISSN-L)
- PMC
- PMC7665054
- OpenAlex
- W3100481436
- Cited by
- 133
- References
- 40
- Citations by year
- 2026: 20 · 2025: 25 · 2024: 25 · 2023: 19 · 2022: 21 · 2021: 18 · 2020: 5
- FWCI
- 6.19
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Nature Communications (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Radboud Repository (Radboud University) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- Funders
- EC | Horizon 2020 Framework Programme (720270, 785907)
- Deutsche Forschungsgemeinschaft (DynaVision)
- Nvidia (GPU Program)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- deep neural network, individual differences, weight initialization, visual recognition
- MeSH terms
- Cognitive Neuroscience, Animals, Brain, Individuality, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Individual differences among deep neural network models. https://app.cochat.ai/papers/10.1038/s41467-020-19632-w
- The paper
- Mehrer, J., Spoerer, C. J., Kriegeskorte, N., & Kietzmann, T. C. (2020). Individual differences among deep neural network models. Nature Communications. https://doi.org/10.1038/s41467-020-19632-w
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.