CoChat

Representations and generalization in artificial and brain neural networks

Open access
Li, Qianyi, Sorscher, Ben, Sompolinsky, Haim · 2024 · Proceedings of the National Academy of Sciences

The DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index.

Abstract

Humans and animals excel at generalizing from limited data, a capability yet to be fully replicated in artificial intelligence. This perspective investigates generalization in biological and artificial deep neural networks (DNNs), in both in-distribution and out-of-distribution contexts. We introduce two hypotheses: First, the geometric properties of the neural manifolds associated with discrete cognitive entities, such as objects, words, and concepts, are powerful order parameters. They link the neural substrate to the generalization capabilities and provide a unified methodology bridging gaps between neuroscience, machine learning, and cognitive science. We overview recent progress in studying the geometry of neural manifolds, particularly in visual object recognition, and discuss theories connecting manifold dimension and radius to generalization capacity. Second, we suggest that the theory of learning in wide DNNs, especially in the thermodynamic limit, provides mechanistic insights into the learning processes generating desired neural representational geometries and generalization. This includes the role of weight norm regularization, network architecture, and hyper-parameters. We will explore recent advances in this theory and ongoing challenges. We also discuss the dynamics of learning and its relevance to the issue of representational drift in the brain.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1073/pnas.2311805121

Cites

75 of 75 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1. (A) Illustration of three layers in a visual hierarchy where the population response of the first layer is mapped into intermediate layer by F1 and into the last layer by F2 (Top) (10). The transformation of per-stimuli responses is associated with changes in the geometry of the object manifold, the collection of responses to stimuli of the same object (colored blue for a “dog” manifold and pink for a “cat” manifold). Changes in geometry may result in transforming object manifolds that are not linearly separable (in the first and intermediate layers) into separable ones in the last layer (separating hyperplane, colored orange). (B) Changes in classification capacity αC, manifold radius RM, manifold dimension DM, and classification margin κ across the layers of pre-trained DNNs (ResNets).
Fig. 1. (A) Illustration of three layers in a visual hierarchy where the population response of the first layer is mapped into intermediate layer by F1 and into the last layer by F2 (Top) (10). The transformation of per-stimuli responses is associated with changes in the geometry of the object manifold, the collection of responses to stimuli of the same object (colored blue for a “dog” manifold and pink for a “cat” manifold). Changes in geometry may result in transforming object manifolds that are not linearly separable (in the first and intermediate layers) into separable ones in the last layer (separating hyperplane, colored orange). (B) Changes in classification capacity αC, manifold radius RM, manifold dimension DM, and classification margin κ across the layers of pre-trained DNNs (ResNets).CC BY · PubMed Central
Fig. 2. (A and B) Examples of novel objects, here “coatis” (blue) and “numbats” (green), are presented to the ventral visual pathway (Top), modeled by a trained DNN (Bottom), eliciting a pattern of activity across IT-like neurons in the feature layer. We model concept learning as learning a linear readout w to classify these activity patterns. (C), Generalization accuracy is very high across pairs of novel objects from the ImageNet21k dataset when using a pre-trained DNN (orange), but poor when using a randomly initialized DNN (blue), or a linear classifier in the pixel space of input images (gray). (D) Few-shot learning improves along the ventral visual hierarchy from pixels to V1 to V4 to IT, due to orchestrated transformations of object manifold geometry. The layerwise behavior of a trained ResNet50 (blue), Alexnet (light blue), and an untrained ResNet50 (gray) is included for comparison. We align V1, V4, and IT to the most similar ResNet layer under the BrainScore metric (20) (see ref. 11 for details).
Fig. 2. (A and B) Examples of novel objects, here “coatis” (blue) and “numbats” (green), are presented to the ventral visual pathway (Top), modeled by a trained DNN (Bottom), eliciting a pattern of activity across IT-like neurons in the feature layer. We model concept learning as learning a linear readout w to classify these activity patterns. (C), Generalization accuracy is very high across pairs of novel objects from the ImageNet21k dataset when using a pre-trained DNN (orange), but poor when using a randomly initialized DNN (blue), or a linear classifier in the pixel space of input images (gray). (D) Few-shot learning improves along the ventral visual hierarchy from pixels to V1 to V4 to IT, due to orchestrated transformations of object manifold geometry. The layerwise behavior of a trained ResNet50 (blue), Alexnet (light blue), and an untrained ResNet50 (gray) is included for comparison. We align V1, V4, and IT to the most similar ResNet layer under the BrainScore metric (20) (see ref. 11 for details).CC BY · PubMed Central
Fig. 3. (A) We compare the empirical generalization error in 1-, 2-, and 5-shot learning experiments to the prediction from our geometric theory (Eq. 3) on all pairs of objects from the ImageNet21k dataset, using object manifolds derived from a trained ResNet50. x-axis: SNR obtained by estimating neural manifold geometry. y-axis: Empirical generalization error measured in few-shot learning experiments. Theoretical prediction (dashed line) shows a good match with experiments. (B) We provide additional examples of 5-shot prototype learning experiments in a ResNet50 (colored points), along with the prediction from our geometric theory (dashed line), on four randomly selected novel visual objects from the ImageNet21k dataset. Each panel plots the generalization error of one novel visual object (e.g., “Virginia bluebell”) against all 999 other novel visual objects. Each point represents the average generalization error on one such pair of objects. x-axis: SNR (Eq. 3) obtained by estimating neural manifold geometry. y-axis: Empirical generalization error measured in few-shot learning experiments. Theoretical prediction (dashed line) shows a good match with experiments. (C) In a pre-trained ResNet50 (blue) dimensionality expands dramatically in the early layers and contracts in the later layers, while in the primate visual pathway (Black) dimensionality contracts from the V1-like layer to V4, then expands from V4 to IT. (D) Single-manifold eigenspectra in macaque V4 (black) and the corresponding layer of a pre-trained ResNet50 (blue).
Fig. 3. (A) We compare the empirical generalization error in 1-, 2-, and 5-shot learning experiments to the prediction from our geometric theory (Eq. 3) on all pairs of objects from the ImageNet21k dataset, using object manifolds derived from a trained ResNet50. x-axis: SNR obtained by estimating neural manifold geometry. y-axis: Empirical generalization error measured in few-shot learning experiments. Theoretical prediction (dashed line) shows a good match with experiments. (B) We provide additional examples of 5-shot prototype learning experiments in a ResNet50 (colored points), along with the prediction from our geometric theory (dashed line), on four randomly selected novel visual objects from the ImageNet21k dataset. Each panel plots the generalization error of one novel visual object (e.g., “Virginia bluebell”) against all 999 other novel visual objects. Each point represents the average generalization error on one such pair of objects. x-axis: SNR (Eq. 3) obtained by estimating neural manifold geometry. y-axis: Empirical generalization error measured in few-shot learning experiments. Theoretical prediction (dashed line) shows a good match with experiments. (C) In a pre-trained ResNet50 (blue) dimensionality expands dramatically in the early layers and contracts in the later layers, while in the primate visual pathway (Black) dimensionality contracts from the V1-like layer to V4, then expands from V4 to IT. (D) Single-manifold eigenspectra in macaque V4 (black) and the corresponding layer of a pre-trained ResNet50 (blue).CC BY · PubMed Central
Fig. 4. (A) Two stages of learning of Langevin dynamics with small T, σ0 controls the width of weight distribution at initialization, σ controls the size of the solution space, and T relates to the sampling speed. (B) Example trajectories of the predictor from three different initializations, the dynamics is initially deterministic and starts to fluctuate as Θt drifts in the solution space after reaching zero training error.
Fig. 4. (A) Two stages of learning of Langevin dynamics with small T, σ0 controls the width of weight distribution at initialization, σ controls the size of the solution space, and T relates to the sampling speed. (B) Example trajectories of the predictor from three different initializations, the dynamics is initially deterministic and starts to fluctuate as Θt drifts in the solution space after reaching zero training error.CC BY · PubMed Central
Fig. 5. (A) Schematics for the BPKR approach. A renormalization factor is introduced at each step during backward integration until all the network weights are averaged out. (B) Theory (black solid line) and simulation (blue points) of generalization error εg=⟨ϵg(x,y(x))⟩{x,y(x)} on binary MNIST classification in fully connected ReLU networks, for small (Top) and large (Bottom) σ. The approximate theory for ReLU networks agrees remarkably well with the numerics.
Fig. 5. (A) Schematics for the BPKR approach. A renormalization factor is introduced at each step during backward integration until all the network weights are averaged out. (B) Theory (black solid line) and simulation (blue points) of generalization error εg=⟨ϵg(x,y(x))⟩{x,y(x)} on binary MNIST classification in fully connected ReLU networks, for small (Top) and large (Bottom) σ. The approximate theory for ReLU networks agrees remarkably well with the numerics.CC BY · PubMed Central
Fig. 6. (A and B) NNGP and mean layer-wise kernels in classifying eight MNIST digits SI Appendix, 3, (57). (C) Generalization error averaged across test examples for finite width random feature model (blue), infinitely wide network following the NNGP theory (yellow), and the learned network following the BPKR theory (red, overlaying NNGP theory). (D) The NNGP kernel, and the mean layer-wise kernel of hidden layer l=2,4. For a 4-hidden-layer ReLU network trained on four MNIST digits grouped into two higher-order categories of even vs. odd. The values of the kernel are small since we take relatively small σ (SI Appendix, 3).
Fig. 6. (A and B) NNGP and mean layer-wise kernels in classifying eight MNIST digits SI Appendix, 3, (57). (C) Generalization error averaged across test examples for finite width random feature model (blue), infinitely wide network following the NNGP theory (yellow), and the learned network following the BPKR theory (red, overlaying NNGP theory). (D) The NNGP kernel, and the mean layer-wise kernel of hidden layer l=2,4. For a 4-hidden-layer ReLU network trained on four MNIST digits grouped into two higher-order categories of even vs. odd. The values of the kernel are small since we take relatively small σ (SI Appendix, 3).CC BY · PubMed Central
Fig. 7. (A) Signal as a function of hidden layer depth l. For ReLU networks in the thermodynamic limit, signal increases with layer depth (blue). For linear networks (yellow, purple) and ReLU networks in the infinite width limit (red), signal remains unchanged across l. Error bars are across all distinct pairs of manifolds/digits. (B) Dimension as a function of l. In the infinite width limit, dimension remains constant with l in linear networks (purple) and increases with l in ReLU networks (red). In the thermodynamic limit, dimension decreases in linear networks (yellow), and is non-monotonic in ReLU networks (blue), similar to Fig. 3C. Error bars are across all manifolds/digits.
Fig. 7. (A) Signal as a function of hidden layer depth l. For ReLU networks in the thermodynamic limit, signal increases with layer depth (blue). For linear networks (yellow, purple) and ReLU networks in the infinite width limit (red), signal remains unchanged across l. Error bars are across all distinct pairs of manifolds/digits. (B) Dimension as a function of l. In the infinite width limit, dimension remains constant with l in linear networks (purple) and increases with l in ReLU networks (red). In the thermodynamic limit, dimension decreases in linear networks (yellow), and is non-monotonic in ReLU networks (blue), similar to Fig. 3C. Error bars are across all manifolds/digits.CC BY · PubMed Central
Fig. 8. (A) Comparison of the generalization error dynamics between a network fully trained under Langevin dynamics (Eq. 14, shown in red), and a network with a frozen at time t0 in the diffusive learning stage, and Wt randomly drifting afterward (shown in blue). τ denotes the difference between the current time t and t0. (B) Both the temporal correlation of the top right singular vector (ρu(τ)) and the correlation between u(t) and Y (ρu,Y) remain close to 1, representing the constant alignment between the Top Right singular vector of the representation and the training labels. Temporal correlation of the Top Left singular vector (ρv(τ)) gradually decreases with the time difference τ, representing the random drift in the feature space.
Fig. 8. (A) Comparison of the generalization error dynamics between a network fully trained under Langevin dynamics (Eq. 14, shown in red), and a network with a frozen at time t0 in the diffusive learning stage, and Wt randomly drifting afterward (shown in blue). τ denotes the difference between the current time t and t0. (B) Both the temporal correlation of the top right singular vector (ρu(τ)) and the correlation between u(t) and Y (ρu,Y) remain close to 1, representing the constant alignment between the Top Right singular vector of the representation and the training labels. Temporal correlation of the Top Left singular vector (ρv(τ)) gradually decreases with the time difference τ, representing the random drift in the feature space.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 75 references

Research works printed but not linked: 5

  1. L. Petrini F. Cagnetta U. M. Tomasini A. Favero M. Wyart How deep neural networks learn compositional data: The random hierarchy model. arXiv [Preprint] (2023). http://arxiv.org/abs/2307.02129 (Accessed 28 November 2023). · arXiv
  2. C. Gerbelot E. Troiani F. Mignacco F. Krzakala L. Zdeborova Rigorous dynamical mean field theory for stochastic gradient descent methods. arXiv [Preprint] (2022). http://arxiv.org/abs/2210.06591 (Accessed 28 November 2023). · arXiv
  3. S. Ariosto . Statistical mechanics of deep learning beyond the infinite-width limit. arXiv [Preprint] (2022). http://arxiv.org/abs/2209.04882 (Accessed 28 November 2023). · arXiv
  4. S. Dubova Y. M. Lu B. McKenna H. T. Yau Universality for the global spectrum of random inner-product kernel matrices in the polynomial regime. arXiv [Preprint] (2023). http://arxiv.org/abs/2310.18280 (Accessed 28 November 2023). · arXiv
  5. E. Sezener . A rapid and efficient learning rule for biological neural circuits. BioRxiv [Preprint] (2021). https://www.biorxiv.org/content/10.1101/2021.03.10.434756v1 (Accessed 28 November 2023). · doi:10.1101/2021.03.10.434756v1

Cited by

See all 37

Metadata

Authors
Li, Q., Sorscher, B., & Sompolinsky, H.
Affiliations
  • Harvard University (US)
  • Stanford University (US)
Corresponding author
  • Li, Qianyi
  • Sorscher, Ben
  • Sompolinsky, Haim
Venue
Proceedings of the National Academy of Sciences
Published
2024-06-24
Volume
121
Issue
27
Pages
e2311805121-e2311805121
ISSN
  • 0027-8424 (ISSN-L)
  • 1091-6490
OpenAlex
W4399940446
Cited by
27
References
64
Citations by year
2026: 4 · 2025: 19 · 2024: 4
FWCI
0.93
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
DOD | USN | Office of Naval Research (No.N0014-23-1-2051)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
Keywords
generalization, neural manifolds, representational geometry, deep neural network, out-of-distribution generalization, visual object recognition, thermodynamic limit, wide neural networks, representational drift, mental representation
MeSH terms
Animals, Artificial Intelligence, Brain, Cognition, Generalization, Psychological, Humans, Models, Neurological, Neural Networks, Computer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Representations and generalization in artificial and brain neural networks. https://app.cochat.ai/papers/10.1073/pnas.2311805121
The paper
Li, Q., Sorscher, B., & Sompolinsky, H. (2024). Representations and generalization in artificial and brain neural networks. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.2311805121

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.