CoChat
Sign Up
Cited by 104Memory and Neural MechanismsMemory Processes and InfluencesNeural dynamics and brain function

The DOI matches its Crossref record; 124 of the 124 references it cites were found in a registry or an index.

Organizing memories for generalization in complementary learning systems

Open access
Sun, Weinan, Advani, Madhu, Spruston, Nelson, Saxe, Andrew, Fitzgerald, James E.·2023·Nature Neuroscience
DOI 10.1038/s41593-023-01382-9PMC PMC10400413

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Memorization and generalization are complementary cognitive processes that jointly promote adaptive behavior. For example, animals should memorize safe routes to specific water sources and generalize from these memories to discover environmental features that predict new ones. These functions depend on systems consolidation mechanisms that construct neocortical memory traces from hippocampal precursors, but why systems consolidation only applies to a subset of hippocampal memories is unclear. Here we introduce a new neural network formalization of systems consolidation that reveals an overlooked tension—unregulated neocortical memory transfer can cause overfitting and harm generalization in an unpredictable world. We resolve this tension by postulating that memories only consolidate when it aids generalization. This framework accounts for partial hippocampal–cortical memory transfer and provides a normative principle for reconceptualizing numerous observations in the field. Generalization-optimized systems consolidation thus provides new insight into how adaptive behavior benefits from complementary learning systems specialized for memorization and generalization.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41593-023-01382-9

Cites

124 of 124 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Neural network model of systems consolidation. a, Our theoretical framework assumes that the neocortex extracts and encodes environmental relationships within the weights between distributed neocortical neurons in a process mediated by hippocampal reactivation. b, Individual experiences or memory reactivations are represented as columns of neuronal activations. We color the output neuron in red, and its activity is determined by the purple input neurons that are connected to it. Neurons that are not connected to the output neuron are colored gray. This example is for illustrative purposes, and an input neuron in one relationship could be an output neuron in another relationship. c, Learning modifies weights between the input and output neurons to reproduce each past experience. d, Cartoon of the teacher–student–notebook formalism; subscripts ‘i’ and ‘o’ refer to input and output layers. e, Neural network model architecture used in most simulations, unless otherwise noted. The teacher is a linear, shallow network with fixed weights that transforms an N-dimensional input into a scalar y, with a noise term ε added to vary the signal-to-noise ratio of the teacher. The student is typically a size-matched network to the teacher, with trainable weights w. The notebook is a Hopfield network that is bidirectionally connected to the student that serves as a one-shot learning module for memory encoding and replay (see Methods for details). f–k, Stages of learning and inferences in the model. The student is activated by each of the teacher-generated examples while the notebook encodes this example through one-shot Hebbian plasticity (f). The notebook can reactivate the encoded examples offline and reactivate the student (g). The notebook can reactivate previously encoded memories offline to induce memory recall in the student (h) and drive student learning (i). The student can use either the notebook or internal weights for inference (j and k). T, teacher; S, student; N, notebook.
Fig. 1 Neural network model of systems consolidation. a, Our theoretical framework assumes that the neocortex extracts and encodes environmental relationships within the weights between distributed neocortical neurons in a process mediated by hippocampal reactivation. b, Individual experiences or memory reactivations are represented as columns of neuronal activations. We color the output neuron in red, and its activity is determined by the purple input neurons that are connected to it. Neurons that are not connected to the output neuron are colored gray. This example is for illustrative purposes, and an input neuron in one relationship could be an output neuron in another relationship. c, Learning modifies weights between the input and output neurons to reproduce each past experience. d, Cartoon of the teacher–student–notebook formalism; subscripts ‘i’ and ‘o’ refer to input and output layers. e, Neural network model architecture used in most simulations, unless otherwise noted. The teacher is a linear, shallow network with fixed weights that transforms an N-dimensional input into a scalar y, with a noise term ε added to vary the signal-to-noise ratio of the teacher. The student is typically a size-matched network to the teacher, with trainable weights w. The notebook is a Hopfield network that is bidirectionally connected to the student that serves as a one-shot learning module for memory encoding and replay (see Methods for details). f–k, Stages of learning and inferences in the model. The student is activated by each of the teacher-generated examples while the notebook encodes this example through one-shot Hebbian plasticity (f). The notebook can reactivate the encoded examples offline and reactivate the student (g). The notebook can reactivate previously encoded memories offline to induce memory recall in the student (h) and drive student learning (i). The student can use either the notebook or internal weights for inference (j and k). T, teacher; S, student; N, notebook.CC BY · PubMed Central
Fig. 2 The predictability of experience controls the dynamics of systems consolidation. a–h, Dynamics of student generalization error, student memorization error, notebook generalization error and notebook memorization error when optimizing for student memorization (a, c, e and g) or generalization (b, d, f and h) performance. The student’s input dimension is N = 100, and the number of patterns stored in the notebook is P = 100 (all encoded at epoch = 1; epochs in the x axis correspond to time passage during systems consolidation). The notebook contains M = 2,000 units, with a sparsity a = 0.05. During each epoch, 100 patterns are randomly sampled from the P stored patterns for reactivating and training the student. The student’s learning rate is 0.015. Teachers differed in their levels of predictability (a and b, SNR = ∞; c and d, SNR = 4; e and f, SNR = 0.05; g and h, SNR ranges from 2−4 to 24). i–n, Methods for regulating consolidation. i, Using a validation set to estimate optimal early stopping time (SNR = 4, P = N = 100, 10% of P are used as validation set and not used for training). Filled red dot marks the generalization error at the optimal early stopping time (optimal ES), and dashed red dot marks the generalization error at the early stopping time estimated by the validation set (estimated ES). The vertical gray dashed line marks the estimated early stopping time. j, Generalization errors at optimal (solid red lines) vs estimated early stopping time (dashed red lines), as a function of the validation set fraction, SNR and α (P/N). The blue shading indicates the validation set fraction from 10% to 20%. k, Illustration of maximum likelihood estimation (MLE; Supplementary Information Section 9.2). l, MLE predicts SNR well from teacher-generated data. m, Initial learning speed monotonically increases as a function of SNR. n, Initial learning speed serves as a good feature for estimating true SNR in numerical simulations (P = N = 1,000).
Fig. 2 The predictability of experience controls the dynamics of systems consolidation. a–h, Dynamics of student generalization error, student memorization error, notebook generalization error and notebook memorization error when optimizing for student memorization (a, c, e and g) or generalization (b, d, f and h) performance. The student’s input dimension is N = 100, and the number of patterns stored in the notebook is P = 100 (all encoded at epoch = 1; epochs in the x axis correspond to time passage during systems consolidation). The notebook contains M = 2,000 units, with a sparsity a = 0.05. During each epoch, 100 patterns are randomly sampled from the P stored patterns for reactivating and training the student. The student’s learning rate is 0.015. Teachers differed in their levels of predictability (a and b, SNR = ∞; c and d, SNR = 4; e and f, SNR = 0.05; g and h, SNR ranges from 2−4 to 24). i–n, Methods for regulating consolidation. i, Using a validation set to estimate optimal early stopping time (SNR = 4, P = N = 100, 10% of P are used as validation set and not used for training). Filled red dot marks the generalization error at the optimal early stopping time (optimal ES), and dashed red dot marks the generalization error at the early stopping time estimated by the validation set (estimated ES). The vertical gray dashed line marks the estimated early stopping time. j, Generalization errors at optimal (solid red lines) vs estimated early stopping time (dashed red lines), as a function of the validation set fraction, SNR and α (P/N). The blue shading indicates the validation set fraction from 10% to 20%. k, Illustration of maximum likelihood estimation (MLE; Supplementary Information Section 9.2). l, MLE predicts SNR well from teacher-generated data. m, Initial learning speed monotonically increases as a function of SNR. n, Initial learning speed serves as a good feature for estimating true SNR in numerical simulations (P = N = 1,000).CC BY · PubMed Central
Fig. 3 Go-CLS mirrors memory research findings. a–d, Memorization (a and b) and generalization (c and d) scores for the integrated student–notebook system as a function of time and SNR, when optimized for student memorization (a and c) or generalization (b and d). Memory and generalization scores are translated from respective error values by score = (E0 − Et)/E0, where E0 and Et are the generalization or memory errors of a zero-weight student and a trained student at epoch = t, respectively. The effect of notebook lesion on memory performance (cyan lines and open circles, open circles simply demarcate the cyan lines at hypothetical ‘recent’ and ‘remote’ timepoints) depended on optimization objective and time (a and b). e, Go-CLS can reproduce a diversity of retrograde amnesia curves (see Methods for model details). f, Memory and generalization scores are positively correlated after notebook lesioning.
Fig. 3 Go-CLS mirrors memory research findings. a–d, Memorization (a and b) and generalization (c and d) scores for the integrated student–notebook system as a function of time and SNR, when optimized for student memorization (a and c) or generalization (b and d). Memory and generalization scores are translated from respective error values by score = (E0 − Et)/E0, where E0 and Et are the generalization or memory errors of a zero-weight student and a trained student at epoch = t, respectively. The effect of notebook lesion on memory performance (cyan lines and open circles, open circles simply demarcate the cyan lines at hypothetical ‘recent’ and ‘remote’ timepoints) depended on optimization objective and time (a and b). e, Go-CLS can reproduce a diversity of retrograde amnesia curves (see Methods for model details). f, Memory and generalization scores are positively correlated after notebook lesioning.CC BY · PubMed Central
Fig. 4 Normative benefits of complementary learning systems for generalization. a–c, Schematics illustrating learning systems that can use both the student and the notebook (a), only the student’s weights (b) and only the notebook weights (c) for inference. In machine-learning terminology, these systems implement batch learning, online learning and nearest-neighbor regression. d, Generalization error as a function of normalized data quantity (or α, defined as α = P/N) for each learning system (SNR = 1,000); dashed vertical line indicates α = 1. e, Advantage of Go-CLS over optimal online learning as a function of SNR and normalized data quantity, measured by the difference in generalization error. f, Generalization error as a function of normalized data quantity for the combined system, learning either through Go-CLS or standard systems consolidation (SNR = 2.5). g, Severity of overfitting, measured by the difference in generalization error between standard systems consolidation and Go-CLS.
Fig. 4 Normative benefits of complementary learning systems for generalization. a–c, Schematics illustrating learning systems that can use both the student and the notebook (a), only the student’s weights (b) and only the notebook weights (c) for inference. In machine-learning terminology, these systems implement batch learning, online learning and nearest-neighbor regression. d, Generalization error as a function of normalized data quantity (or α, defined as α = P/N) for each learning system (SNR = 1,000); dashed vertical line indicates α = 1. e, Advantage of Go-CLS over optimal online learning as a function of SNR and normalized data quantity, measured by the difference in generalization error. f, Generalization error as a function of normalized data quantity for the combined system, learning either through Go-CLS or standard systems consolidation (SNR = 2.5). g, Severity of overfitting, measured by the difference in generalization error between standard systems consolidation and Go-CLS.CC BY · PubMed Central
Fig. 5 Many forms of unpredictability demand regulated systems consolidation. a, The student–notebook learning system. b–d, Example teachers with unpredictable elements. b, A teacher that linearly transforms inputs into noisy outputs. c, A teacher that applies a nonlinear activation function at the output unit and cannot be fully predicted by a linear student. d, A teacher that only partially reveals the relevant inputs to the learning system. e,f, Varying predictability within the three different teachers all lead to quantitatively similar learning dynamics (complex teacher implements a sine function at the output unit, see Methods for simulation details). g–i, The degree of predictability can vary in many ways. For example, the same inputs can differentially predict various outputs (g), features can cross-predict each other with varying levels of predictability (h) and different learning systems could attend to different teacher features to predict the same output (i). j, Cartoon illustrating a child’s experience at a lake with her father. k, Cartoon illustrating conceptual differences between what is consolidated in standard systems consolidation and Go-CLS.
Fig. 5 Many forms of unpredictability demand regulated systems consolidation. a, The student–notebook learning system. b–d, Example teachers with unpredictable elements. b, A teacher that linearly transforms inputs into noisy outputs. c, A teacher that applies a nonlinear activation function at the output unit and cannot be fully predicted by a linear student. d, A teacher that only partially reveals the relevant inputs to the learning system. e,f, Varying predictability within the three different teachers all lead to quantitatively similar learning dynamics (complex teacher implements a sine function at the output unit, see Methods for simulation details). g–i, The degree of predictability can vary in many ways. For example, the same inputs can differentially predict various outputs (g), features can cross-predict each other with varying levels of predictability (h) and different learning systems could attend to different teacher features to predict the same output (i). j, Cartoon illustrating a child’s experience at a lake with her father. k, Cartoon illustrating conceptual differences between what is consolidated in standard systems consolidation and Go-CLS.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 3

  1. MacKay, D. J. C. Information Theory, Inference and Learning Algorithms (Cambridge University Press, 2003).
  2. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014).
  3. Pezeshki, M., Mitra, A., Bengio, Y. & Lajoie, G. Multi-scale feature learning dynamics: insights for double descent. In Proceedings of the 39th International Conference on Machine Learning (Eds. Chaudhuri, K. et al.) 17669–17690 (PMLR, 2022).

Cited by

Sorted by citations

See all 104

Related

See all

Metadata

Authors
Sun, W., Advani, M., Spruston, N., Saxe, A., & Fitzgerald, J. E.
Affiliations
  • Howard Hughes Medical Institute (US)
  • Harvard University (US)
  • Canadian Institute for Advanced Research (CA)
Corresponding author
  • Saxe, Andrew
  • Fitzgerald, James E.
ORCID
Venue
Nature Neuroscience
Publisher
Springer Science and Business Media LLC
Published
2023-07-20
Volume
26
Issue
8
Pages
1438-1448
ISSN
  • 1097-6256 (ISSN-L)
  • 1546-1726
DOI
10.1038/s41593-023-01382-9
PMC
PMC10400413
OpenAlex
W4384922492
Cited by
103
References
124
Citations by year
2026: 21 · 2025: 30 · 2024: 30 · 2023: 5
FWCI
12.85
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Neuroscience (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • UCL Discovery (University College London) (submittedVersion, cc-by)
Funders
  • Howard Hughes Medical Institute
  • Swartz Foundation
  • Wellcome Trust (216386/Z/19/Z)
  • Royal Society
  • Gatsby Charitable Foundation
  • Swartz Foundation CIFAR Azrieli Global Scholars program
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
Keywords
complementary learning systems, systems consolidation, memory generalization, hippocampus, neocortex, memory transfer, overfitting, neural network model, memory consolidation, adaptive behavior
MeSH terms
Memory Consolidation, Animals, Generalization, Psychological, Hippocampus, Learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review