Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
Open accessThe DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Abstract
Significance: Artificial neural networks can suffer from catastrophic forgetting, in which learning a new task causes the network to forget how to perform previous tasks. While previous studies have proposed various methods that can alleviate forgetting over small numbers ( ⩽ 10) of tasks, it is uncertain whether they can prevent forgetting across larger numbers of tasks. In this study, we propose a neuroscience-inspired scheme, called “context-dependent gating,” in which mostly nonoverlapping sets of units are active for any one task. Importantly, context-dependent gating has a straightforward implementation, requires little extra computational overhead, and when combined with previous methods to stabilize connection weights, can allow networks to maintain high performance across large numbers of sequentially presented tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
(2014) · Cited by 33,419
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
An Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803 · cited 1×
“In contrast, context-dependent signals in the brain, likely originating from areas such as the prefrontal cortex, project to various cortical areas and allow neural circuits to process incoming information in a task-dependent manner [16, 17, 18].”
BackgroundAdvances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
Li, Z., & Hoiem, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2773081 · Cited by 4,202
Dynamic predictions: Oscillations and synchrony in top–down processing
Engel, A. K., Fries, P., & Singer, W. (2001) · Nature Reviews Neuroscience · doi:10.1038/35094565 · Cited by 3,454 · cited 2×
“In contrast, context-dependent signals in the brain, likely originating from areas such as the prefrontal cortex, project to various cortical areas and allow neural circuits to process incoming information in a task-dependent manner [16, 17, 18].”
BackgroundNeuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,233
ImageNet: A large-scale hierarchical image database
Deng, J. H., Dong, W. F., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206848 · Cited by 2,714
Tononi, G., & Cirelli, C. (2014) · Neuron · doi:10.1016/j.neuron.2013.12.025 · Cited by 2,356 · cited 2×
“Consistent with this notion, neuroscience studies have proposed that diverse mechanisms operating at the systems (10, 11), morphological (2, 4), and transcriptional (12) levels all act to stabilize learned information, raising the question as to whether several complementary algorithms are required to support continua…”
BackgroundHigh-Dimensional Continuous Control Using Generalized Advantage Estimation
Schulman, J., Moritz, P., Levine, S., Jordan, M., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1506.02438 · Cited by 2,103
The Fine Structure of the Nervous System.
TENNYSON, V. M. (1970) · Archives of Neurology · doi:10.1001/archneur.1970.00480240094019 · Cited by 1,948
Continual Learning Through Synaptic Intelligence
Zenke, F., Poole, B., & Ganguli, S. (2017) · arXiv · doi:10.48550/arxiv.1703.04200 · Cited by 1,784
Dynamic reconfiguration of human brain networks during learning
Bassett, D. S., Wymbs, N. F., Porter, M. A., Mucha, P. J., Carlson, J. M., & Grafton, S. T. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1018985108 · Cited by 1,705 · cited 1×
“Strategies that encourage the development of a modular representation, as is believed to occur in the brain (25), might be required to fully implement continual and transfer learning (26).”
BackgroundMemory Aware Synapses: Learning What (not) to Forget
Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., & Tuytelaars, T. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_9 · Cited by 1,590
(1991) · Medical Entomology and Zoology · Cited by 1,512
Rusu, A. A., Rabinowitz, N. C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., & Hadsell, R. (2016) · arXiv · doi:10.48550/arxiv.1606.04671 · Cited by 1,307
Morphological Changes in Dendritic Spines Associated with Long-Term Synaptic Plasticity
Yuste, R., & Bonhoeffer, T. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.1071 · Cited by 1,269
PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning
Mallya, A., & Lazebnik, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00810 · Cited by 1,258
Rapid formation and selective stabilization of synapses for enduring motor memories
Xu, T., Yu, X., Perlik, A. J., Tobin, W. F., Zweig, J. A., Tennant, K., Jones, T., & Zuo, Y. (2009) · Nature · doi:10.1038/nature08389 · Cited by 1,193
Stably maintained dendritic spines are associated with lifelong memories
Yang, G., Pan, F., & Gan, W.-B. (2009) · Nature · doi:10.1038/nature08577 · Cited by 1,182 · cited 5×
“Particularly, skill acquisition and retention is associated with the creation and stabilization of dendritic spines (6, 7).”
BackgroundMethodInfluentialAn Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
Goodfellow, I. J., Mirza, M., Xiao, D., Courville, A., & Bengio, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6211 · Cited by 1,089
Rapid Actin-Based Plasticity in Dendritic Spines
Fischer, M., Kaech, S., Knutti, D., & Matus, A. (1998) · Neuron · doi:10.1016/s0896-6273(00)80467-5 · Cited by 905 · cited 2×
“rphology, and hence the functional connectivity of the associated synapses, can be either dynamic or stable, with lifetimes ranging from seconds and years (Kasai et al., 2003; Yoshihara et al., 2009; Fischer et al., 1998). Particularly, skill acquisition and retention is associated with the creation and stabilization…”
BackgroundResultStructure-stability-function relationships of dendritic spines
Kasai, H., Matsuzaki, M., Noguchi, J., Yasumatsu, N., & Nakahara, H. (2003) · Trends in Neurosciences · doi:10.1016/s0166-2236(03)00162-0 · Cited by 847
Neural constraints on learning
Sadtler, P. T., Quick, K. M., Golub, M. D., Chase, S. M., Ryu, S. I., Tyler-Kabara, E. C., Yu, B. M., & Batista, A. P. (2014) · Nature · doi:10.1038/nature13665 · Cited by 713 · cited 1×
“Although it is unknown whether a similar mechanism occurs in the brain, the idea that a downstream cortical network could control the parameter space allowed for learning in an upstream network (Sadtler et al., 2014) is an appealing idea.”
BackgroundLi, Z., & Hoiem, D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_37 · Cited by 659
PathNet: Evolution Channels Gradient Descent in Super Neural Networks
Fernando, C., Banarse, D., Blundell, C., Zwólš, Y., Ha, D., Rusu, A. A., Pritzel, A., & Wierstra, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.08734 · Cited by 591
Task representations in neural networks trained to perform many cognitive tasks
Yang, G. R., Joglekar, M. R., Song, H. F., Newsome, W. T., & Wang, X.-J. (2019) · Nature Neuroscience · doi:10.1038/s41593-018-0310-2 · Cited by 549
Branch-specific dendritic Ca2+ spikes cause persistent synaptic plasticity
Cichon, J., & Gan, W.-B. (2015) · Nature · doi:10.1038/nature14251 · Cited by 500
Variational Continual Learning
Nguyen, C. V., Li, Y., Bui, T. D., & Turner, R. E. (2017) · arXiv · doi:10.48550/arxiv.1710.10628 · Cited by 491 · cited 3×
“To our knowledge, these numbers of tasks are an order of magnitude greater compared to previously published work (Zenke et al., 2017; Kirkpatrick et al., 2017; He and Jaeger, 2017; Nguyen et al., 2017; Serrà et al., 2018).”
MethodResultVariational Continual Learning
Turner, R., Bui, T., Li, Y., & Cuong, N. (2019) · Apollo - University of Cambridge Repository · doi:10.17863/cam.35471 · Cited by 438
Parallel processing by cortical inhibition enables context-dependent behavior
Kuchibhotla, K. V., Gill, J. V., Lindsay, G. W., Papadoyannis, E. S., Field, R. E., Sten, T. A. H., Miller, K. D., & Froemke, R. C. (2017) · Nature Neuroscience · doi:10.1038/nn.4436 · Cited by 364
Engaging in an auditory task suppresses responses in auditory cortex
Otazu, G. H., Tai, L.-H., Yang, Y., & Zador, A. M. (2009) · Nature Neuroscience · doi:10.1038/nn.2306 · Cited by 308
Compartmentalization of GABAergic inhibition by dendritic spines
Chiu, C. Q., Lur, G., Morse, T. M., Carnevale, N. T., Ellis-Davies, G. C. R., & Higley, M. J. (2013) · Science · doi:10.1126/science.1234274 · Cited by 307
Johnston, K., Levin, H. M., Koval, M. J., & Everling, S. (2007) · Neuron · doi:10.1016/j.neuron.2006.12.023 · Cited by 303
Dendritic spine formation and stabilization
Yoshihara, Y., De Roo, M., & Muller, D. (2009) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2009.05.013 · Cited by 292 · cited 5×
“…have proposed that diverse mechanisms operating at the systems (Cichon and Gan, 2015; Tononi and Cirelli, 2014), morphological (Kasai et al., 2003; Yoshihara et al., 2009), and transcriptional (Kukushkin and Carew, 2017) levels all act to stabilize learned information, raising the question as to…”
BackgroundInfluentialOne-shot Learning with Memory-Augmented Neural Networks
Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., & Lillicrap, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.06065 · Cited by 278
One-shot learning by inverting a compositional causal process
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2013) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 223
Overcoming catastrophic forgetting with hard attention to the task
Serrà, J., Surís, D., Miron, M., & Karatzoglou, A. (2018) · arXiv · doi:10.48550/arxiv.1801.01423 · Cited by 213 · cited 2×
“We tested the Hard Attention to Task (HAT) method (33) and found that it could learn 100 sequential MNIST permutations with a mean accuracy of 0 20 40 60 80 100 Task number 0 0.”
MethodKukushkin, N. V., & Carew, T. J. (2017) · Neuron · doi:10.1016/j.neuron.2017.05.029 · Cited by 83
Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks
Velez, R., & Clune, J. (2017) · PLOS ONE · doi:10.1371/journal.pone.0187736 · Cited by 41
Overcoming Catastrophic Interference by ConceptorsPreprint
He, X., & Jaeger, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.04853 · Cited by 15
Yang, G. R., Song, H., Newsome, W. T., & Wang, X. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/183632 · Cited by 13
Cited by
Three types of incremental learning
van de Ven, G. M., Tuytelaars, T., & Tolias, A. S. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00568-3 · Cited by 665
Brain-inspired replay for continual learning with artificial neural networks
van de Ven, G. M., Siegelmann, H. T., & Tolias, A. S. (2020) · Nature Communications · doi:10.1038/s41467-020-17866-2 · Cited by 474
If deep learning is the answer, what is the question?
Saxe, A., Nelli, S., & Summerfield, C. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-00395-8 · Cited by 397
“These tools are more effective when coupled with a gating process that overtly earmarks neural subspaces for new learning, in a way that resembles top-down attention in the primate neocortex 131,132 .”
BackgroundArtificial Neural Networks for Neuroscientists: A Primer
Yang, G. R., & Wang, X.-J. (2020) · Neuron · doi:10.1016/j.neuron.2020.09.005 · Cited by 376
Thalamocortical Circuit Motifs: A General Framework
Halassa, M. M., & Sherman, S. M. (2019) · Neuron · doi:10.1016/j.neuron.2019.06.005 · Cited by 322
Metadata
- Authors
- Masse, N. Y., Grant, G. D., & Freedman, D. J.
- Venue
- Proceedings of the National Academy of Sciences
- Published
- 2018-10-12
- Volume
- 115
- Issue
- 44
- Pages
- E10467-E10475
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- OpenAlex
- W2787295326
- Cited by
- 255
- References
- 47
- FWCI
- 14.99
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.