Loss of plasticity in deep continual learning
Open accessPublished version of a preprint
The DOI matches its Crossref record; 111 of the 111 references it cites were found in a registry or an index; 17 of the 36 sources its claims cite were checked for support.
Abstract
Artificial neural networks, deep-learning methods and the backpropagation algorithm 1 form the foundation of modern machine learning and artificial intelligence. These methods are almost always used in two phases, one in which the weights of the network are updated and one in which the weights are held constant while the network is used or evaluated. This contrasts with natural learning and many applications, which require continual learning. It has been unclear whether or not deep learning methods work in continual learning settings. Here we show that they do not—that standard deep-learning methods gradually lose plasticity in continual-learning settings until they learn no better than a shallow network. We show such loss of plasticity using the classic ImageNet dataset and reinforcement-learning problems across a wide range of variations in the network and the learning algorithm. Plasticity is maintained indefinitely only by algorithms that continually inject diversity into the network, such as our continual backpropagation algorithm, a variation of backpropagation in which a small fraction of less-used units are continually and randomly reinitialized. Our results indicate that methods based on gradient descent are not enough—that sustained deep learning requires a random, non-gradient component to maintain variability and plasticity.
Overview
CoChat Summary
TL;DR
Standard deep-learning methods gradually lose plasticity in continual-learning settings, eventually learning no better than shallow or linear networks across supervised and reinforcement-learning tasks. Continual backpropagation, which selectively reinitializes a small fraction of least-used units, maintains plasticity apparently indefinitely in these experiments.
Key findings
- In Continual ImageNet, unmodified backpropagation learned up to 88% correct on early tasks but by the 2,000th task had lost substantial plasticity for all tested step sizes, ending near or below a linear baseline; L2 regularization, Shrink and Perturb, and continual backpropagation maintained plasticity, with continual backpropagation best.
- In class-incremental CIFAR-100, the base deep-learning system was initially better than retraining from scratch but after 40 classes showed increasing loss of plasticity; by 100 classes its accuracy was 5% lower than the retrained network. Continual backpropagation eliminated this loss and reached 76.13% final accuracy on all 100 classes.
- In reinforcement learning, standard PPO on the non-stationary ant performed well for the first 2 million steps but worsened after friction changes; on the stationary ant, PPO improved for about 3 million steps then collapsed, failing every episode after 20 million steps. PPO with continual backpropagation plus L2 regularization maintained or improved performance.
- Loss of plasticity correlated with more dead/dormant units, growing average weight magnitude, and lower effective rank; Adam, Dropout, and normalization often worsened plasticity, whereas L2 regularization and Shrink and Perturb reduced it.
Methods
- Supervised continual-learning experiments included Continual ImageNet (1,000 classes, 700 images each, 600 train/100 test, binary class pairs, downsampled 32×32 images, convolutional network, SGD with momentum 0.9, 250 epochs per task, mini-batch 100, 30 runs), class-incremental CIFAR-100 (100 classes, 600 images each, 450/50/100 train/validation/test split, increments of 5 classes, 18-layer ResNet, 200 epochs per increment, 30 runs), Online Permuted MNIST (60,000 28×28 images, 800 tasks, three-hidden-layer feed-forward networks with 2,000 units per layer, single pass), and Slowly-Changing Regression (six activation functions, 3 million examples, 100 runs).
- Reinforcement-learning experiments used PPO on Ant-v3, Hopper-v3, and Walker-v3; the non-stationary ant changed friction every 2 million time steps, the stationary ant ran for 50 million steps, and some experiments ran up to 100 million steps. Policy and value networks each had two hidden layers of 256 units and were trained with Adam; non-stationary results averaged 100 runs and stationary results 30 runs.
- Continual backpropagation combines standard backpropagation with selective reinitialization of low-contribution-utility mature units. Utility is a running average with decay rate η = 0.99; outgoing weights of reinitialized units are set to zero; replacement rate ρ and maturity threshold m control reinitialization. For example, CIFAR-100 used ρ = 10⁻⁵ and m = 1,000; ImageNet used m = 100; reinforcement learning used ρ = 10⁻⁴, m = 10⁴, and weight decay 10⁻⁴.
Limitations
- Stated: reinforcement-learning demonstrations are less definitive than supervised ones because of greater stochasticity, algorithm and setting variety, and confounding between an algorithm’s learning ability and its ability to generate informative data; more runs and experimental work are needed.
- Stated: continual backpropagation’s utility measure is heuristic and not global, it does not address forgetting, and benchmarks focus on plasticity rather than forgetting or speed of learning; L2 regularization and Shrink and Perturb are sensitive to hyperparameters, and the ReDo comparison is preliminary.
- Evident from the text: Continual ImageNet resets output heads at task changes, giving privileged information about task boundaries and not used in other experiments; a systematic study with large language models was not possible because a single training run would cost millions of dollars.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures









Verification
Claims the paper makes
9 Valid · 8 Not valid · 5 Not checked · 167 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Another existing method that reduced loss of plasticity is Shrink and Perturb11 (orange line in Figs. 1 and 2).”
§ Loss of plasticity in deep continual learning > Maintaining plasticity
What the sources say
- Supportedcites [11] Ash, J. & Adams, R. P. On warm-starting neural network training. Adv. Neural Inf. Process. Syst. 33, 3884–3894 (2020). source
Section 4 (p. 6) reports that Shrink and Perturb remedies the warm-start problem when a network is trained on newly supplied samples: it closes the generalization gap while speeding training. This supports the citing sentence’s characterization of the method as reducing loss of plasticity. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“This is different from the type of loss of plasticity in which the system can keep optimizing new objectives but lose the ability to generalize11,12.”
§ Loss of plasticity in deep continual learning > Methods > Extended discussion
What the sources say
- Supportedcites [11] Ash, J. & Adams, R. P. On warm-starting neural network training. Adv. Neural Inf. Process. Syst. 33, 3884–3894 (2020). source
The Introduction (p. 1) says warm-starting does not damage training accuracy, yet the model performs worse on test samples. This supports the cited type of loss of plasticity: the network can fit the updated training objective while generalizing less well. (LLM, conf=0.90)
- Supportedcites [12] Berariu, T. et al. A study on the plasticity of neural networks. Preprint at https://arxiv.org/abs/2106.00042 (2021). source
The Introduction (p. 1) distinguishes two forms of loss of plasticity: one in which a model can still reduce training error but generalizes worse, and another in which it cannot optimize the new task. That directly supports the distinction attributed to this paper. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Lyle et al.24 showed that the standard values of β1 and β2 cause a large loss of plasticity.”
§ Loss of plasticity in deep continual learning > Methods > Details and further analysis in reinforcement learning
What the sources say
- Could not verifycites Lyle et al.24 Lyle, C. et al. Understanding plasticity in neural networks. In Proc. 40th International Conference on Machine Learning 23190–23211 (PMLR, 2023). sourceExperimental — accuracy not yet measured
Section 3.2 discusses mechanisms of plasticity loss, and the Additional information section reports using Adam, but neither attributes a large loss of plasticity to Adam’s standard β1 and β2 values. Searches for β1, β2, their typical values, and optimizer momentum found no passage testing or making that claim. The indexed passages provide no page numbers. (LLM, conf=0.72)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, the problem settings in these papers were offline as they had two separate phases, one for learning and the other for evaluation.”
§ Loss of plasticity in deep continual learning > Methods > Extended discussion
What the sources say
- Could not verifycites [120] Finn, C., Abbeel, P. & Levine, S. Model-agnostic meta-learning for fast adaptation of deep networks. In Proc. 34th International Conference on Machine Learning 1126–1135 (PMLR, 2017). sourceExperimental — accuracy not yet measured
Our copy of MAML contains no passage characterizing prior fast-adaptation works as 'offline' with two separate learning and evaluation phases, and it never frames the topic as adapting to changes in a data stream. Its Related Work (Sec. 4, p. 4-5) distinguishes prior methods on other grounds entirely — learned update rules/extra parameters (Andrychowicz et al., 2016; Ravi & Larochelle, 2017), architecture constraints such as recurrent or Siamese networks (Santoro et al., 2016; Koch, 2015), test-time nonparametric components (Vinyals et al., 2016), and pretrained-initialization approaches — while the two-stage meta-training/then-adaptation structure appears only as the paper's own problem set-up (Sec. 2.1, p. 2), not as a critique of other papers. Searched: 'offline problem setting / separate learning and evaluation phases', 'data stream / online continual adaptation', 'quickly adapting to changes'. Since our copy neither states nor contradicts the attributed point, it cannot be verified. (LLM, conf=0.68)
- Not checked — no full text of the sourcecites [121] Wang, Y.-X., Ramanan, D. & Hebert, M. Growing a brain: fine-tuning by increasing model capacity. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2471–2480 (IEEE, 2017). source
- Partly supportedcites [122] Nagabandi, A. et al. Learning to adapt in dynamic, real-world environments through meta-reinforcement learning. In Proc. 7th International Conference on Learning Representations (ICLR, 2019). sourceExperimental — accuracy not yet measured
Section 4 (p. 5) distinguishes a train-time algorithm from a test-time algorithm, supporting the claim that there are separate phases. But the test-time algorithm adapts the model at every timestep, and Section 6.4 (p. 9) explicitly describes this as online adaptation. Calling the paper's problem setting offline is therefore not supported. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These works focused on deep reinforcement learning systems that use large replay buffers.”
§ Loss of plasticity in deep continual learning > Methods > Details and further analysis in reinforcement learning
What the sources say
- Not checked yetcites [25] Sokar, G., Agarwal, R., Castro, P. S. & Evci, U. The dormant neuron phenomenon in deep reinforcement learning. In Proc. 40th International Conference on Machine Learning 32145–32168 (PMLR, 2023). source
- Supportedcites [83] Nikishin, E. et al. Deep reinforcement learning with plasticity injection. Adv. Neural Inf. Process. Syst. 36, 1–18 (2023). source
The Experimental Setup describes experiments with a Double DQN agent trained on 57 Atari games. The Additional information section says that the Atari 200M setting uses a replay buffer holding 4 million frames, supporting the description of the systems studied. The paper distinguishes this from the plasticity-injection method itself, which does not rely on a buffer. (LLM, conf=0.90)
- Not matchedcites [84] D’Oro, P. et al. Sample-efficient reinforcement learning by breaking the replay ratio barrier. In Proc. 11th International Conference on Learning Representations (ICLR, 2023).
Its entry has no title and year that we can search by.
- Partly supportedcites [85] Schwarzer, M. et al. Bigger, better, faster: human-level Atari with human-level efficiency. In Proc. 40th International Conference on Machine Learning 30365–30380 (PMLR, 2023). sourceExperimental — accuracy not yet measured
The Background and Method sections describe deep reinforcement learning agents that use replay buffers and periodic resets; the Discussion links resets to network plasticity. But the paper emphasizes large replay ratios—the number of learning updates relative to new experience—not large replay buffers. Our copy does not establish that the buffers themselves are large. (LLM, conf=0.88)
- Supportedcites [86] Lee, H. et al. PLASTIC: improving input and label plasticity for sample efficient reinforcement learning. Adv. Neural Inf. Process. Syst. 36, 1–26 (2023). source
Section 4.3 describes PLASTIC as a plasticity-preserving approach for off-policy reinforcement learning. The hyperparameter tables in “D Broader Impact” and “Additional information” list replay-buffer capacities of 100,000 for Atari and 1,000,000 for DMC, supporting the description of these systems as using large replay buffers. (LLM, conf=0.88)
- Not matchedcites [87] Delfosse, Q., Schramowski, P., Mundt, M., Molina, A. & Kersting, K. Adaptive rational activations to boost deep reinforcement learning. In Proc. 12th International Conference on Learning Representations (ICLR, 2024).
Its entry has no title and year that we can search by.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
88 more claims
Create a free account to see all 189 claims and to verify the 1 reference still open.
- The quote only partly states this claim. 1
- It may state two separate claims as one. 67
- Probably not a substantial claim. 39
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Loss of plasticity in deep continual learning > Abstract
“Plasticity is maintained indefinitely only by algorithms that continually inject diversity into the network, such as our continual backpropagation algorithm, a variation of backpropagation in which a small fraction of less-used units are continually and randomly reinitialized.”
§ Loss of plasticity in deep continual learning > Plasticity loss in supervised learning
“the continual backpropagation, L2 regularization and Shrink and Perturb algorithms maintain plasticity, apparently indefinitely (c).”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Loss of plasticity in deep continual learning > Methods > Class-incremental CIFAR-100
“In Fig. 2d, we plot the stable rank of the representation in the penultimate layer of the network and the percentage of dead units in the full network.”
§ Loss of plasticity in deep continual learning > Plasticity loss in supervised learning
“c, The number of network units that are active less than 1% of the time increases rapidly for the base deep-learning system, but less so for Shrink and Perturb and continual backpropagation systems. d, A low stable rank means that the units of a network do not provide much diversity;”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Loss of plasticity in deep continual learning > Main
“continual backpropagation is inspired by a long history of methods for automatically generating and testing features, starting with Selfridge’s Pandemonium in 1959 (refs. 19,20,31–35).”
§ Loss of plasticity in deep continual learning > References
“31. Selfridge, O. G. Pandemonium: a paradigm for learning. In Mechanization of Thought Processes: Proceedings of a Symposium Held at the National Physical Laboratory 511–531 (Her Majesty’s Stationary Office, 1958).”
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“The residual networks of today are more widely used and produce better results than strictly layered networks 38 .”
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1×
“For example, a systematic study with large language models would not be possible today because just a single training run with one of these networks would require computation costing millions of dollars.”
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“We now use a computationally cheap problem based on the MNIST dataset 56 to test the generality of loss of plasticity across various conditions.”
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,002 · cited 1×
“The variation and selection is reminiscent of trial-and-error processes in evolution and behaviour 44–47 and has precursors in many earlier ideas, including Keifer–Wolfowitz methods 48 and restart methods 49 in engineering and feature-search methods in machine learning 31–35 , 50 .”
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764 · cited 6×
“The details of the specific instance of the Slowly-Changing Regression problem we use in this paper and the learning network used to predict its output are listed in Extended Data Table 4.”
InfluentialImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 1×
“Here we use ImageNet, a classic object-recognition test bed 36 , which played a pivotal role in the rise of deep learning 37 and is still influential today.”
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · cited 1×
“Artificial neural networks, deep-learning methods and the backpropagation algorithm 1 form the foundation of modern machine learning and artificial intelligence.”
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 1×
“Machine learning and artificial intelligence have made remarkable progress in the past decade, with landmark successes in natural-language processing 2,3 , biology 4 , game playing 5–8 and robotics 9,10 .”
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“Recent applications of deep learning such as GPT-3 [4] and DallE [5] were also trained on data as a single large batch.”
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · cited 1×
“On the other hand, PPO augmented with”
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1993.341010 · Cited by 14,265
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,630
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782 · cited 1×
“Altogether, these results, along with other extensive results in Methods, constitute substantial evidence of plasticity loss.”
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · cited 1×
“Such train-once settings were used in all the early successes of deep learning in speech recognition [1] and image classification [2].”
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 1×
“On the other hand, PPO augmented with”
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899 · cited 2×
“Loss of plasticity is different from catastrophic forgetting, which concerns poor performance on old examples even if they are not presented again 28–30 .”
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · cited 1×
“Altogether, these results, along with other extensive results in Methods, constitute substantial evidence of plasticity loss.”
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955 · cited 1×
“[79] claimed adding noise always worsened their network performance.”
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382 · cited 4×
“The effectiveness of continual backpropagation shows that the problem of plasticity loss is not inherent in artificial neural networks.”
InfluentialHan, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662
Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
Clevert, D.-A., Unterthiner, T., & Hochreiter, S. (2015) · arXiv · doi:10.48550/arxiv.1511.07289 · Cited by 4,513 · cited 1×
“For ELU and Swish, we used gain= 2 , as was done in the original papers 59,62 .”
On the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988 · cited 1×
“Class-incremental continual learning 39 involves sequentially adding new classes while testing on all classes seen so far.”
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770 · cited 2×
“It is difficult for current deep-learning methods to remember useful information as they tend to forget previously learned information [6, 7, 37].”
5th International Conference on Learning Representations (ICLR 17)
Havrylov, S., & Titov, I. (2017) · Cited by 3,503
Evolving Neural Networks through Augmenting Topologies
Stanley, K. O., & Miikkulainen, R. (2002) · Evolutionary Computation · doi:10.1162/106365602320169811 · Cited by 3,258
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
Training Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117
Zero-shot text-to-image generation
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., & Sutskever, I. (2021) · arXiv · doi:10.48550/arxiv.2102.12092 · Cited by 2,417
Searching for activation functions
Ramachandran, P., Zoph, B., & Le, Q. V. (2017) · arXiv · doi:10.48550/arxiv.1710.05941 · Cited by 2,335 · cited 2×
“In Slowly-Changing Regression, we study loss of plasticity for networks with six popular activation functions: sigmoid, tanh, ELU 59 , leaky ReLU 60 , ReLU 61 and Swish 62 .”
Blind variation and selective retentions in creative thought as in other knowledge processes
Campbell, D. T. (1960) · Psychological Review · doi:10.1037/h0040373 · Cited by 2,329
Catastrophic forgetting in connectionist networks
French, R. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01294-2 · Cited by 2,311 · cited 2×
“It is difficult for current deep-learning methods to remember useful information as they tend to forget previously learned information [6, 7, 37].”
Proceedings of IEEE International Conference on Computer Vision
(1995) · doi:10.1109/iccv.1995 · Cited by 2,280
Neural Networks: Tricks of the Trade: Second Edition
(2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8 · Cited by 2,116
Continual Learning Through Synaptic Intelligence
Zenke, F., Poole, B., & Ganguli, S. (2017) · arXiv · doi:10.48550/arxiv.1703.04200 · Cited by 1,784
Restart procedures for the conjugate gradient method
Powell, M. J. D. (1977) · Mathematical Programming · doi:10.1007/bf01593790 · Cited by 1,717
Learning dexterous in-hand manipulation
Andrychowicz, O. M., Baker, B., Chociej, M., Józefowicz, R., McGrew, B., Pachocki, J., Petron, A., Plappert, M., Powell, G., Ray, A., et al. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364919887447 · Cited by 1,643 · cited 2×
“Machine learning and artificial intelligence have made remarkable progress in the past decade, with landmark successes in natural-language processing 2,3 , biology 4 , game playing 5–8 and robotics 9,10 .”
IEEE Transactions on Neural Networks and Learning Systems
(2013) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2013.2264575 · Cited by 1,573
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
Frankle, J., & Carbin, M. (2018) · arXiv · doi:10.48550/arxiv.1803.03635 · Cited by 1,529
Gradient Episodic Memory for Continual Learning
Lopez-Paz, D., & Ranzato, M. (2017) · arXiv · doi:10.48550/arxiv.1706.08840 · Cited by 1,403
Rusu, A. A., Rabinowitz, N. C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., & Hadsell, R. (2016) · arXiv · doi:10.48550/arxiv.1606.04671 · Cited by 1,307
How Does Batch Normalization Help Optimization?
Santurkar, S., Tsipras, D., Ilyas, A., & Madry, A. (2018) · arXiv · doi:10.48550/arxiv.1805.11604 · Cited by 1,092
Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
Chaudhry, A., Dokania, P. K., Ajanthan, T., & Torr, P. H. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_33 · Cited by 1,091 · cited 1×
“[20] observed loss of plasticity in continual image classification problems.”
An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
Goodfellow, I. J., Mirza, M., Xiao, D., Courville, A., & Bengio, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6211 · Cited by 1,089
Dota 2 with Large Scale Deep Reinforcement Learning
OpenAI, :, Berner, C., Brockman, G., Chan, B., Cheung, V., Dębiak, P., Dennison, C., Farhi, D., Fischer, Q., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.06680 · Cited by 1,065
The Psychology of Learning and Motivation (Vol. 17)
D., C. P., & Bower, G. H. (1985) · The American Journal of Psychology · doi:10.2307/1422448 · Cited by 830
DeepStack: Expert-level artificial intelligence in heads-up no-limit poker
Moravčík, M., Schmid, M., Burch, N., Lisý, V., Morrill, D., Bard, N., Davis, T., Waugh, K., Johanson, M., & Bowling, M. (2017) · Science · doi:10.1126/science.aam6960 · Cited by 786
Understanding Batch NormalizationPreprint
Bjorck, N., Gomes, C. P., Selman, B., & Weinberger, K. Q. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.02375 · Cited by 777
Champion-level drone racing using deep reinforcement learning
Kaufmann, E., Bauersfeld, L., Loquercio, A., Müller, M., Koltun, V., & Scaramuzza, D. (2023) · Nature · doi:10.1038/s41586-023-06419-4 · Cited by 710 · cited 1×
“Machine learning and artificial intelligence have made remarkable progress in the past decade, with landmark successes in natural-language processing 2,3 , biology 4 , game playing 5–8 and robotics 9,10 .”
Three types of incremental learning
van de Ven, G. M., Tuytelaars, T., & Tolias, A. S. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00568-3 · Cited by 665 · cited 1×
“We use it here because it is the standard practice in deep continual learning for this type of problem in which the learning system has to learn a sequence of independent tasks 52 .”
Kaelbling, L. P. (1993) · doi:10.7551/mitpress/4168.001.0001 · Cited by 599 · cited 1×
“Continual backpropagation overcomes the limitation of previous work 34,35 Find the unit with smallest utility and record its index as r Reinitialize input weights: resample w l −1 [:, r ] from distribution d l Reinitialize output weights: set w l [ r ,”
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016) · arXiv · doi:10.48550/arxiv.1606.01540 · Cited by 582 · cited 1×
“The experiments presented in the main text were conducted using the Ant-v3 environment from OpenAI Gym 79 .”
Mocanu, D. C., Mocanu, E., Stone, P., Nguyen, P. H., Gibescu, M., & Liotta, A. (2018) · Nature Communications · doi:10.1038/s41467-018-04316-3 · Cited by 524 · cited 1×
“The idea of selective reinitialization is similar to the emerging idea of dynamic sparse training 110–112 .”
Cognitive systems based on adaptive algorithms
Holland, J. H., & Reitman, J. S. (1977) · ACM SIGART Bulletin · doi:10.1145/1045343.1045373 · Cited by 469
2015 IEEE Symposium Series on Computational Intelligence
(2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2479518 · Cited by 443
The State of Sparsity in Deep Neural Networks
Gale, T., Elsen, E., & Hooker, S. (2019) · arXiv · doi:10.48550/arxiv.1902.09574 · Cited by 440
Ellis, A. W., & Lambon Ralph, M. A. (2000) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037//0278-7393.26.5.1103 · Cited by 406 · cited 1×
“Ellis and Lambon Ralph [16], Zevin and Seidenberg, [17], and Bonin et al.”
Age of Acquisition Effects in Word Reading and Other Tasks
Zevin, J. D., & Seidenberg, M. S. (2002) · Journal of Memory and Language · doi:10.1006/jmla.2001.2834 · Cited by 374 · cited 1×
“Ellis and Lambon Ralph [16], Zevin and Seidenberg, [17], and Bonin et al.”
Learning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement Learning
Nagabandi, A., Clavera, I., Liu, S., Fearing, R. S., Abbeel, P., Levine, S., & Finn, C. (2018) · arXiv · doi:10.48550/arxiv.1803.11347 · Cited by 370 · cited 1×
“On the other hand, PPO augmented with”
Outracing champion Gran Turismo drivers with deep reinforcement learning
Wurman, P. R., Barrett, S., Kawamoto, K., MacGlashan, J., Subramanian, K., Walsh, T. J., Capobianco, R., Devlic, A., Eckert, F., Fuchs, F., et al. (2022) · Nature · doi:10.1038/s41586-021-04357-7 · Cited by 369 · cited 1×
“Machine learning and artificial intelligence have made remarkable progress in the past decade, with landmark successes in natural-language processing 2,3 , biology 4 , game playing 5–8 and robotics 9,10 .”
Using additive noise in back-propagation training
Holmstrom, L., & Koistinen, P. (1992) · IEEE Transactions on Neural Networks · doi:10.1109/72.105415 · Cited by 355 · cited 1×
“The main benefits of adding noise have been reported to be avoiding overfitting and improving training performance 123–125 .”
A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
Chrabąszcz, P., Loshchilov, I., & Hutter, F. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.08819 · Cited by 336
Why the Law of Effect will not Go Away*
DENNETT, D. C. (1975) · Journal for the Theory of Social Behaviour · doi:10.1111/j.1468-5914.1975.tb00350.x · Cited by 330
Dying ReLU and Initialization: Theory and Numerical Examples
Lu, L., Shin, Y., Su, Y., & Em Karniadakis, G. (2020) · Communications in Computational Physics · doi:10.4208/cicp.oa-2020-0165 · Cited by 281
Meta-Learning Representations for Continual LearningPreprint
Javed, K., & White, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12588 · Cited by 274
Learning Dexterous In-Hand Manipulation
OpenAI, Andrychowicz, M., Baker, B., Chociej, M., Jozefowicz, R., McGrew, B., Pachocki, J., Petron, A., Plappert, M., Powell, G., et al. (2018) · arXiv · doi:10.48550/arxiv.1808.00177 · Cited by 248
Online Continual Learning with Maximal Interfered RetrievalPreprint
Aljundi, R., Belilovsky, E., Tuytelaars, T., Charlin, L., Caccia, M., Lin, M., & Caccia, L. (2019) · arXiv (Cornell University) · Cited by 245
The influence of age of acquisition in word reading and other tasks: A never ending story?
Bonin, P., Barry, C., Méot, A., & Chalard, M. (2004) · Journal of Memory and Language · doi:10.1016/j.jml.2004.02.001 · Cited by 226 · cited 1×
“Loss of plasticity in artificial neural networks was first shown at the turn of the century in the psychology literature 13–15 , before the development of deep-learning methods.”
Implicit Regularization in Deep Learning May Not Be Explainable by NormsPreprint
Razin, N., & Cohen, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.06398 · Cited by 188
Growing a Brain: Fine-Tuning by Increasing Model Capacity
Wang, Y.-X., Ramanan, D., & Hebert, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.323 · Cited by 154 · cited 1×
“On the other hand, PPO augmented with”
Online Continual Learning with Maximally Interfered Retrieval
Aljundi, R., Caccia, L., Belilovsky, E., Caccia, M., Lin, M., Charlin, L., & Tuytelaars, T. (2019) · arXiv · doi:10.48550/arxiv.1908.04742 · Cited by 140
Online Learning with Random Representations
Sutton, R. S., & Whitehead, S. D. (1993) · Elsevier eBooks · doi:10.1016/b978-1-55860-307-3.50047-2 · Cited by 129 · cited 1×
“The threshold θ i is set to be equal to ( m + 1) × β − S i , in which β ∈ [0, 1] and S i is the number of input weights with negative value 63 .”
Self-citationChasing Sparsity in Vision Transformers: An End-to-End Exploration
Chen, T., Cheng, Y., Gan, Z., Yuan, L., Zhang, L., & Wang, Z. (2021) · arXiv · doi:10.48550/arxiv.2106.04533 · Cited by 111 · cited 1×
“On the other hand, PPO augmented with”
Continual Backprop: Stochastic Gradient Descent with Persistent RandomnessPreprint
Dohare, S., Sutton, R. S., & Mahmood, A. R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.06325 · Cited by 106
Self-citationLearning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference
Riemer, M., Cases, I., Ajemian, R., Liu, M., Rish, I., Tu, Y., & Tesauro, G. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.11910 · Cited by 98 · cited 1×
“Because of this, in almost all applications, deep learning is restricted to a special training phase and then turned off when the network is actually used.”
Child: A First Step Towards Continual Learning
Ring, M. B. (1998) · Learning to Learn · doi:10.1007/978-1-4615-5529-2_11 · Cited by 84
Deep Reinforcement Learning with Plasticity Injection
Nikishin, E., Oh, J., Ostrovski, G., Lyle, C., Pascanu, R., Dabney, W., & Barreto, A. (2023) · neural information processing systems · doi:10.52202/075280-1614 · Cited by 83
Dynamic Sparse Training for Deep Reinforcement Learning
Sokar, G., Mocanu, E., Mocanu, D. C., Pechenizkiy, M., & Stone, P. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/477 · Cited by 78 · cited 1×
“Dynamic sparse training has achieved promising results in supervised and reinforcement learning problems [66, 67], where dynamic sparse networks achieve performance close to dense networks even at high sparsity levels.”
On Warm-Starting Neural Network Training
Ash, J. T., & Adams, R. P. (2019) · arXiv · doi:10.48550/arxiv.1910.08475 · Cited by 77
Kashyap, R. L., Blaydon, C. C., & Fu, K. S. (1970) · Mathematics in Science and Engineering/Mathematics in science and engineering · doi:10.1016/s0076-5392(08)60499-3 · Cited by 76
Continual Learning: Applications and the Road ForwardPreprint
Verwimp, E., Aljundi, R., Ben-David, S., Bethge, M., Cossu, A., Gepperth, A., Hayes, T. L., Hüllermeier, E., Kanan, C., Kudithipudi, D., et al. (2023) · Transactions on machine learning research · doi:10.48550/arxiv.2311.11908 · Cited by 71 · cited 1×
“There are many desirable properties for an efficient continual-learning system 98,99 .”
(2021) · International Conference on Learning Representations · Cited by 71
PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learning
Lee, H., Cho, H., Kim, H., Gwak, D., Kim, J., Choo, J., Yun, S.-Y., & Yun, C. (2023) · Neural Information Processing Systems · doi:10.52202/075280-2720 · Cited by 51
Continual Learning as Computationally Constrained Reinforcement Learning
Kumar, S., Marklund, H., Rao, A., Zhu, Y., Jeon, H. J., Liu, Y., & Roy, B. V. (2025) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000116 · Cited by 50
Measuring Saturation in Neural Networks
Bosman, A. S., & Engelbrecht, A. P. (2015) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2015.202 · Cited by 45 · cited 1×
“Step size = 0.01 Step size = 0.001”
InfluentialDirections of Curvature as an Explanation for Loss of PlasticityPreprint
Lewandowski, A., Tanaka, H., Schuurmans, D., & Machado, M. C. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.00246 · Cited by 45
Online Normalization for Training Neural NetworksPreprint
Chiley, V., Sharapov, I., Kosson, A., Köster, U., Reece, R., de la Fuente, S. S., Subbiah, V., & James, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.05894 · Cited by 45
TRAINABILITY OF ReLU NETWORKS AND DATA-DEPENDENT INITIALIZATION
Shin, Y., & Karniadakis, G. E. (2020) · Journal of Machine Learning for Modeling and Computing · doi:10.1615/jmachlearnmodelcomput.2020034126 · Cited by 31
An Evolutionary Pattern Recognition Network
Klopf, A. H., & Gose, E. E. (1969) · IEEE Transactions on Systems Science and Cybernetics · doi:10.1109/tssc.1969.300268 · Cited by 26
Understanding and Preventing Capacity Loss in Reinforcement Learning
Lyle, C., Rowland, M., & Dabney, W. (2022) · arXiv · doi:10.48550/arxiv.2204.09560 · Cited by 13
Understanding plasticity in neural networks
Lyle, C., Zheng, Z., Nikishin, E., Pires, B. A., Pascanu, R., & Dabney, W. (2023) · arXiv · doi:10.48550/arxiv.2303.01486 · Cited by 11
Loss of Plasticity in Continual Deep Reinforcement Learning
Abbas, Z., Zhao, R., Modayil, J., White, A., & Machado, M. C. (2023) · arXiv · doi:10.48550/arxiv.2303.07507 · Cited by 10
Bigger, Better, Faster: Human-level Atari with human-level efficiency
Schwarzer, M., Obando-Ceron, J., Courville, A., Bellemare, M., Agarwal, R., & Castro, P. S. (2023) · arXiv · doi:10.48550/arxiv.2305.19452 · Cited by 9 · cited 1×
“Recent work has improved performance in many reinforcement learning problems by applying plasticitypreserving methods [85, 86, 87, 88].”
The Dormant Neuron Phenomenon in Deep Reinforcement Learning
Sokar, G., Agarwal, R., Castro, P. S., & Evci, U. (2023) · arXiv · doi:10.48550/arxiv.2302.12902 · Cited by 8
Loss of Plasticity in Deep Continual LearningPreprint
Dohare, S., Hernandez-Garcia, J., Rahman, P., et al. (2023) · Research Square · doi:10.21203/rs.3.rs-3256479/v1 · Cited by 7
Self-citationDisentangling the Causes of Plasticity Loss in Neural Networks
Lyle, C., Zheng, Z., Khetarpal, K., van Hasselt, H., Pascanu, R., Martens, J., & Dabney, W. (2024) · arXiv · doi:10.48550/arxiv.2402.18762 · Cited by 5 · cited 1×
“Recent work 103,104 has made exciting progress in this direction and it remains an important avenue for future work.”
Deep Reinforcement Learning with Plasticity Injection
Nikishin, E., Oh, J., Ostrovski, G., Lyle, C., Pascanu, R., Dabney, W., & Barreto, A. (2023) · arXiv · doi:10.48550/arxiv.2305.15555 · Cited by 4 · cited 1×
“These results show that plasticity loss can be catastrophic in both deep reinforcement learning as well as deep supervised learning.”
Continual Learning as Computationally Constrained Reinforcement Learning
Kumar, S., Marklund, H., Rao, A., Zhu, Y., Jeon, H. J., Liu, Y., & Van Roy, B. (2023) · arXiv · doi:10.48550/arxiv.2307.04345 · Cited by 4 · cited 1×
“New papers [84] are already attempting to fill this gap in the literature and future work that formalizes these concepts will be a valuable contribution to our understanding of continual learning.”
A study on the plasticity of neural networks
Berariu, T., Czarnecki, W., De, S., Bornschein, J., Smith, S., Pascanu, R., & Clopath, C. (2021) · arXiv · doi:10.48550/arxiv.2106.00042 · Cited by 3 · cited 3×
“Plasticity loss with modern methods was visible in some recent works 11,16–18 and most recently has begun to be explored explicitly 12,19–27 .”
PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learning
Lee, H., Cho, H., Kim, H., Gwak, D., Kim, J., Choo, J., Yun, S.-Y., & Yun, C. (2023) · arXiv · doi:10.48550/arxiv.2306.10711 · Cited by 2
Adding Gradient Noise Improves Learning for Very Deep Networks
Neelakantan, A., Vilnis, L., Le, Q. V., Sutskever, I., Kaiser, L., Kurach, K., & Martens, J. (2015) · arXiv · doi:10.48550/arxiv.1511.06807 · cited 1×
“On the other hand, PPO augmented with”
OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
Empirical Design in Reinforcement Learning
Patterson, A., Neumann, S., White, M., & White, A. (2023) · arXiv · doi:10.48550/arxiv.2304.01315
Research works printed but not linked: 36
- Bahdanau, D., Cho, K. & Bengio, Y. Neural machine translation by jointly learning to align and translate. In Proc. 3rd International Conference on Learning Representations (eds Bengio, Y. & LeCun, Y.) (ICLR, 2015).
- Achille, A., Rovere, M. & Soatto, S. Critical learning periods in deep networks. In Proc. 6th International Conference on Learning Representations (eds Murray, I., Ranzato, M. & Vinyals, O.) (ICLR, 2018).
- Nikishin, E., Schwarzer, M., D’Oro, P., Bacon, P.-L. & Courville, A. The primacy bias in deep reinforcement learning. In Proc. 39th International Conference on Machine Learning 16828–16847 (PMLR, 2022).
- Dohare, S. The Interplay of Search and Gradient Descent in Semi-stationary Learning Problems. Master’s thesis, Univ. Alberta (2020).
- Rahman, P. Toward Generate-and-test Algorithms for Continual Feature Discovery. Master’s thesis, Univ. Alberta (2021).
- Kumar, S., Marklund, H. & Van Roy, B. Maintaining plasticity in continual learning via regenerative regularization. In Proc. 3rd Conference on Lifelong Learning Agents (PMLR, 2024).
- Selfridge, O. G. Pandemonium: a paradigm for learning. In Mechanization of Thought Processes: Proceedings of a Symposium Held at the National Physical Laboratory 511–531 (Her Majesty’s Stationary Office, 1958).
- Mahmood, A. R. & Sutton, R. S. Representation search through generate and test. In Proc. AAAI Workshop: Learning Rich Representations from Low-Level Sensors 16–21 (2013).
- Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012).
- Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. In Proc. 3rd International Conference on Learning Representations (eds Bengio, Y. & LeCun, Y.) (ICLR, 2015).
- Yang, Y., Zhang, G., Xu, Z. & Katabi, D. Harnessing structures for value-based planning and reinforcement learning. In Proc. 7th International Conference on Learning Representations (ICLR, 2019).
- Maas, A. L., Hannun, A. Y. & Ng, A. Y. Rectifier nonlinearities improve neural network acoustic models. In Proc. 30th International Conference on Machine Learning (eds Dasgupta, S. & McAllester, D.) (JMLR, 2013).
- Nair, V. & Hinton, G. E. Rectified linear units improve restricted Boltzmann machines. In Proc. 27th International Conference on Machine Learning 807–814 (Omnipress, 2010).
- Roy, O. & Vetterli, M. The effective rank: a measure of effective dimensionality. In Proc. 2007 15th European Signal Processing Conference 606–610 (IEEE, 2007).
- Smith, S. L., Dherin, B., Barrett, D. & De, S. On the origin of implicit regularization in stochastic gradient descent. In Proc. 9th International Conference on Learning Representations (ICLR, 2021).
- Goodfellow, I., Bengio, Y. & Courville, A. Deep Learning (MIT Press, 2016).
- Ioffe, S. & Szegedy, C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In Proc. 32nd International Conference on Machine Learning 448–456 (PMLR, 2015).
- Ceron, J. S. O. & Castro, P. S. Revisiting rainbow: promoting more insightful and inclusive deep reinforcement learning research. In Proc. 38th International Conference on Machine Learning 1373–1383 (PMLR, 2021).
- Igl, M., Farquhar, G., Luketina, J., Boehmer, W. & Whiteson, S. Transient non-stationarity and generalisation in deep reinforcement learning. In Proc. 9th International Conference on Learning Representations (ICLR, 2021).
- Kumar, A., Agarwal, R., Ghosh, D. & Levine, S. Implicit under-parameterization inhibits data-efficient deep reinforcement learning. In Proc. 9th International Conference on Learning Representations (ICLR, 2021).
- D’Oro, P. et al. Sample-efficient reinforcement learning by breaking the replay ratio barrier. In Proc. 11th International Conference on Learning Representations (ICLR, 2023).
- Delfosse, Q., Schramowski, P., Mundt, M., Molina, A. & Kersting, K. Adaptive rational activations to boost deep reinforcement learning. In Proc. 12th International Conference on Learning Representations (ICLR, 2024).
- Yoon, J., Yang, E., Lee, J. & Hwang, S. J. Lifelong learning with dynamically expandable networks. In Proc. 6th International Conference on Learning Representations (eds Murray, I., Ranzato, M. & Vinyals, O.) (ICLR, 2018).
- Golkar, S., Kagan, M. & Cho, K. in Proc. NeurIPS 2019 Workshop on Real Neurons & Hidden Units: Future Directions at the Intersection of Neuroscience and Artificial Intelligence 146 (NeurIPS, 2019).
- Rajasegaran, J., Hayat, M., Khan, S. H., Khan, F. & Shao, L. Random path selection for continual learning. Adv. Neural Inf. Process. Syst. 32, 1–11 (2019).
- Veniat, T., Denoyer, L. & Ranzato, M. Efficient continual learning with modular networks and task-driven priors. In Proc. 9th International Conference on Learning Representations (ICLR, 2021).
- Rusu, A. A. et al. in Proc. 1st Annual Conference on Robot Learning 262–270 (PMLR, 2017).
- Chen, J., Nguyen, T., Gorur, D. & Chaudhry, A. Is forgetting less a good inductive bias for forward transfer? In Proc. 11th International Conference on Learning Representations (ICLR, 2023).
- LeCun, Y., Denker, J. & Solla, S. Optimal brain damage. Adv. Neural Inf. Process. Syst. 2, 598–605 (1989).
- Liu, J., Xu, Z., Shi, R., Cheung, R. C. C. & So, H. K. H. Dynamic sparse training: find efficient sparse network from scratch with trainable masked layers. In Proc. 8th International Conference on Learning Representations (ICLR, 2020).
- Elsayed, M. & Mahmood, A. R. Addressing catastrophic forgetting and loss of plasticity in neural networks. In Proc. 12th International Conference on Learning Representations (ICLR, 2024).
- Bellec, G., Kappel, D., Maass, W. & Legenstein, R. Deep rewiring: training very sparse deep networks. In Proc. 6th International Conference on Learning Representations (eds Murray, I., Ranzato, M. & Vinyals, O.) (ICLR, 2018).
- Evci, U., Gale, T., Menick, J., Castro, P. S. & Elsen, E. Rigging the lottery: making all tickets winners. In Proc. 37th International Conference on Machine Learning 2943–2952 (PMLR, 2020).
- Graesser, L., Evci, U., Elsen, E. & Castro, P. S. The state of sparse training in deep reinforcement learning. In Proc. 39th International Conference on Machine Learning 7766–7792 (PMLR, 2022).
- Zhou, G., Sohn, K. & Lee, H. Online incremental feature learning with denoising autoencoders. In Proc. 15th International Conference on Artificial Intelligence and Statistics 1453–1461 (PMLR, 2012).
- Sutskever, I., Martens, J., Dahl, G. & Hinton, G. On the importance of initialization and momentum in deep learning. In Proc. 30th International Conference on Machine Learning (eds Dasgupta, S. & McAllester, D.) 1139–1147 (JMLR, 2013).
Cited by
Onciul, R., Tataru, C.-I., Dumitru, A. V., Crivoi, C., Serban, M., Covache-Busuioc, R.-A., Radoi, M. P., & Toader, C. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14020550 · Cited by 113
“Unlike traditional AI models, which often suffer from catastrophic forgetting, neuroscience-informed algorithms incorporate mechanisms like memory consolidation and synaptic plasticity to enable incremental learning [172].”
Yu, H., Zhang, H., Zhang, Z., & Yang, S. (2025) · eTransportation · doi:10.1016/j.etran.2025.100420 · Cited by 86
Continual learning and catastrophic forgetting
van de Ven, G. M., Soures, N., & Kudithipudi, D. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-15754-7.00073-0 · Cited by 211
“It was recently shown that incremental training of deep neural networks on a sequence of tasks can lead to a substantial loss of plasticity (Dohare et al., 2023), highlighting that rapid adaptation is still an open problem in continual learning.”
BackgroundArtificial intelligence-enhanced echocardiography in cardiovascular disease management
Myhre, P. L., Grenne, B. L., Asch, F. M., Delgado, V., Khera, R., Lafitte, S., Lang, R. M., Pellikka, P. A., Sengupta, P. P., Vemulapalli, S., et al. (2025) · Nature Reviews Cardiology · doi:10.1038/s41569-025-01197-0 · Cited by 79
Dong, X., Ding, H., Gao, D., Zheng, G., Wang, J., & Lang, Q. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.103092 · Cited by 27
Metadata
- Authors
- Dohare, S., Hernandez-Garcia, J. F., Lan, Q., Rahman, P., Mahmood, A. R., & Sutton, R. S.
- Affiliations
- University of Alberta (CA)
- Canadian Institute for Advanced Research (CA)
- Corresponding author
- Dohare, Shibhansh
- Venue
- Nature
- Published
- 2024-08-21
- Volume
- 632
- Issue
- 8026
- Pages
- 768-774
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- PMC
- PMC11338828
- OpenAlex
- W4401793915
- Cited by
- 172
- References
- 92
- Citations by year
- 2026: 91 · 2025: 70 · 2024: 10
- FWCI
- 25.13
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Nature (publishedVersion, cc-by)
- Research Square (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Canadian Institute for Advanced Research
- Alberta Machine Intelligence Institute
- DeepMind
- Alliance de recherche numérique du Canada
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- continual learning, neuroplasticity, catastrophic forgetting, deep neural network, network diversity, gradient-based learning, reinforcement learning, ImageNet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.