CoChat

Loss of plasticity in deep continual learning

Open access
Dohare, Shibhansh, Hernandez-Garcia, J. Fernando, Lan, Qingfeng, Rahman, Parash, Mahmood, A. Rupam, Sutton, Richard S. · 2024 · Nature

Published version of a preprint

The DOI matches its Crossref record; 109 of the 109 references it cites were found in a registry or an index; 17 of the 36 sources its claims cite were checked for support.

Abstract

Artificial neural networks, deep-learning methods and the backpropagation algorithm 1 form the foundation of modern machine learning and artificial intelligence. These methods are almost always used in two phases, one in which the weights of the network are updated and one in which the weights are held constant while the network is used or evaluated. This contrasts with natural learning and many applications, which require continual learning. It has been unclear whether or not deep learning methods work in continual learning settings. Here we show that they do not—that standard deep-learning methods gradually lose plasticity in continual-learning settings until they learn no better than a shallow network. We show such loss of plasticity using the classic ImageNet dataset and reinforcement-learning problems across a wide range of variations in the network and the learning algorithm. Plasticity is maintained indefinitely only by algorithms that continually inject diversity into the network, such as our continual backpropagation algorithm, a variation of backpropagation in which a small fraction of less-used units are continually and randomly reinitialized. Our results indicate that methods based on gradient descent are not enough—that sustained deep learning requires a random, non-gradient component to maintain variability and plasticity.

Overview

CoChat Summary

TL;DR

Standard deep-learning methods gradually lose plasticity in continual-learning settings, eventually learning no better than shallow or linear networks across supervised and reinforcement-learning tasks. Continual backpropagation, which selectively reinitializes a small fraction of least-used units, maintains plasticity apparently indefinitely in these experiments.

Key findings

  • In Continual ImageNet, unmodified backpropagation learned up to 88% correct on early tasks but by the 2,000th task had lost substantial plasticity for all tested step sizes, ending near or below a linear baseline; L2 regularization, Shrink and Perturb, and continual backpropagation maintained plasticity, with continual backpropagation best.
  • In class-incremental CIFAR-100, the base deep-learning system was initially better than retraining from scratch but after 40 classes showed increasing loss of plasticity; by 100 classes its accuracy was 5% lower than the retrained network. Continual backpropagation eliminated this loss and reached 76.13% final accuracy on all 100 classes.
  • In reinforcement learning, standard PPO on the non-stationary ant performed well for the first 2 million steps but worsened after friction changes; on the stationary ant, PPO improved for about 3 million steps then collapsed, failing every episode after 20 million steps. PPO with continual backpropagation plus L2 regularization maintained or improved performance.
  • Loss of plasticity correlated with more dead/dormant units, growing average weight magnitude, and lower effective rank; Adam, Dropout, and normalization often worsened plasticity, whereas L2 regularization and Shrink and Perturb reduced it.

Methods

  • Supervised continual-learning experiments included Continual ImageNet (1,000 classes, 700 images each, 600 train/100 test, binary class pairs, downsampled 32×32 images, convolutional network, SGD with momentum 0.9, 250 epochs per task, mini-batch 100, 30 runs), class-incremental CIFAR-100 (100 classes, 600 images each, 450/50/100 train/validation/test split, increments of 5 classes, 18-layer ResNet, 200 epochs per increment, 30 runs), Online Permuted MNIST (60,000 28×28 images, 800 tasks, three-hidden-layer feed-forward networks with 2,000 units per layer, single pass), and Slowly-Changing Regression (six activation functions, 3 million examples, 100 runs).
  • Reinforcement-learning experiments used PPO on Ant-v3, Hopper-v3, and Walker-v3; the non-stationary ant changed friction every 2 million time steps, the stationary ant ran for 50 million steps, and some experiments ran up to 100 million steps. Policy and value networks each had two hidden layers of 256 units and were trained with Adam; non-stationary results averaged 100 runs and stationary results 30 runs.
  • Continual backpropagation combines standard backpropagation with selective reinitialization of low-contribution-utility mature units. Utility is a running average with decay rate η = 0.99; outgoing weights of reinitialized units are set to zero; replacement rate ρ and maturity threshold m control reinitialization. For example, CIFAR-100 used ρ = 10⁻⁵ and m = 1,000; ImageNet used m = 100; reinforcement learning used ρ = 10⁻⁴, m = 10⁴, and weight decay 10⁻⁴.

Limitations

  • Stated: reinforcement-learning demonstrations are less definitive than supervised ones because of greater stochasticity, algorithm and setting variety, and confounding between an algorithm’s learning ability and its ability to generate informative data; more runs and experimental work are needed.
  • Stated: continual backpropagation’s utility measure is heuristic and not global, it does not address forgetting, and benchmarks focus on plasticity rather than forgetting or speed of learning; L2 regularization and Shrink and Perturb are sensitive to hyperparameters, and the ReDo comparison is preliminary.
  • Evident from the text: Continual ImageNet resets output heads at task changes, giving privileged information about task boundaries and not used in other experiments; a systematic study with large language models was not possible because a single training run would cost millions of dollars.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41586-024-07711-7

Cites

109 of 109 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Plasticity loss in Continual ImageNet. a–c, In a sequence of binary classification tasks using ImageNet pictures (a), the conventional backpropagation algorithm loses plasticity at all step sizes (b), whereas the continual backpropagation, L2 regularization and Shrink and Perturb algorithms maintain plasticity, apparently indefinitely (c). All results are averaged over 30 runs; the solid lines represent the mean and the shaded regions correspond to ±1 standard error.
Fig. 1 Plasticity loss in Continual ImageNet. a–c, In a sequence of binary classification tasks using ImageNet pictures (a), the conventional backpropagation algorithm loses plasticity at all step sizes (b), whereas the continual backpropagation, L2 regularization and Shrink and Perturb algorithms maintain plasticity, apparently indefinitely (c). All results are averaged over 30 runs; the solid lines represent the mean and the shaded regions correspond to ±1 standard error.CC BY · PubMed Central
Fig. 2 Plasticity loss in class-incremental CIFAR-100. a, An incrementally growing image-classification problem. b, Initially, accuracy is improved by incremental training compared with a network trained from scratch, but after 40 classes, accuracy degrades substantially in a base deep-learning system, less so for a Shrink and Perturb learning system and not at all for a learning system based on continual backpropagation. c, The number of network units that are active less than 1% of the time increases rapidly for the base deep-learning system, but less so for Shrink and Perturb and continual backpropagation systems. d, A low stable rank means that the units of a network do not provide much diversity; the base deep-learning system loses much more diversity than the Shrink and Perturb and continual backpropagation systems. All results are averaged over 30 runs; the solid lines represent the mean and the shaded regions correspond to ±1 standard error.
Fig. 2 Plasticity loss in class-incremental CIFAR-100. a, An incrementally growing image-classification problem. b, Initially, accuracy is improved by incremental training compared with a network trained from scratch, but after 40 classes, accuracy degrades substantially in a base deep-learning system, less so for a Shrink and Perturb learning system and not at all for a learning system based on continual backpropagation. c, The number of network units that are active less than 1% of the time increases rapidly for the base deep-learning system, but less so for Shrink and Perturb and continual backpropagation systems. d, A low stable rank means that the units of a network do not provide much diversity; the base deep-learning system loses much more diversity than the Shrink and Perturb and continual backpropagation systems. All results are averaged over 30 runs; the solid lines represent the mean and the shaded regions correspond to ±1 standard error.CC BY · PubMed Central
Fig. 3 Maintaining plasticity in a non-stationary reinforcement-learning problem. a, The reinforcement-learning agent controls torques at the eight joints of the simulated ant (red circles) to maximize forward motion and minimize penalties. b, Here we use a version of the ant problem in which the friction on contact with the ground is abruptly changed every 2 million time steps. c, The standard PPO learning algorithm fails catastrophically on the non-stationary ant problem. If the optimizer of PPO (Adam) is tuned in a custom way, then the failure is less severe, but adding continual backpropagation or L2 regularization is necessary to perform well indefinitely. These results are averaged over 100 runs; the solid lines represent the mean and the shaded regions represent the 95% bootstrapped confidence interval.
Fig. 3 Maintaining plasticity in a non-stationary reinforcement-learning problem. a, The reinforcement-learning agent controls torques at the eight joints of the simulated ant (red circles) to maximize forward motion and minimize penalties. b, Here we use a version of the ant problem in which the friction on contact with the ground is abruptly changed every 2 million time steps. c, The standard PPO learning algorithm fails catastrophically on the non-stationary ant problem. If the optimizer of PPO (Adam) is tuned in a custom way, then the failure is less severe, but adding continual backpropagation or L2 regularization is necessary to perform well indefinitely. These results are averaged over 100 runs; the solid lines represent the mean and the shaded regions represent the 95% bootstrapped confidence interval.CC BY · PubMed Central
Fig. 4 Results on a stationary ant-locomotion problem. a, The four reinforcement-learning algorithms performed similarly on this and the non-stationary problem (compare with Fig. 3c). b,c, A closer look inside the networks reveals a similar pattern as in supervised learning (compare with Fig. 2c,d). d, The absolute values of the weights of the networks increased steadily under standard and tuned PPO, whereas they decreased and stayed small under L2 regularization with or without continual backpropagation. These results are averaged over 30 runs; the solid lines represent the mean and the shaded regions represent the 95% bootstrapped confidence interval.
Fig. 4 Results on a stationary ant-locomotion problem. a, The four reinforcement-learning algorithms performed similarly on this and the non-stationary problem (compare with Fig. 3c). b,c, A closer look inside the networks reveals a similar pattern as in supervised learning (compare with Fig. 2c,d). d, The absolute values of the weights of the networks increased steadily under standard and tuned PPO, whereas they decreased and stayed small under L2 regularization with or without continual backpropagation. These results are averaged over 30 runs; the solid lines represent the mean and the shaded regions represent the 95% bootstrapped confidence interval.CC BY · PubMed Central
Extended Data Fig. 1 Further results on class-incremental CIFAR-100. a, Test accuracy in class-incremental CIFAR-100. As more classes are added, the classification becomes harder and algorithms naturally show decreasing accuracy with more classes. Each line corresponds to the average of 15 runs. b, Test accuracy of continual backpropagation for different values of the replacement rate parameter with contribution utility and 1,000 maturity threshold. The line corresponding to 10−4 is an average of five runs, whereas the other two lines are an average of 15 runs. The solid lines represent the mean and the shaded regions correspond to ±1 standard error.
Extended Data Fig. 1 Further results on class-incremental CIFAR-100. a, Test accuracy in class-incremental CIFAR-100. As more classes are added, the classification becomes harder and algorithms naturally show decreasing accuracy with more classes. Each line corresponds to the average of 15 runs. b, Test accuracy of continual backpropagation for different values of the replacement rate parameter with contribution utility and 1,000 maturity threshold. The line corresponding to 10−4 is an average of five runs, whereas the other two lines are an average of 15 runs. The solid lines represent the mean and the shaded regions correspond to ±1 standard error.CC BY · PubMed Central
Extended Data Fig. 2 Loss of plasticity in the Slowly-Changing Regression problem. a, The target function and the input in the Slowly-Changing Regression problem. The input has m + 1 bits. One of the flipping bits is chosen after every T time steps and its value is flipped. The next m − f bits are i.i.d. at every time step and the last bit is always one. The target function is represented by a neural network with a single hidden layer of LTUs. Each weight in the target network is −1 or 1. b, Loss of plasticity is robust across different activations. These results are averaged over 100 runs; the solid lines represent the mean and the shaded regions correspond to ±1 standard error.
Extended Data Fig. 2 Loss of plasticity in the Slowly-Changing Regression problem. a, The target function and the input in the Slowly-Changing Regression problem. The input has m + 1 bits. One of the flipping bits is chosen after every T time steps and its value is flipped. The next m − f bits are i.i.d. at every time step and the last bit is always one. The target function is represented by a neural network with a single hidden layer of LTUs. Each weight in the target network is −1 or 1. b, Loss of plasticity is robust across different activations. These results are averaged over 100 runs; the solid lines represent the mean and the shaded regions correspond to ±1 standard error.CC BY · PubMed Central
Extended Data Fig. 3 Loss of plasticity in Online Permuted MNIST. a, Left, an MNIST image with the label ‘7’; right, a corresponding permuted image. b, Loss of plasticity in Online Permuted MNIST is robust over step sizes, network sizes and rates of change. c, Evolution of various qualities of a deep network trained by means of backpropagation with different step sizes. Left, over time, the percentage of dead units in the network increases. Centre, the average magnitude of the weights increases over time. Right, the effective rank of the representation of the networks trained with backpropagation decreases over time. The results in these six plots are the average over 30 runs. The solid lines represent the mean and the shaded regions correspond to ±1 standard error. For some lines, the shaded region is thinner than the line width, as standard error is small.
Extended Data Fig. 3 Loss of plasticity in Online Permuted MNIST. a, Left, an MNIST image with the label ‘7’; right, a corresponding permuted image. b, Loss of plasticity in Online Permuted MNIST is robust over step sizes, network sizes and rates of change. c, Evolution of various qualities of a deep network trained by means of backpropagation with different step sizes. Left, over time, the percentage of dead units in the network increases. Centre, the average magnitude of the weights increases over time. Right, the effective rank of the representation of the networks trained with backpropagation decreases over time. The results in these six plots are the average over 30 runs. The solid lines represent the mean and the shaded regions correspond to ±1 standard error. For some lines, the shaded region is thinner than the line width, as standard error is small.CC BY · PubMed Central
Extended Data Fig. 4 Existing deep-learning methods on Online Permuted MNIST. a, Left, online classification accuracy of various algorithms on Online Permuted MNIST. Shrink and Perturb has almost no drop in online classification accuracy over time. Continual backpropagation did not show any loss of plasticity and had the best level of performance. Centre left, over time, the percentage of dead units increases in all methods except for continual backpropagation; it has almost zero dead units throughout learning. Centre right, the average magnitude of the weights increases over time for all methods except for L2 regularization, Shrink and Perturb and continual backpropagation. These are also the three best-performing methods, which suggests that small weights are important for fast learning. Right, the effective rank of the representation of all methods drops over time. However, continual backpropagation maintains a higher effective rank than both backpropagation and Shrink and Perturb. Among all the algorithms, only continual backpropagation maintains a high effective rank, low weight magnitude and low percentage of dead units. The results correspond to the average over 30 independent runs. The shaded regions correspond to ±1 standard error. b, Performance of various algorithms on Online Permuted MNIST for various hyperparameter combinations. For each method, we show three different hyperparameter settings. The parameter settings that were used in the left panel in a are marked with a solid square next to their label. The results correspond to the average of over 30 runs for settings marked with a solid square and 10 runs for the rest. The solid lines represent the mean and the shaded regions correspond to ±1 standard error.
Extended Data Fig. 4 Existing deep-learning methods on Online Permuted MNIST. a, Left, online classification accuracy of various algorithms on Online Permuted MNIST. Shrink and Perturb has almost no drop in online classification accuracy over time. Continual backpropagation did not show any loss of plasticity and had the best level of performance. Centre left, over time, the percentage of dead units increases in all methods except for continual backpropagation; it has almost zero dead units throughout learning. Centre right, the average magnitude of the weights increases over time for all methods except for L2 regularization, Shrink and Perturb and continual backpropagation. These are also the three best-performing methods, which suggests that small weights are important for fast learning. Right, the effective rank of the representation of all methods drops over time. However, continual backpropagation maintains a higher effective rank than both backpropagation and Shrink and Perturb. Among all the algorithms, only continual backpropagation maintains a high effective rank, low weight magnitude and low percentage of dead units. The results correspond to the average over 30 independent runs. The shaded regions correspond to ±1 standard error. b, Performance of various algorithms on Online Permuted MNIST for various hyperparameter combinations. For each method, we show three different hyperparameter settings. The parameter settings that were used in the left panel in a are marked with a solid square next to their label. The results correspond to the average of over 30 runs for settings marked with a solid square and 10 runs for the rest. The solid lines represent the mean and the shaded regions correspond to ±1 standard error.CC BY · PubMed Central
Extended Data Fig. 5 Further results in stationary reinforcement-learning problems. a, Similar to Fig. 4, the performance of standard PPO drops over time. However, unlike in Fig. 4, the performance of PPO with L2 regularization gets worse over time in Hopper-v3. On the other hand, PPO with continual backpropagation and L2 regularization can keep improving with time. b, Comparison of continual backpropagation and ReDo on Ant-v3. The performance of PPO with ReDo and L2 regularization worsens over time, whereas PPO with continual backpropagation and L2 regularization keeps improving over time. c, PPO with standard Adam leads to large updates in the policy network compared with proper Adam (β1 = β1 = 0.99), which explains why PPO with proper Adam performs much better than standard PPO. d, Comparison of two forms of utility in continual backpropagation, when using a running estimate of instantaneous utility and when using just the instantaneous utility. Both variations have similar performance. All these results are averaged over 30 runs; the solid lines represent the mean and the shaded regions correspond to 95% bootstrapped confidence interval.
Extended Data Fig. 5 Further results in stationary reinforcement-learning problems. a, Similar to Fig. 4, the performance of standard PPO drops over time. However, unlike in Fig. 4, the performance of PPO with L2 regularization gets worse over time in Hopper-v3. On the other hand, PPO with continual backpropagation and L2 regularization can keep improving with time. b, Comparison of continual backpropagation and ReDo on Ant-v3. The performance of PPO with ReDo and L2 regularization worsens over time, whereas PPO with continual backpropagation and L2 regularization keeps improving over time. c, PPO with standard Adam leads to large updates in the policy network compared with proper Adam (β1 = β1 = 0.99), which explains why PPO with proper Adam performs much better than standard PPO. d, Comparison of two forms of utility in continual backpropagation, when using a running estimate of instantaneous utility and when using just the instantaneous utility. Both variations have similar performance. All these results are averaged over 30 runs; the solid lines represent the mean and the shaded regions correspond to 95% bootstrapped confidence interval.CC BY · PubMed Central

Verification

Claims the paper makes

9 Valid · 8 Not valid · 5 Not checked · 167 Own results and interpretations

88 more claims

Create a free account to see all 189 claims and to verify the 1 reference still open.

See all claims — free account
  • The quote only partly states this claim. 1
  • It may state two separate claims as one. 67
  • Probably not a substantial claim. 39

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Show all 109 references

Research works printed but not linked: 38

  1. Bahdanau, D., Cho, K. & Bengio, Y. Neural machine translation by jointly learning to align and translate. In Proc. 3rd International Conference on Learning Representations (eds Bengio, Y. & LeCun, Y.) (ICLR, 2015).
  2. Achille, A., Rovere, M. & Soatto, S. Critical learning periods in deep networks. In Proc. 6th International Conference on Learning Representations (eds Murray, I., Ranzato, M. & Vinyals, O.) (ICLR, 2018).
  3. Nikishin, E., Schwarzer, M., D’Oro, P., Bacon, P.-L. & Courville, A. The primacy bias in deep reinforcement learning. In Proc. 39th International Conference on Machine Learning 16828–16847 (PMLR, 2022).
  4. Dohare, S. The Interplay of Search and Gradient Descent in Semi-stationary Learning Problems. Master’s thesis, Univ. Alberta (2020).
  5. Rahman, P. Toward Generate-and-test Algorithms for Continual Feature Discovery. Master’s thesis, Univ. Alberta (2021).
  6. Kumar, S., Marklund, H. & Van Roy, B. Maintaining plasticity in continual learning via regenerative regularization. In Proc. 3rd Conference on Lifelong Learning Agents (PMLR, 2024).
  7. Selfridge, O. G. Pandemonium: a paradigm for learning. In Mechanization of Thought Processes: Proceedings of a Symposium Held at the National Physical Laboratory 511–531 (Her Majesty’s Stationary Office, 1958).
  8. Mahmood, A. R. & Sutton, R. S. Representation search through generate and test. In Proc. AAAI Workshop: Learning Rich Representations from Low-Level Sensors 16–21 (2013).
  9. Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012).
  10. Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. In Proc. 3rd International Conference on Learning Representations (eds Bengio, Y. & LeCun, Y.) (ICLR, 2015).
  11. Yang, Y., Zhang, G., Xu, Z. & Katabi, D. Harnessing structures for value-based planning and reinforcement learning. In Proc. 7th International Conference on Learning Representations (ICLR, 2019).
  12. Maas, A. L., Hannun, A. Y. & Ng, A. Y. Rectifier nonlinearities improve neural network acoustic models. In Proc. 30th International Conference on Machine Learning (eds Dasgupta, S. & McAllester, D.) (JMLR, 2013).
  13. Nair, V. & Hinton, G. E. Rectified linear units improve restricted Boltzmann machines. In Proc. 27th International Conference on Machine Learning 807–814 (Omnipress, 2010).
  14. Glorot, X. & Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proc. 13th International Conference on Artificial Intelligence and Statistics 249–256 (PMLR, 2010).
  15. Roy, O. & Vetterli, M. The effective rank: a measure of effective dimensionality. In Proc. 2007 15th European Signal Processing Conference 606–610 (IEEE, 2007).
  16. Smith, S. L., Dherin, B., Barrett, D. & De, S. On the origin of implicit regularization in stochastic gradient descent. In Proc. 9th International Conference on Learning Representations (ICLR, 2021).
  17. Goodfellow, I., Bengio, Y. & Courville, A. Deep Learning (MIT Press, 2016).
  18. Ioffe, S. & Szegedy, C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In Proc. 32nd International Conference on Machine Learning 448–456 (PMLR, 2015).
  19. Ceron, J. S. O. & Castro, P. S. Revisiting rainbow: promoting more insightful and inclusive deep reinforcement learning research. In Proc. 38th International Conference on Machine Learning 1373–1383 (PMLR, 2021).
  20. Igl, M., Farquhar, G., Luketina, J., Boehmer, W. & Whiteson, S. Transient non-stationarity and generalisation in deep reinforcement learning. In Proc. 9th International Conference on Learning Representations (ICLR, 2021).
  21. Kumar, A., Agarwal, R., Ghosh, D. & Levine, S. Implicit under-parameterization inhibits data-efficient deep reinforcement learning. In Proc. 9th International Conference on Learning Representations (ICLR, 2021).
  22. D’Oro, P. et al. Sample-efficient reinforcement learning by breaking the replay ratio barrier. In Proc. 11th International Conference on Learning Representations (ICLR, 2023).
  23. Delfosse, Q., Schramowski, P., Mundt, M., Molina, A. & Kersting, K. Adaptive rational activations to boost deep reinforcement learning. In Proc. 12th International Conference on Learning Representations (ICLR, 2024).
  24. Yoon, J., Yang, E., Lee, J. & Hwang, S. J. Lifelong learning with dynamically expandable networks. In Proc. 6th International Conference on Learning Representations (eds Murray, I., Ranzato, M. & Vinyals, O.) (ICLR, 2018).
  25. Golkar, S., Kagan, M. & Cho, K. in Proc. NeurIPS 2019 Workshop on Real Neurons & Hidden Units: Future Directions at the Intersection of Neuroscience and Artificial Intelligence 146 (NeurIPS, 2019).
  26. Rajasegaran, J., Hayat, M., Khan, S. H., Khan, F. & Shao, L. Random path selection for continual learning. Adv. Neural Inf. Process. Syst. 32, 1–11 (2019).
  27. Veniat, T., Denoyer, L. & Ranzato, M. Efficient continual learning with modular networks and task-driven priors. In Proc. 9th International Conference on Learning Representations (ICLR, 2021).
  28. Rusu, A. A. et al. in Proc. 1st Annual Conference on Robot Learning 262–270 (PMLR, 2017).
  29. Chen, J., Nguyen, T., Gorur, D. & Chaudhry, A. Is forgetting less a good inductive bias for forward transfer? In Proc. 11th International Conference on Learning Representations (ICLR, 2023).
  30. LeCun, Y., Denker, J. & Solla, S. Optimal brain damage. Adv. Neural Inf. Process. Syst. 2, 598–605 (1989).
  31. Han, S., Mao, H. & Dally, W. J. Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding. In Proc. 4th International Conference on Learning Representations (ICLR, 2016).
  32. Liu, J., Xu, Z., Shi, R., Cheung, R. C. C. & So, H. K. H. Dynamic sparse training: find efficient sparse network from scratch with trainable masked layers. In Proc. 8th International Conference on Learning Representations (ICLR, 2020).
  33. Elsayed, M. & Mahmood, A. R. Addressing catastrophic forgetting and loss of plasticity in neural networks. In Proc. 12th International Conference on Learning Representations (ICLR, 2024).
  34. Bellec, G., Kappel, D., Maass, W. & Legenstein, R. Deep rewiring: training very sparse deep networks. In Proc. 6th International Conference on Learning Representations (eds Murray, I., Ranzato, M. & Vinyals, O.) (ICLR, 2018).
  35. Evci, U., Gale, T., Menick, J., Castro, P. S. & Elsen, E. Rigging the lottery: making all tickets winners. In Proc. 37th International Conference on Machine Learning 2943–2952 (PMLR, 2020).
  36. Graesser, L., Evci, U., Elsen, E. & Castro, P. S. The state of sparse training in deep reinforcement learning. In Proc. 39th International Conference on Machine Learning 7766–7792 (PMLR, 2022).
  37. Zhou, G., Sohn, K. & Lee, H. Online incremental feature learning with denoising autoencoders. In Proc. 15th International Conference on Artificial Intelligence and Statistics 1453–1461 (PMLR, 2012).
  38. Sutskever, I., Martens, J., Dahl, G. & Hinton, G. On the importance of initialization and momentum in deep learning. In Proc. 30th International Conference on Machine Learning (eds Dasgupta, S. & McAllester, D.) 1139–1147 (JMLR, 2013).

Cited by

See all 172

Metadata

Authors
Dohare, S., Hernandez-Garcia, J. F., Lan, Q., Rahman, P., Mahmood, A. R., & Sutton, R. S.
Affiliations
  • University of Alberta (CA)
  • Canadian Institute for Advanced Research (CA)
Corresponding author
Dohare, Shibhansh
Venue
Nature
Published
2024-08-21
Volume
632
Issue
8026
Pages
768-774
ISSN
  • 0028-0836 (ISSN-L)
  • 1476-4687
OpenAlex
W4401793915
Cited by
172
References
92
Citations by year
2026: 91 · 2025: 70 · 2024: 10
FWCI
25.13
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature (publishedVersion, cc-by)
  • Research Square (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
  • Canadian Institute for Advanced Research
  • Alberta Machine Intelligence Institute
  • DeepMind
  • Alliance de recherche numérique du Canada
  • Natural Sciences and Engineering Research Council of Canada
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords
continual learning, neuroplasticity, catastrophic forgetting, deep neural network, network diversity, gradient-based learning, reinforcement learning, ImageNet

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Loss of plasticity in deep continual learning. https://app.cochat.ai/papers/10.1038/s41586-024-07711-7
The paper
Dohare, S., Hernandez-Garcia, J. F., Lan, Q., Rahman, P., Mahmood, A. R., & Sutton, R. S. (2024). Loss of plasticity in deep continual learning. Nature. https://doi.org/10.1038/s41586-024-07711-7

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.