CoChat
Sign Up
Cited by 91Domain Adaptation and Few-Shot LearningNeural Networks and ApplicationsData Stream Mining Techniques

Incorporating neuro-inspired adaptability for continual learning in artificial intelligence

Open access
Wang, Liyuan, Zhang, Xingxing, Li, Qian, Zhang, Mingtian, Su, Hang, Zhu, Jun, Zhong, Yi·2023·Nature Machine Intelligence
DOI 10.1038/s42256-023-00747-w
Open this paper in CoChat — free accountOpen PDF

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s42256-023-00747-w

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 58 · Cited by 91

Research works printed but not linked: 79

  1. Chen, Z., Liu, B.: Lifelong machine learning. Synthesis Lectures on Artificial Intelligence and Machine Learning 12(3), 1-207 (2018)
  2. Parisi, G.I., Kemker, R., Part, J.L., Kanan, C., Wermter, S.: Continual lifelong learning with neural networks: A review. Neural Networks (2019)
  3. Kudithipudi, D., Aguilar-Simon, M., Babb, J., Bazhenov, M., Blackiston, D., Bongard, J., Brna, A.P., Chakravarthi Raja, S., Cheney, N., Clune, J., et al.: Biological underpinnings for lifelong learning machines. Nature Machine Intelligence 4(3), 196-210 (2022)
  4. McClelland, J.L., McNaughton, B.L., O'Reilly, R.C.: Why there are com- plementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychological Review 102(3), 419 (1995)
  5. Wang, L., Zhang, X., Su, H., Zhu, J.: A comprehensive survey of continual learning: Theory, method and application. arXiv preprint arXiv:2302.00487 (2023) · arXiv
  6. Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: Proceedings of the International Conference on Machine Learning, pp. 3987-3995 (2017)
  7. Ritter, H., Botev, A., Barber, D.: Online structured laplace approxi- mations for overcoming catastrophic forgetting. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
  8. Shin, H., Lee, J.K., Kim, J., Kim, J.: Continual learning with deep gener- ative replay. In: Advances in Neural Information Processing Systems, pp. 2990-2999 (2017)
  9. Wang, L., Zhang, X., Yang, K., Yu, L., Li, C., Hong, L., Zhang, S., Li, Z., Zhong, Y., Zhu, J.: Memory replay with data compression for contin- ual learning. In: Proceedings of the International Conference on Learning Representations (2021)
  10. Serra, J., Suris, D., Miron, M., Karatzoglou, A.: Overcoming catas- trophic forgetting with hard attention to the task. In: Proceedings of the International Conference on Machine Learning, pp. 4548-4557 (2018)
  11. Fernando, C., Banarse, D., Blundell, C., Zwols, Y., Ha, D., Rusu, A.A., Pritzel, A., Wierstra, D.: Pathnet: Evolution channels gradient descent in super neural networks. arXiv preprint arXiv:1701.08734 (2017) · arXiv
  12. Delange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021)
  13. Hadsell, R., Rao, D., Rusu, A.A., Pascanu, R.: Embracing change: Contin- ual learning in deep neural networks. Trends in Cognitive Sciences 24(12), 1028-1040 (2020)
  14. Shuai, Y., Lu, B., Hu, Y., Wang, L., Sun, K., Zhong, Y.: Forgetting is regulated through rac activity in drosophila. Cell 140(4), 579-589 (2010)
  15. Waddell, S.: Neural plasticity: Dopamine tunes the mushroom body output network. Current Biology 26(3), 109-112 (2016)
  16. Modi, M.N., Shuai, Y., Turner, G.C.: The drosophila mushroom body: From architecture to algorithm in a learning circuit. Annual Review of Neuroscience 43, 465-484 (2020)
  17. Aso, Y., Sitaraman, D., Ichinose, T., Kaun, K.R., Vogt, K., Belliart- Guérin, G., Plaçais, P.-Y., Robie, A.A., Yamagata, N., Schnaitmann, C., et al.: Mushroom body output neurons encode valence and guide memory-based action selection in drosophila. Elife 3, 04580 (2014)
  18. Aso, Y., Rubin, G.M.: Dopaminergic neurons write and update memories with cell-type-specific rules. Elife 5, 16135 (2016)
  19. Gao, Y., Shuai, Y., Zhang, X., Peng, Y., Wang, L., He, J., Zhong, Y., Li, Q.: Genetic dissection of active forgetting in labile and consolidated memories in drosophila. Proceedings of the National Academy of Sciences 116(42), 21191-21197 (2019)
  20. Zhao, J., Zhang, X., Zhao, B., Hu, W., Diao, T., Wang, L., Zhong, Y., Li, Q.: Genetic dissection of mutual interference between two consecutive learning tasks in drosophila. Elife 12, 83516 (2023)
  21. Richards, B.A., Frankland, P.W.: The persistence and transience of memory. Neuron 94(6), 1071-1084 (2017)
  22. Dong, T., He, J., Wang, S., Wang, L., Cheng, Y., Zhong, Y.: Inability to activate rac1-dependent forgetting contributes to behavioral inflexibility in mutants of multiple autism-risk genes. Proceedings of the National Academy of Sciences 113(27), 7644-7649 (2016)
  23. Zhang, X., Li, Q., Wang, L., Liu, Z.-J., Zhong, Y.: Active protection: Learning-activated raf/mapk activity protects labile memory from rac1- independent forgetting. Neuron 98(1), 142-155 (2018)
  24. Davis, R.L., Zhong, Y.: The biology of forgetting-a perspective. Neuron 95(3), 490-503 (2017)
  25. Cervantes-Sandoval, I., Chakraborty, M., MacMullen, C., Davis, R.L.: Scribble scaffolds a signalosome for active forgetting. Neuron 90(6), 1230-1242 (2016)
  26. Noyes, N.C., Phan, A., Davis, R.L.: Memory suppressor genes: Modulat- ing acquisition, consolidation, and forgetting. Neuron 109(20), 3211-3227 (2021)
  27. Handler, A., Graham, T.G., Cohn, R., Morantte, I., Siliciano, A.F., Zeng, J., Li, Y., Ruta, V.: Distinct dopamine receptor pathways underlie the temporal sensitivity of associative learning. Cell 178(1), 60-75 (2019)
  28. McCurdy, L.Y., Sareen, P., Davoudian, P.A., Nitabach, M.N.: Dopamin- ergic mechanism underlying reward-encoding of punishment omission during reversal learning in drosophila. Nature Communications 12(1), 1-17 (2021)
  29. Berry, J.A., Cervantes-Sandoval, I., Nicholas, E.P., Davis, R.L.: Dopamine is required for learning and forgetting in drosophila. Neuron 74(3), 530- 542 (2012)
  30. Berry, J.A., Phan, A., Davis, R.L.: Dopamine neurons mediate learning and forgetting through bidirectional modulation of a memory trace. Cell Reports 25(3), 651-662 (2018)
  31. Aitchison, L., Jegminat, J., Menendez, J.A., Pfister, J.-P., Pouget, A., Latham, P.E.: Synaptic plasticity as bayesian inference. Nature Neuro- science 24(4), 565-571 (2021)
  32. Schug, S., Benzing, F., Steger, A.: Presynaptic stochasticity improves energy efficiency and helps alleviate the stability-plasticity dilemma. Elife 10, 69884 (2021)
  33. Wang, L., Zhang, M., Jia, Z., Li, Q., Bao, C., Ma, K., Zhu, J., Zhong, Y.: Afec: Active forgetting of negative transfer in continual learning. In: Advances in Neural Information Processing Systems, vol. 34 (2021)
  34. Benzing, F.: Unifying importance based regularisation methods for contin- ual learning. In: Proceedings of the International Conference on Artificial Intelligence and Statistics, pp. 2372-2396 (2022)
  35. Bouton, M.E.: Context, time, and memory retrieval in the interference paradigms of pavlovian learning. Psychological Bulletin 114(1), 80 (1993)
  36. Shuai, Y., Hirokawa, A., Ai, Y., Zhang, M., Li, W., Zhong, Y.: Dissecting neural pathways for forgetting in drosophila olfactory aversive memory. Proceedings of the National Academy of Sciences 112(48), 6663-6672 (2015)
  37. Chen, L., Chen, Z., Jiang, L., Liu, X., Xu, L., Zhang, B., Zou, X., Gao, J., Zhu, Y., Gong, X., et al.: Ai of brain and cognitive sciences: From the perspective of first principles. arXiv preprint arXiv:2301.08382 (2023) · arXiv
  38. Endo, K., Tsuchimoto, Y., Kazama, H.: Synthesis of conserved odor object representations in a random, divergent-convergent network. Neuron 108(2), 367-381 (2020)
  39. Long, M., Cao, Y., Wang, J., Jordan, M.: Learning transferable fea- tures with deep adaptation networks. In: Proceedings of the International Conference on Machine Learning, pp. 97-105 (2015)
  40. Wang, L., Zhang, X., Li, Q., Zhu, J., Zhong, Y.: Coscl: Cooperation of small continual learners is stronger than a big one. In: Proceedings of the European Conference on Computer Vision, pp. 254-271 (2022)
  41. Riemer, M., Cases, I., Ajemian, R., Liu, M., Rish, I., Tu, Y., Tesauro, G.: Learning to learn without forgetting by maximizing transfer and min- imizing interference. In: Proceedings of the International Conference on Learning Representations (2018)
  42. Schwarz, J., Czarnecki, W., Luketina, J., Grabska-Barwinska, A., Teh, Y.W., Pascanu, R., Hadsell, R.: Progress & compress: A scalable framework for continual learning. In: Proceedings of the International Conference on Machine Learning, pp. 4528-4537 (2018)
  43. Jung, S., Ahn, H., Cha, S., Moon, T.: Continual learning with node- importance based adaptive group sparse regularization. In: Advances in Neural Information Processing Systems, vol. 33, pp. 3647-3658 (2020)
  44. Cha, S., Hsu, H., Hwang, T., Calmon, F., Moon, T.: Cpr: Classifier- projection regularization for continual learning. In: Proceedings of the International Conference on Learning Representations (2020)
  45. Lomonaco, V., Maltoni, D.: Core50: a new dataset and benchmark for continuous object recognition. In: Conference on Robot Learning, pp. 17- 26 (2017). PMLR
  46. Ryan, T.J., Frankland, P.W.: Forgetting as a form of adaptive engram cell plasticity. Nature Reviews Neuroscience 23(3), 173-186 (2022)
  47. Luo, L., Hensch, T.K., Ackerman, L., Barbel, S., Jan, L.Y., Nung Jan, Y.: Differential effects of the rac gtpase on purkinje cell axons and dendritic trunks and spines. Nature 379(6568), 837-840 (1996)
  48. Tashiro, A., Minden, A., Yuste, R.: Regulation of dendritic spine mor- phology by the rho family of small gtpases: Antagonistic roles of rac and rho. Cerebral Cortex 10(10), 927-938 (2000)
  49. Hayashi-Takagi, A., Takaki, M., Graziane, N., Seshadri, S., Murdoch, H., Dunlop, A.J., Makino, Y., Seshadri, A.J., Ishizuka, K., Srivastava, D.P., et al.: Disrupted-in-schizophrenia 1 (disc1) regulates spines of the glutamate synapse via rac1. Nature Neuroscience 13(3), 327-332 (2010)
  50. Hayashi-Takagi, A., Yagishita, S., Nakamura, M., Shirai, F., Wu, Y.I., Loshbaugh, A.L., Kuhlman, B., Hahn, K.M., Kasai, H.: Labelling and optical erasure of synaptic memory traces in the motor cortex. Nature 525(7569), 333-338 (2015)
  51. Martens, J., Grosse, R.: Optimizing neural networks with kronecker- factored approximate curvature. In: Proceedings of the International Conference on Machine Learning, pp. 2408-2417 (2015)
  52. Knoblauch, J., Husain, H., Diethe, T.: Optimal continual learning has perfect memory and is np-hard. In: Proceedings of the International Conference on Machine Learning, pp. 5327-5337 (2020). PMLR
  53. Deng, D., Chen, G., Hao, J., Wang, Q., Heng, P.-A.: Flattening sharpness for dynamic gradient projection memory benefits continual learning. In: Advances in Neural Information Processing Systems, vol. 34 (2021)
  54. Mirzadeh, S.I., Farajtabar, M., Pascanu, R., Ghasemzadeh, H.: Under- standing the role of training regimes in continual learning. Advances in Neural Information Processing Systems 33, 7308-7320 (2020)
  55. McAllester, D.A.: Pac-bayesian model averaging. In: Proceedings of the Twelfth Annual Conference on Computational Learning Theory, pp. 164- 170 (1999)
  56. Pham, Q., Liu, C., Sahoo, D., Steven, H.: Contextual transformation net- works for online continual learning. In: Proceedings of the International Conference on Learning Representations (2021)
  57. Lopez-Paz, D., et al.: Gradient episodic memory for continual learning. In: Advances in Neural Information Processing Systems, pp. 6467-6476 (2017)
  58. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 618-626 (2017) References
  59. Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision. pp. 139-154 (2018)
  60. Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., Vaughan, J.W.: A theory of learning from different domains. Machine Learning 79(1), 151-175 (2010)
  61. Cha, S., Hsu, H., Hwang, T., Calmon, F., Moon, T.: Cpr: Classifier-projection reg- ularization for continual learning. In: Proceedings of the International Conference on Learning Representations (2020)
  62. Chaudhry, A., Ranzato, M., Rohrbach, M., Elhoseiny, M.: Efficient lifelong learning with a-gem. In: Proceedings of the International Conference on Learning Representations (2018)
  63. Davis, R.L., Zhong, Y.: The biology of forgetting-a perspective. Neuron 95(3), 490-503 (2017)
  64. Dong, T., He, J., Wang, S., Wang, L., Cheng, Y., Zhong, Y.: Inability to activate rac1-dependent forgetting contributes to behavioral inflexibility in mutants of multiple autism-risk genes. Proceedings of the National Academy of Sciences 113(27), 7644-7649 (2016)
  65. Foret, P., Kleiner, A., Mobahi, H., Neyshabur, B.: Sharpness-aware minimization for efficiently improving generalization. In: Proceedings of the International Conference on Learning Representations (2020)
  66. Gao, Y., Shuai, Y., Zhang, X., Peng, Y., Wang, L., He, J., Zhong, Y., Li, Q.: Ge- netic dissection of active forgetting in labile and consolidated memories in drosophila. Proceedings of the National Academy of Sciences 116(42), 21191-21197 (2019)
  67. Jung, S., Ahn, H., Cha, S., Moon, T.: Continual learning with node-importance based adaptive group sparse regularization. In: Advances in Neural Information Processing Systems. vol. 33, pp. 3647-3658 (2020)
  68. Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catas- trophic forgetting in neural networks. Proceedings of the National Academy of Sciences 114(13), 3521-3526 (2017)
  69. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolu- tional neural networks. In: Advances in Neural Information Processing Systems. vol. 25, pp. 1097-1105 (2012)
  70. Kudithipudi, D., Aguilar-Simon, M., Babb, J., Bazhenov, M., Blackiston, D., Bongard, J., Brna, A.P., Chakravarthi Raja, S., Cheney, N., Clune, J., et al.: Biological un- derpinnings for lifelong learning machines. Nature Machine Intelligence 4(3), 196-210 (2022)
  71. Richards, B.A., Frankland, P.W.: The persistence and transience of memory. Neuron 94(6), 1071-1084 (2017)
  72. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy opti- mization algorithms. arXiv preprint arXiv:1707.06347 (2017) · arXiv
  73. Schwarz, J., Czarnecki, W., Luketina, J., Grabska-Barwinska, A., Teh, Y.W., Pascanu, R., Hadsell, R.: Progress & compress: A scalable framework for continual learning. In: Proceedings of the International Conference on Machine Learning. pp. 4528-4537 (2018)
  74. Shuai, Y., Lu, B., Hu, Y., Wang, L., Sun, K., Zhong, Y.: Forgetting is regulated through rac activity in drosophila. Cell 140(4), 579-589 (2010)
  75. van de Ven, G.M., Tuytelaars, T., Tolias, A.S.: Three types of incremental learning. Nature Machine Intelligence pp. 1-13 (2022)
  76. Wang, L., Zhang, M., Jia, Z., Li, Q., Bao, C., Ma, K., Zhu, J., Zhong, Y.: Afec: Active forgetting of negative transfer in continual learning. In: Advances in Neural Information Processing Systems. vol. 34 (2021)
  77. Wang, L., Zhang, X., Li, Q., Zhu, J., Zhong, Y.: Coscl: Cooperation of small continual learners is stronger than a big one. In: Proceedings of the European Conference on Computer Vision. pp. 254-271 (2022)
  78. Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: Proceedings of the International Conference on Machine Learning. pp. 3987-3995 (2017)
  79. Zhang, X., Li, Q., Wang, L., Liu, Z.J., Zhong, Y.: Active protection: Learning-activated raf/mapk activity protects labile memory from rac1-independent forgetting. Neuron 98(1), 142-155 (2018)

Authors
Wang, L., Zhang, X., Li, Q., Zhang, M., Su, H., Zhu, J., & Zhong, Y.
Affiliations
  • Center for Life Sciences (CN)
  • Tsinghua University (CN)
Corresponding author
  • Zhu, Jun
  • Zhong, Yi
ORCID
Venue
Nature Machine Intelligence
Publisher
Springer Science and Business Media LLC
Published
2023-11-16
Volume
5
Issue
12
Pages
1356-1368
ISSN
2522-5839 (ISSN-L)
DOI
10.1038/s42256-023-00747-w
OpenAlex
W4386302335
Cited by
91
References
58
Citations by year
2026: 23 · 2025: 42 · 2024: 25 · 2023: 1
FWCI
8.05
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Machine Intelligence (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Natural Science Foundation of China (U19A2081, 2020AAA0106302, STI2030, 32021002, 62061136001, 92248303)
  • Tsinghua University
  • National Key Research and Development Program of China (STI2030, 2020AAA0106302)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Data Stream Mining Techniques
Keywords
continual learning, catastrophic forgetting, active forgetting, memory stability, task-incremental learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review