CoChat
Sign Up
PreprintCited by 55Domain Adaptation and Few-Shot LearningTopic ModelingAdvanced Neural Network Applications

The DOI matches its DataCite record.

Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning

Open access
Ke, Zixuan, Liu, Bing, Ma, Nianzu, Xu, Hu, Shu, Lei·2021·arXiv
DOI 10.48550/arxiv.2112.02706arXiv 2112.02706
Open this paper in CoChat — free accountOpen PDF

Abstract

Continual learning (CL) learns a sequence of tasks incrementally with the goal of achieving two main objectives: overcoming catastrophic forgetting (CF) and encouraging knowledge transfer (KT) across tasks. However, most existing techniques focus only on overcoming CF and have no mechanism to encourage KT, and thus do not do well in KT. Although several papers have tried to deal with both CF and KT, our experiments show that they suffer from serious CF when the tasks do not have much shared knowledge. Another observation is that most current CL methods do not use pre-trained models, but it has been shown that such models can significantly improve the end task performance. For example, in natural language processing, fine-tuning a BERT-like pre-trained language model is one of the most effective approaches. However, for CL, this approach suffers from serious CF. An interesting question is how to make the best use of pre-trained models for CL. This paper proposes a novel model called CTR to solve these problems. Our experimental results demonstrate the effectiveness of CTR

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2112.02706

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Using a pre-trained transformer network and adapter modules is a common practice in existing continual learning methods in the natural language processing community.

    “Note that using a pre-trained transformer network and adapter modules is a common practice in existing continual learning methods in the natural language processing community [28, 27].”

    From Open-world continual learning: Unifying novelty detection and continual learning (2025) · cites this paper as [28]

    The Introduction (p. 2) says that most continual-learning techniques do not use pre-trained models, contrary to the claim that using them is common practice. Related Work (p. 3) identifies one NLP continual-learning method that uses adapters with BERT, but does not present that combination as common practice. (LLM, conf=0.88)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 68

  1. [1][1] H. Ahn, S. Cha, D. Lee, and T. Moon. Uncertainty-based continual learning with adaptive regularization. In NIPS , pages 4394-4404, 2019. -
  2. [2][2] P. Buzzega, M. Boschini, A. Porrello, D. Abati, and S. Calderara. Dark experience for general continual learning: a strong, simple baseline. arXiv preprint arXiv:2004.07211 , 2020. - · arXiv
  3. [3][3] A. Carlson, J. Betteridge, B. Kisiel, B. Settles, E. Hruschka, and T. Mitchell. Toward an architecture for never-ending language learning. In AAAI , volume 24, 2010. -
  4. [4][4] A. Chaudhry, M. Ranzato, M. Rohrbach, and M. Elhoseiny. Efficient lifelong learning with A-GEM. In ICLR , 2019. -
  5. [5][5] Z. Chen and B. Liu. Lifelong machine learning. Synthesis Lectures on Artificial Intelligence and Machine Learning , 12(3):1-207, 2018. -
  6. [6][6] Z. Chen, N. Ma, and B. Liu. Lifelong learning for sentiment classification. In ACL , pages 750-756, 2015. -
  7. [7][7] Y.-S. Chuang, S.-Y. Su, and Y.-N. Chen. Lifelong language knowledge distillation. arXiv preprint arXiv:2010.02123 , 2020. - · arXiv
  8. [8][8] J. Devlin, M. Chang, K. Lee, and K. Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In J. Burstein, C. Doran, and T. Solorio, editors, NAACL-HLT , pages 4171-4186. Association for Computational Linguistics, 2019. -
  9. [9][9] X. Ding, B. Liu, and P. S. Yu. A holistic lexicon-based approach to opinion mining. In Proceedings of the 2008 international conference on web search and data mining , pages 231-240, 2008. -
  10. [10][10] C. Fernando, D. Banarse, C. Blundell, Y. Zwols, D. Ha, A. A. Rusu, A. Pritzel, and D. Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. CoRR , abs/1701.08734, 2017. - · arXiv
  11. [11][11] E. Grave, P. Bojanowski, P. Gupta, A. Joulin, and T. Mikolov. Learning word vectors for 157 languages. In LREC , 2018. -
  12. [12][12] C. Greco, B. Plank, R. Fernández, and R. Bernardi. Psycholinguistics meets continual learning: Measuring catastrophic forgetting in visual question answering. arXiv preprint arXiv:1906.04229 , 2019. - · arXiv
  13. [13][13] P. Gupta, Y. Chaudhary, T. Runkler, and H. Schuetze. Neural topic modeling with continual lifelong learning. In International Conference on Machine Learning , pages 3907-3917. PMLR, 2020. -
  14. [14][14] X. He and H. Jaeger. Overcoming catastrophic interference using conceptor-aided backpropagation. In ICLR , 2018. -
  15. [15][15] G. E. Hinton, A. Krizhevsky, and S. D. Wang. Transforming auto-encoders. In International conference on artificial neural networks , pages 44-51. Springer, 2011. -
  16. [16][16] N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. de Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly. Parameter-efficient transfer learning for NLP. In K. Chaudhuri and R. Salakhutdinov, editors, ICML , volume 97 of Proceedings of Machine Learning Research , pages 2790-2799. PMLR, 2019. -
  17. [17][17] M. Hu and B. Liu. Mining and summarizing customer reviews. In Proceedings of ACM SIGKDD , pages 168-177, 2004. -
  18. [18][18] W. Hu, Q. Qin, M. Wang, J. Ma, and B. Liu. Continual learning by using information of each class holistically. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 7797-7805, 2021. -
  19. [19][19] E. Jang, S. Gu, and B. Poole. Categorical reparameterization with gumbel-softmax. In ICLR . OpenReview.net, 2017. -
  20. [20][20] N. Kamra, U. Gupta, and Y. Liu. Deep generative dual memory network for continual learning. CoRR , abs/1710.10368, 2017. - · arXiv
  21. [21][21] Z. Ke, B. Liu, and X. Huang. Continual learning of a mixed sequence of similar and dissimilar tasks. In NeurIPS , 2020. -
  22. [22][22] Z. Ke, B. Liu, H. Wang, and L. Shu. Continual learning with knowledge transfer for sentiment classification. In ECML-PKDD , 2020. -
  23. [23][23] Z. Ke, B. Liu, H. Xu, and L. Shu. Classic: Continual and contrastive learning of aspect sentiment classification tasks. In EMNLP , 2021. -
  24. [24][24] Z. Ke, H. Xu, and B. Liu. Adapting bert for continual learning of a sequence of aspect sentiment classification tasks. In NAACL , pages 4746-4755, 2021. -
  25. [25][25] H. Khayrallah, B. Thompson, K. Duh, and P. Koehn. Regularized training objective for continued training for domain adaptation in neural machine translation. In Proceedings of the 2nd Workshop on Neural Machine Translation and Generation , pages 36-44, 2018. -
  26. [26][26] Y. Kim. Convolutional neural networks for sentence classification. In A. Moschitti, B. Pang, and W. Daelemans, editors, EMNLP , pages 1746-1751. ACL, 2014. -
  27. [27][27] J. Kirkpatrick, R. Pascanu, N. C. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, D. Hassabis, C. Clopath, D. Kumaran, and R. Hadsell. Overcoming catastrophic forgetting in neural networks. CoRR , abs/1612.00796, 2016. - · arXiv
  28. [28][28] K. Lang. Newsweeder: Learning to filter netnews. In Proceedings of the Twelfth International Conference on Machine Learning , pages 331-339, 1995. -
  29. [29][29] M. D. Lange, R. Aljundi, M. Masana, and T. Tuytelaars. Continual learning: A comparative study on how to defy forgetting in classification tasks. CoRR , abs/1909.08383, 2019. - · arXiv
  30. [30][30] S. Lee, J. Kim, J. Jun, J. Ha, and B. Zhang. Overcoming catastrophic forgetting by incremental moment matching. In NIPS , pages 4652-4662, 2017. -
  31. [31][31] Y. Li, L. Zhao, K. Church, and M. Elhoseiny. Compositional language continual learning. In International Conference on Learning Representations , 2019. -
  32. [32][32] Q. Liu, Z. Gao, B. Liu, and Y. Zhang. Automated rule selection for aspect extraction in opinion mining. In IJCAI , 2015. -
  33. [33][33] T. Liu, L. Ungar, and J. Sedoc. Continual learning for sentence representations using conceptors. arXiv preprint arXiv:1904.09187 , 2019. - · arXiv
  34. [34][34] Y. Liu, Y. Su, A. Liu, B. Schiele, and Q. Sun. Mnemonics training: Multi-class incremental learning without forgetting. In CVPR , pages 12242-12251. IEEE, 2020. -
  35. [35][35] Z. Liu, G. I. Winata, A. Madotto, and P. Fung. Exploring fine-tuning techniques for pre-trained cross-lingual models via continual learning. arXiv preprint arXiv:2004.14218 , 2020. - · arXiv
  36. [36][36] D. Lopez-Paz and M. Ranzato. Gradient episodic memory for continual learning. In NIPS , pages 6467-6476, 2017. -
  37. [37][37] G. Lv, S. Wang, B. Liu, E. Chen, and K. Zhang. Sentiment classification by leveraging the shared knowledge from a sequence of domains. In DASFAA , pages 795-811, 2019. -
  38. [38][38] A. Madotto, Z. Lin, Z. Zhou, S. Moon, P. Crook, B. Liu, Z. Yu, E. Cho, and Z. Wang. Continual learning in task-oriented dialogue systems. arXiv preprint arXiv:2012.15504 , 2020. - · arXiv
  39. [39][39] A. Mallya and S. Lazebnik. Packnet: Adding multiple tasks to a single network by iterative pruning. In CVPR , pages 7765-7773, 2018. -
  40. [40][40] M. McCloskey and N. J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation , volume 24, pages 109-165. Elsevier, 1989. -
  41. [41][41] A. Merchant, E. Rahimtoroghi, E. Pavlick, and I. Tenney. What happens to BERT embeddings during fine-tuning? CoRR , abs/2004.14448, 2020. - · arXiv
  42. [42][42] J. Pfeiffer, A. Kamath, A. Rücklé, K. Cho, and I. Gurevych. Adapterfusion: Non-destructive task composition for transfer learning. In EACL , 2020. -
  43. [43][43] Q. Qin, W. Hu, and B. Liu. Using the past knowledge to improve sentiment classification. In EMNLP-findings , 2020. -
  44. [44][44] V. V. Ramasesh, E. Dyer, and M. Raghu. Anatomy of catastrophic forgetting: Hidden representations and task semantics. In ICLR , 2021. -
  45. [45][45] S. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert. icarl: Incremental classifier and representation learning. In CVPR , pages 5533-5542, 2017. -
  46. [46][46] M. Riemer, I. Cases, R. Ajemian, M. Liu, I. Rish, Y. Tu, , and G. Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. In International Conference on Learning Representations , 2019. -
  47. [47][47] M. Rostami, S. Kolouri, and P. K. Pilly. Complementary learning for overcoming catastrophic forgetting using experience replay. In IJCAI , pages 3339-3345, 2019. -
  48. [48][48] A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and R. Hadsell. Progressive neural networks. CoRR , abs/1606.04671, 2016. - · arXiv
  49. [49][49] P. Ruvolo and E. Eaton. ELLA: an efficient lifelong learning algorithm. In ICML , pages 507-515, 2013. -
  50. [50][50] S. Sabour, N. Frosst, and G. E. Hinton. Dynamic routing between capsules. In NIPS , pages 3856-3866, 2017. -
  51. [51][51] A. Seff, A. Beatson, D. Suo, and H. Liu. Continual learning in generative adversarial nets. CoRR , abs/1705.08395, 2017. - · arXiv
  52. [52][52] J. Serrà, D. Suris, M. Miron, and A. Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In ICML , pages 4555-4564, 2018. -
  53. [53][53] Y. Shen, X. Zeng, and H. Jin. A progressive model to enable continual learning for semantic slot filling. In EMNLP-IJCNLP , Nov. 2019. -
  54. [54][54] H. Shin, J. K. Lee, J. Kim, and J. Kim. Continual learning with deep generative replay. In NIPS , pages 2990-2999, 2017. -
  55. [55][55] L. Shu, H. Xu, and B. Liu. Lifelong learning CRF for supervised aspect extraction. In ACL , pages 148-154, 2017. -
  56. [56][56] D. L. Silver, Q. Yang, and L. Li. Lifelong machine learning systems: Beyond learning algorithms. In Lifelong Machine Learning, Papers from the 2013 AAAI Spring Symposium, Palo Alto, California, USA, March 25-27, 2013 , 2013. -
  57. [57][57] C. Sun, L. Huang, and X. Qiu. Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence. In NAACL , pages 380-385, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. -
  58. [58][58] F.-K. Sun, C.-H. Ho, and H.-Y. Lee. Lamol: Language modeling is all you need for lifelong language learning. In ICLR , 2020. -
  59. [59][59] D. Tang, B. Qin, and T. Liu. Aspect level sentiment classification with deep memory network. In EMNLP , pages 214-224, Austin, Texas, Nov. 2016. Association for Computational Linguistics. -
  60. [60][60] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. In NeurIPS , pages 5998-6008, 2017. -
  61. [61][61] H. Wang, B. Liu, S. Wang, N. Ma, and Y. Yang. Forward and backward knowledge transfer for sentiment classification. In ACML , pages 457-472, 2019. -
  62. [62][62] S. Wang, G. Lv, S. Mazumder, G. Fei, and B. Liu. Lifelong learning memory networks for aspect sentiment classification. In IEEE International Conference on Big Data , pages 861-870, 2018. -
  63. [63][63] Z. Wang, S. V. Mehta, B. Póczos, and J. Carbonell. Efficient meta lifelong-learning with limited memory. In EMNLP , 2020. -
  64. [64][64] R. Xia, J. Jiang, and H. He. Distantly supervised lifelong learning for large-scale social media sentiment analysis. IEEE Trans. Affective Computing , 8(4):480-491, 2017. -
  65. [65][65] H. Xu, B. Liu, L. Shu, and P. S. Yu. BERT post-training for review reading comprehension and aspect-based sentiment analysis. In J. Burstein, C. Doran, and T. Solorio, editors, NAACL-HLT , pages 2324-2335. Association for Computational Linguistics, 2019. -
  66. [66][66] H. Xu, S. Xie, L. Shu, and P. S. Yu. Dual attention network for product compatibility and function satisfiability analysis. In AAAI , 2018. -
  67. [67][67] W. Xue and T. Li. Aspect based sentiment analysis with gated convolutional networks. In I. Gurevych and Y. Miyao, editors, ACL , pages 2514-2523. Association for Computational Linguistics, 2018. -
  68. [69][69] F. Zenke, B. Poole, and S. Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning , pages 3987-3995. PMLR, 2017.

Cited by

Sorted by citations

Metadata

Authors
Ke, Z., Liu, B., Ma, N., Xu, H., & Shu, L.
Affiliations
  • University of Illinois Chicago (US)
  • Meta (United States) (US)
  • Amazon (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2021
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2112.02706
arXiv
2112.02706
OpenAlex
W3212560468
Cited by
55
References
0
Citations by year
2025: 12 · 2024: 22 · 2023: 16 · 2022: 5
Type
posted content
Language
en
Open access
Licence
creativecommons.org/publicdomain/zero/1.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (public-domain)
Funders
  • National Science Foundation (1838770, 1910424)
  • Northrop Grumman
  • Defense Advanced Research Projects Agency (HR001120C0023)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
continual learning, catastrophic forgetting, knowledge transfer, pre-trained models, fine-tuning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review