CoChat
Sign Up
Cited by 27Sentiment Analysis and Opinion MiningDomain Adaptation and Few-Shot LearningTopic Modeling

The DOI matches its Crossref record; 2 of the 3 references it cites were found in a registry or an index, and 1 are not checked yet.

Boosting Large Language Models with Continual Learning for Aspect-based Sentiment Analysis

Open access
Ding, Xuanwen, Zhou, Jie, Dou, Liang, Chen, Qin, Wu, Yuanbin, Chen, Arlene, He, Liang·2024·Findings of the Association for Computational Linguistics: EMNLP 2024
DOI 10.18653/v1/2024.findings-emnlp.252
Open this paper in CoChat — free accountOpen PDF

Abstract

Aspect-based sentiment analysis (ABSA) is an important subtask of sentiment analysis, which aims to extract the aspects and predict their sentiments.Most existing studies focus on improving the performance of the target domain by fine-tuning domain-specific models (trained on source domains) based on the target domain dataset.Few works propose continual learning tasks for ABSA, which aim to learn the target domain's ability while maintaining the history domains' abilities.In this paper, we propose a Large Language Model-based Continual Learning (LLM-CL) model for ABSA.First, we design a domain knowledge decoupling module to learn a domain-invariant adapter and separate domain-variant adapters dependently with an orthogonal constraint.Then, we introduce a domain knowledge warmup strategy to align the representation between domain-invariant and domain-variant knowledge.In the test phase, we index the corresponding domainvariant knowledge via domain positioning to not require each sample's domain ID.Extensive experiments over 19 datasets indicate that our LLM-CL model obtains new state-of-the-art performance.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2024.findings-emnlp.252

Cites

0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Balancing memory and learning efficiency is important in LLM-based ABSA continual learning.

    “it emphasized the importance of balancing memory and learning efficiency in LLM-based ABSA continual learning [3].”

    From GPT-based lifelong learning and ANFIS-driven reply memory ratio prediction for aspect-based sentiment analysis (2025) · cites this paper as [3]

    Section 3.1 describes parameter-efficient fine-tuning, and Section 4.2 describes a baseline that stores samples as memory. Neither passage says that balancing memory use against learning efficiency is important in LLM-based ABSA continual learning. Searches for the proposed balance, memory, and efficiency found no passage addressing that point. (LLM, conf=0.73)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 45

  1. Hongjoon Ahn, Sungmin Cha, Donggyu Lee, and Tae- sup Moon. 2019. Uncertainty-based continual learn- ing with adaptive regularization. NeurIPS, 32.
  2. Magdalena Biesialska, Katarzyna Biesialska, and Marta R Costa-jussà. 2020. Continual lifelong learn- ing in natural language processing: A survey. In Proceedings of COLING, pages 6523-6541.
  3. Arslan Chaudhry, Marcus Rohrbach, Mohamed Elho- seiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Marc'Aurelio Ranzato. 2019. On tiny episodic memories in continual learning. arXiv preprint arXiv:1902.10486. · arXiv
  4. Xiaowen Ding, Bing Liu, and Philip S Yu. 2008. A holistic lexicon-based approach to opinion mining. In Proceedings of WSDM, pages 231-240.
  5. Hai Ha Do, Penatiyana WC Prasad, Angelika Maag, and Abeer Alsadoon. 2019. Deep learning for aspect- based sentiment analysis: a comparative review. Ex- pert systems with applications, 118:272-299.
  6. Hao Fei, Fei Li, Chenliang Li, Shengqiong Wu, Jingye Li, and Donghong Ji. 2022. Inheriting the wisdom of predecessors: A multiplex cascade framework for unified aspect-based sentiment analysis. In Proceed- ings of IJCAI, pages 4096-4103.
  7. Ruidan He, Wee Sun Lee, Hwee Tou Ng, and Daniel Dahlmeier. 2018. Exploiting document knowledge for aspect-level sentiment classification. In Proceed- ings of ACL, pages 579-585.
  8. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In ICLR.
  9. Minqing Hu and Bing Liu. 2004. Mining and summa- rizing customer reviews. In Proceedings of SIGKDD, pages 168-177.
  10. Zixuan Ke, Bing Liu, Nianzu Ma, Hu Xu, and Lei Shu. 2021a. Achieving forgetting prevention and knowl- edge transfer in continual learning. Proceedings of NeurIPS, 34:22443-22456.
  11. Zixuan Ke, Bing Liu, Hao Wang, and Lei Shu. 2021b. Continual learning with knowledge transfer for senti- ment classification. In ECML PKDD, pages 683-698. Springer.
  12. Zixuan Ke, Bing Liu, Hu Xu, and Lei Shu. 2021c. Clas- sic: Continual and contrastive learning of aspect senti- ment classification tasks. In Proceedings of EMNLP, pages 6871-6883.
  13. Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim, and Bing Liu. 2023. Continual pre- training of language models. In Proceedings of ICLR.
  14. Zixuan Ke, Hu Xu, and Bing Liu. 2021d. Adapt- ing bert for continual learning of a sequence of as- pect sentiment classification tasks. arXiv preprint arXiv:2112.03271. · arXiv
  15. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Ag- nieszka Grabska-Barwinska, et al. 2017. Over- coming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-3526.
  16. Dingcheng Li, Zheng Chen, Eunah Cho, Jie Hao, Xi- aohu Liu, Fan Xing, Chenlei Guo, and Yang Liu. 2022a. Overcoming catastrophic forgetting during domain adaptation of seq2seq language generation. In Proceedings of NAACL, pages 5441-5454.
  17. Guodun Li, Yuchen Zhai, Qianglong Chen, Xing Gao, Ji Zhang, and Yin Zhang. 2022b. Continual few-shot intent detection. In Proceedings of COLING, pages 333-343.
  18. Ruifan Li, Hao Chen, Fangxiang Feng, Zhanyu Ma, Xiaojie Wang, and Eduard Hovy. 2021. Dual graph convolutional networks for aspect-based sentiment analysis. In Proceedings of ACL, pages 6319-6329.
  19. Xin Li, Lidong Bing, Wai Lam, and Bei Shi. 2018. Transformation networks for target-oriented senti- ment classification. In Proceedings of ACL, pages 946-956.
  20. Qian Liu, Zhiqiang Gao, Bing Liu, and Yuanlin Zhang. 2015. Automated rule selection for aspect extraction in opinion mining. In IJCAI.
  21. Tianlin Liu, Lyle Ungar, and Joao Sedoc. 2019. Contin- ual learning for sentence representations using con- ceptors. arXiv preprint arXiv:1904.09187. · arXiv
  22. Guangyi Lv, Shuai Wang, Bing Liu, Enhong Chen, and Kun Zhang. 2019. Sentiment classification by lever- aging the shared knowledge from a sequence of do- mains. In DASFAA, pages 795-811. Springer.
  23. Natawut Monaikul, Giuseppe Castellucci, Simone Fil- ice, and Oleg Rokhlenko. 2021. Continual learn- ing for named entity recognition. In Proceedings of AAAI, volume 35, pages 13570-13577.
  24. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowl- edge bases? In Proceedings of EMNLP-IJCNLP, pages 2463-2473.
  25. Maria Pontiki, Dimitris Galanis, Haris Papageor- giou, Ion Androutsopoulos, Suresh Manandhar, Mo- hammed AL-Smadi, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, Orphée De Clercq, et al. 2016. Semeval-2016 task 5: Aspect based sentiment anal- ysis. In ProWorkshop on Semantic Evaluation (SemEval-2016), pages 19-30. Association for Com- putational Linguistics.
  26. Chengwei Qin and Shafiq Joty. 2022. LFPT5: A unified framework for lifelong few-shot language learning based on prompt tuning of t5. In ICLR.
  27. Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Ma- dian Khabsa, Mike Lewis, and Amjad Almahairi. 2023. Progressive prompts: Continual learning for language models. arXiv preprint arXiv:2301.12314. · arXiv
  28. Thomas Scialom, Tuhin Chakrabarty, and Smaranda Muresan. 2022. Continual-t0: Progressively instruct- ing 50+ tasks to language models without forgetting. arXiv preprint arXiv:2205.12393. · arXiv
  29. Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. 2018. Overcoming catastrophic forget- ting with hard attention to the task. In ICML, pages 4548-4557. PMLR.
  30. Fabian Suchanek and Anh Tuan Luu. 2023. Knowl- edge bases and language models: Complementing forces. In International Joint Conference on Rules and Reasoning, pages 3-15. Springer.
  31. Fan-Keng Sun, Cheng-Hao Ho, and Hung-Yi Lee. 2019. Lamol: Language modeling for lifelong language learning. arXiv preprint arXiv:1909.03329. · arXiv
  32. Duyu Tang, Bing Qin, and Ting Liu. 2016. Aspect level sentiment classification with deep memory network. In Proceedings of EMNLP, pages 214-224.
  33. Vaibhav Varshney, Mayur Patidar, Rajat Kumar, Lovekesh Vig, and Gautam Shroff. 2022. Prompt augmented generative replay via supervised con- trastive learning for lifelong intent detection. In Find- ings of NAACL, pages 1113-1127.
  34. Kai Wang, Weizhou Shen, Yunyi Yang, Xiaojun Quan, and Rui Wang. 2020a. Relational graph attention network for aspect-based sentiment analysis. In Pro- ceedings of ACL, pages 3229-3238.
  35. Shuai Wang, Guangyi Lv, Sahisnu Mazumder, Geli Fei, and Bing Liu. 2018. Lifelong learning memory net- works for aspect sentiment classification. In Proceed- ings of Big Data, pages 861-870. IEEE.
  36. Shuai Wang, Mianwei Zhou, Sahisnu Mazumder, Bing Liu, and Yi Chang. 2020b. Disentangling aspect and opinion words in sentiment analysis using life- long pu learning. In New Frontiers in Mining Com- plex Patterns: 8th International Workshop, NFMCP 2019, Held in Conjunction with ECML-PKDD 2019, Würzburg, Germany, September 16, 2019, Revised Selected Papers 8, pages 100-115. Springer.
  37. Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuanjing Huang. 2023a. Orthogonal subspace learning for language model continual learning. arXiv preprint arXiv:2310.14152. · arXiv
  38. Yequan Wang, Minlie Huang, Xiaoyan Zhu, and Li Zhao. 2016. Attention-based lstm for aspect-level sentiment classification. In Proceedings of EMNLP, pages 606-615.
  39. Zhicheng Wang, Yufang Liu, Tao Ji, Xiaoling Wang, Yuanbin Wu, Congcong Jiang, Ye Chao, Zhencong Han, Ling Wang, Xu Shao, et al. 2023b. Rehearsal- free continual language learning via efficient parame- ter isolation. In Proceedings of ACL, pages 10933- 10946.
  40. Hang Yan, Junqi Dai, Tuo Ji, Xipeng Qiu, and Zheng Zhang. 2021. A unified generative framework for aspect-based sentiment analysis. In Proceedings of ACL, pages 2416-2429.
  41. Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. 2019. Continual learning of context-dependent pro- cessing in neural networks. Nature Machine Intelli- gence, 1(8):364-372.
  42. Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. 2023. Adaptive budget allocation for parameter-efficient fine-tuning. arXiv preprint arXiv:2303.10512. · arXiv
  43. Wenxuan Zhang, Xin Li, Yang Deng, Lidong Bing, and Wai Lam. 2022. A survey on aspect-based senti- ment analysis: tasks, methods, and challenges. IEEE Transactions on Knowledge and Data Engineering.
  44. Jie Zhou, Jimmy Xiangji Huang, Qin Chen, Qinmin Vi- vian Hu, Tingting Wang, and Liang He. 2019. Deep learning for aspect-level sentiment classification: Sur- vey, vision, and challenges. IEEE access, 7:78454- 78483.
  45. Yan Zhou, Fuqing Zhu, Pu Song, Jizhong Han, Tao Guo, and Songlin Hu. 2021. An adaptive hybrid framework for cross-domain aspect-based sentiment analysis. In Proceedings of AAAI, volume 35, pages 14630-14637.

Cited by

Sorted by citations

Related

Metadata

Authors
Ding, X., Zhou, J., Dou, L., Chen, Q., Wu, Y., Chen, A., & He, L.
Affiliations
East China Normal University (CN)
ORCID
Venue
Findings of the Association for Computational Linguistics: EMNLP 2024
Publisher
Association for Computational Linguistics
Published
2024
Pages
4367-4377
DOI
10.18653/v1/2024.findings-emnlp.252
OpenAlex
W4404781635
Cited by
24
References
0
Citations by year
2026: 13 · 2025: 9 · 2024: 2
FWCI
7.08
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • National Natural Science Foundation of China (62307028)
  • Science and Technology Commission of Shanghai Municipality (23ZR1441800, 22511105901, 21511100402)
  • National Science and Technology Major Project (2021ZD0114002)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
aspect-based sentiment analysis, continual learning, large language models, orthogonality constraints, multi-domain adaptation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review