Learning on the Job: Online Lifelong and Continual Learning
Open accessAbstract
One of the hallmarks of the human intelligence is the ability to learn continuously, accumulate the knowledge learned in the past and use the knowledge to help learn more and learn better. It is hard to imagine a truly intelligent system without this capability. This type of learning differs significantly than the classic machine learning (ML) paradigm of isolated single-task learning. Although there is already research on learning a sequence of tasks incrementally under the names of lifelong learning or continual learning, they still follow the traditional two-phase separate training and testing paradigm in learning each task. The tasks are also given by the user. This paper adds on-the-job learning to the mix to emphasize the need to learn during application (thus online) after the model has been deployed, which traditional ML cannot do. It aims to leverage the learned knowledge to discover new tasks, interact with humans and the environment, make inferences, and incrementally learn the new tasks on the fly during applications in a self-supervised and interactive manner. This is analogous to human on-the-job learning after formal training. We use chatbots and self-driving cars as examples to discuss the need, some initial work, and key challenges and opportunities in building this capability.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The current dominant machine learning paradigm makes the closed-world assumption, meaning that classes seen in testing or deployment must have been seen during training.
“The current dominant machine learning paradigm (ML) makes the closedworld assumption , which means that the classes of objects seen by the system in testing or deployment must have been seen during training”
From Open-world continual learning: Unifying novelty detection and continual learning (2025) · cites this paper as [39]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899 · see citing works
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988 · see citing works
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770 · see citing works
Flagged first, then by citations
Research works printed but not linked: 32
- Bendale, A. and Boult, T. 2015. Towards Open World Recognition. In Proceedings of CVPR -2015.
- Bou Ammar, H.; Eaton, E.; Luna, Jose Marcio, and Ruvolo, Paul. 2015. Autonomous Cross-domain Knowledge Transfer in Lifelong Policy Gradient Reinforcement Learning. In Proceedings of AAAI-2015 .
- Chen, Z. and Liu, B. 2014. Topic Modeling using Topics from Many Domains, Lifelong Learning and Big Data. In Proceedings of ICML-2014 .
- Chen, Z.; Ma, N. and Liu, B. 2015. Lifelong Learning for Sentiment Classification. In Proceedings of ACL-2015 .
- De Rosa, R.; Mensink, T. and Caputo, B. 2016. Online Open World Recognition. ArXiv:1604.02275. · arXiv
- Fei, G. and Liu, B. 2016. Breaking the Closed World Assumption in Text Classification. In Proceedings of NAACL-HLT-2016 .
- He, X. and Jaeger, H. 2018. Overcoming Catastrophic Interference Using Conceptor-aided Backpropagation. In Proceedings of ICLR-2018 .
- Hu, W.; Liu, Z.; Liu, B.; Tao, C.; Tao, Z.; Ma, J.; Zhao, D. and Yan, R.. 2019. Overcoming Catastrophic Forgetting for Continual Learning via Model Adaptation. In Proceedings of ICLR-2019 .
- Lake, B.; Salakhutdinov, R.; Gross, J. and Tenenbaum J. 2011. One shot learning of simple visual concepts. In Proceedings of the Annual Meeting of the Cognitive Science Society . 33(33).
- Lee, S.; Kim, J.; Jun, J.; Ha, J. and Zhang, B. 2017. Overcoming Catastrophic Forgetting by Incremental Moment Matching. In Proceedings of NIPS-2017 .
- Li, Z. and Hoiem, D. 2016. Learning without Forgetting. In Proceedings of ECCV-2016 , 9908(1):614 -629.
- Lv. G.; Wang, S.; Liu, B.; Chen, E. and Zhang K. 2019. Sentiment Classification by Leveraging the Shared Knowledge from a Sequence of Domains. In Proceedings of the 24th International Conference on Database Systems for Advanced Application s ( DASFAA-2019 ).
- Masse, N. Y.; Grant, G. D. and Freedman, D. J. 2018. Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization. arXiv:1802.01569. · arXiv
- Mazumder, S.; Liu, B.; Wang, S. and Ma, N. 2019a. Lifelong and Interactive Learning of Factual Knowledge in Dialogues. In Proceedings of SIGDIAL-2019 .
- Mazumder, S.; Liu, B.; Wang, S. and Esmaeilpour, S. 2019b. Building an Application Independent Natural Language Interface. arXiv:1910.14084, 2019. · arXiv
- Mitchell, T.; Cohen, W.; Hruschka, E.; Talukdar, P.; Betteridge, J.; Carlson, A.; Dalvi, B.; Gardner, M.; Kisiel, B.; Krishnamurthy, J.; Lao, N.; Mazaitis, K.; Mohamed, T.; Nakashole, N.; Platanios, E.; Ritter, A.; Samadi, M.; Settles, B.; Wang, R.; Wijaya, D.; Gupta, A.; Chen, X.; Saparov, A.; Greaves, M.; and Welling, J.; 2015. Never-ending learning. In Proceedings of AAAI-05 .
- Palatucci, M.; Pomerleau, D.; Hinton, G. and Mitchell, T. 2009. Zero-shot learning with semantic output codes. In Proceedings of 23rd Annual Conference on Neural Information Processing Systems ( NIPS-2009 ). 1410 -1418.
- Parisi, G. I; Kemker, R.; Part, J. L; Kanan, C. and Wermter, S. 2018. Continual Lifelong Learning with Neural Networks: A review. arXiv:1802.07569. · arXiv
- Ruvolo, P. and Eaton, E. 2013. ELLA: An Efficient Lifelong Learning Algorithm. In Proceedings of ICML-2013 , 507 -515.
- Scheirer, W. J.; de Rezende Rocha, A.; Sapkota, A. and Boult, T. E. 2013. Towards open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence , 35(7), 1757 -1772.
- Schwarz, J.; Luketina, J.; Czarnecki, W. M; Grabska-Barwinska, A.; Teh, Y. W.; Pascanu, R. and Hadsell, R. 2018. Progress & Compress: A Scalable Framework for Continual Learning. In Proceedings of ICML-2018 .
- Seff, A.; Beatson, A.; Suo, D. and Liu, H. 2017. Continual Learning in Generative Adversarial Nets. arXiv:1705.08395. · arXiv
- Shin, H. Lee, J.; Kim, J. and Kim J. 2017 Continual Learning with Deep Generative Replay. In Advances in Neural Information Processing Systems , 2990 -2999.
- Shu, L.; Xu, H. and Liu, B. 2017a. DOC: Deep Open Classification of Text Documents. In Proceedings of EMNLP-2017 .
- Shu, L.; Xu, H. and Liu, B. 2017b. Lifelong Learning CRF for Supervised Aspect Extraction. In Proceedings of ACL-2017 .
- Shu, L.; Xu, H. and Liu, B. 2018. Unseen Class Discovery in Open-world Classification. arXiv:1801.05609. · arXiv
- Silver, D. L.; Yang, Q. and Li, L. 2013. Lifelong Machine Learning Systems: Beyond Learning Algorithms. In Proceedings of AAAI Spring Symposium on Lifelong Machine Learning .
- Thrun, S. 1996. Is learning the n-th thing any easier than learning the first? In Proceedings of NIPS-1996 .
- Thrun, S. and Mitchell, T. 1995. Lifelong Robot Learning. In L. Steels (ed.), The Biology and Technology of Intelligent Autonomous Agents , Springer-Verlag.
- Xu, H.; Liu, B., Lei S. and Yu, P. 2019. Open-world Learning and Application to Product Classification. In Proceedings of WWW-2019 .
- Yang, Z.; Yang, D.; Dyer, C.; He, X.; Smola, A.; Hovy, E. 2016. Hierarchical Attention Networks for Document Classification. In Proceedings of NAACL-HLT-2016 .
- Yoon, J.; Yang, E.; Lee, J. and Hwang, S. 2018. Lifelong Learning with Dynamically Expandable Networks. In Proceedings of ICLR-2018 .
Cited by
Sorted by citations
Open-world continual learning: Unifying novelty detection and continual learning
Kim, G., Xiao, C., Konishi, T., Ke, Z., & Liu, B. (2025) · Artificial Intelligence · doi:10.1016/j.artint.2024.104237 · Cited by 19 · see citing works
Self-citation
- Authors
- Liu, B.
- Affiliations
- Peking University (CN)
- Corresponding author
- Liu, Bing
- ORCID
- Liu, Bing
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2020-04-03
- Volume
- 34
- Issue
- 09
- Pages
- 13544-13549
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W3037024089
- Cited by
- 47
- References
- 3
- Citations by year
- 2026: 4 · 2025: 4 · 2024: 6 · 2023: 13 · 2022: 6 · 2021: 11 · 2020: 3
- FWCI
- 3.11
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- www.aaai.org/ (unspecified)
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- on-the-job learning, lifelong learning, continual learning, online learning, self-supervised learning, interactive learning, chatbots, autonomous vehicles, knowledge accumulation, human-AI interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
