The DOI matches its Crossref record, with a small difference in title or year; 435 of the 474 references it cites were found in a registry or an index.
Towards Continual Reinforcement Learning: A Review and Perspectives
Open accessAbstract
In this article, we aim to provide a literature review of different formulations and approaches to continual reinforcement learning (RL), also known as lifelong or non-stationary RL. We begin by discussing our perspective on why RL is a natural fit for studying continual learning. We then provide a taxonomy of different continual RL formulations by mathematically characterizing two key properties of non-stationarity, namely, the scope and driver non-stationarity. This offers a unified view of various formulations. Next, we review and present a taxonomy of continual RL approaches. We go on to discuss evaluation of continual RL agents, providing an overview of benchmarks used in the literature and important metrics for understanding agent performance. Finally, we highlight open problems and challenges in bridging the gap between the current state of continual RL and findings in neuroscience. While still in its early days, the study of continual RL has the promise to develop better incremental reinforcement learners that can function in increasingly realistic applications where non-stationarity plays a vital role. These include applications such as those in the fields of healthcare, education, logistics, and robotics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Another line develops affordance theory in RL, where the environment constrains and suggests possible actions (Graves et al., 2022; Khetarpal et al., 2020). Rather than treating all actions as always available, agents learn which are afforded in each situation, mirroring selective, skillful responses in enactive perception.”
From Toward Enactive Artificial Intelligence (2026) · cites this paper as Khetarpal et al., 2020
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“While continual learning does not necessarily emphasize agent-gathered data or sustained interaction with the environment, it aligns with enactive views in emphasizing ongoing adaptation under non-stationarity.”
From Toward Enactive Artificial Intelligence (2026) · cites this paper as Khetarpal et al., 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Research works printed but not linked: 26
- PMLR, 2021.
- David Lopez-Paz and Marc'Aurelio Ranzato. Gradient episodic memory for continuum learning. Advances in Neural Information Processing Systems, 2017.
- Morgan Kaufmann Publishers Inc., 1994. ISBN 1-55860-335-2. URL http://dl.acm.org/citation.cfm?id=3091574.3091594.
- Qi Cai, Zhuoran Yang, and Zhaoran Wang. Sample-efficient reinforcement learning for pomdps with linear function approximations. arXiv preprint arXiv:2204.09787, 2022. · arXiv
- Weitong Zhang, Jiafan He, Dongruo Zhou, Amy Zhang, and Quanquan Gu. Provably efficient representation learning in low-rank markov decision processes. arXiv preprint arXiv:2106.11935, 2021a. · arXiv
- Biwei Huang, Fan Feng, Chaochao Lu, Sara Magliacane, and Kun Zhang. Adarl: What, where, and how to adapt in transfer reinforcement learning. arXiv preprint arXiv:2107.02729, 2021. · arXiv
- PMLR, 2022.
- Ayush Jain, Norio Kosaka, Kyung-Min Kim, and Joseph J Lim. Know your action set: Learning action relations for reinforcement learning. In International Conference on Learning Representations, 2021.
- Jürgen Schmidhuber. Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-... hook. PhD thesis, Technische Universität München, 1987.
- Pascal Klink, Carlo D'Eramo, Jan Peters, and Joni Pajarinen. Boosted curriculum reinforce- ment learning. In International Conference on Learning Representations, 2021.
- Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. Advances in Neural Information Processing Systems, 33:5824-5836, 2020b.
- Cristian Buciluǎ, Rich Caruana, and Alexandru Niculescu-Mizil. Model compression. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 535-541. ACM, 2006.
- Maximilian Igl, Gregory Farquhar, Jelena Luketina, JW Böhmer, and Shimon Whiteson. Transient non-stationarity and generalisation in deep reinforcement learning. In 9th International Conference on Learning Representations, 2021.
- Youngmin Oh, Jinwoo Shin, Eunho Yang, and Sung Ju Hwang. Model-augmented prioritized experience replay. In International Conference on Learning Representations, 2021.
- Kevin Lu, Aditya Grover, Pieter Abbeel, and Igor Mordatch. Reset-free lifelong learning with skill-space planning. In International Conference on Learning Representations, 2021.
- David Abel, Dilip Arumugam, Lucas Lehnert, and Michael L Littman. Toward good abstractions for lifelong learning. In NIPS Workshop on Hierarchical Reinforcement Learning, 2017.
- Chi Jin, Akshay Krishnamurthy, Max Simchowitz, and Tiancheng Yu. Reward-free explo- ration for reinforcement learning. In International Conference on Machine Learning, pages 4870-4879. PMLR, 2020.
- Dagui Chen, Qi Yan, Shangqi Guo, Zhile Yang, Xin Su, and Feng Chen. Learning effective subgoals with multi-task hierarchical reinforcement learning. Scaling-Up Reinforce- ment Learning (SURL) Workshop. URl: http://surl. tirl. info/proceedings/SURL- 2019 paper 10. pdf, 2019.
- Lorenzo Moro, Amarildo Likmeta, Enrico Prati, Marcello Restelli, et al. Goal-directed planning via hindsight experience replay. In International Conference on Learning Representations, pages 1-16, 2022.
- Andrew Levy, George Konidaris, Robert Platt, and Kate Saenko. Learning multi-level hierarchies with hindsight. arXiv preprint arXiv:1712.00948, 2017. · arXiv
- Curran Associates, Inc., 2019. URL http://papers.nips.cc/paper/8458-meta-learning-representations-for -continual-learning.pdf.
- Frederic Kaplan and Pierre-Yves Oudeyer. Curiosity-driven development. In Proceedings of the International Workshop on Synergistic Intelligence Dynamics. Citeseer, 2006.
- Karl Cobbe, Oleg Klimov, Chris Hesse, Taehoon Kim, and John Schulman. Quantifying generalization in reinforcement learning. In International Conference on Machine Learning, pages 1282-1289. PMLR, 2019a.
- Hiroshi Imamizu. Separated modules for visuomotor control and learning in the cerebellum: a functional mri study. In NeuroImage: Third International Conference on Functional Mapping of the Human Brain, volume 5, page S598. Academic Press, 1997.
- Morgan Kaufmann Publishers Inc. ISBN 1558607781.
- Amy Greenwald and Keith Hall. Correlated-Q learning. In ICML, page 242-249. AAAI Press, 2003. ISBN 1577351894.
Cited by
Sorted by citations
Items 101–200 of 228
Continual Learning with Columnar Spiking Neural Networks
Larionov, D., Bazenkov, N., & Kiselev, M. (2025) · Optical Memory and Neural Networks · doi:10.3103/s1060992x25601629 · Cited by 4
Deep Reinforcement Learning for Trajectory Generation and Optimisation of UAVs
Akhtar, M., Maqsood, A. M., & Verbeke, M. (2023) · International Conference on Recent Advances in Space Technologies · doi:10.1109/rast57548.2023.10197856 · Cited by 4
Deep Reinforcement Learning for Network Slice Placement and the DeepNetSlice Toolkit
Pasquali, A., Lomonaco, V., Bacciu, D., & Paganelli, F. (2024) · IEEE International Conference on Machine Learning for Communication and Networking (ICMLCN) · doi:10.1109/icmlcn59089.2024.10625023 · Cited by 4
Disentangled representations for causal cognition
Torresan, F., & Baltieri, M. (2024) · Physics of Life Reviews · doi:10.1016/j.plrev.2024.10.003 · Cited by 5
Error Resilient Online Reinforcement Learning Using Adaptive Statistical Checks
Amarnath, C., Mejri, M., Isenberg, J., & Chatterjee, A. (2025) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2025.3529820 · Cited by 4
Intrinsic gradient oxygen-driven second-order memristors for continual reinforcement learning
Ming, J., Wang, R., Fu, J., Liu, J., Wu, S., Liu, S., Shao, H., Li, Y., Wang, J., Wang, Y., et al. (2026) · Nature Communications · doi:10.1038/s41467-026-70014-0 · Cited by 6
QoE Optimization for VR Streaming: a Continual RL Framework in Digital Twin-empowered MEC
Yu, J., Alhilal, A. Y., Zhou, T., Hui, P., & Tsang, D. H. K. (2023) · Globecom · doi:10.1109/globecom54140.2023.10437100 · Cited by 3
Irfan, B., Staffa, M., Bobu, A., & Churamani, N. (2024) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3610978.3638159 · Cited by 3
Amin, M. R. M., Sivakumaran, V. M., Gan, K. H., Darmaraj, S. R., Hoo, W. C., & Yew, L. K. (2024) · Advances in educational marketing, administration, and leadership book series · doi:10.4018/979-8-3693-6351-5.ch009 · Cited by 3
A Framework for Neurosymbolic Goal-Conditioned Continual Learning in Open World Environments
Lorang, P., Goel, S., Shukla, Y., Zips, P., & Scheutz, M. J. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801627 · Cited by 3
Barros, P. V. A., Bloem, A. C., Hootsmans, I. M., Opheij, L. M., Toebosch, R., Barakova, E. I., & Sciutti, A. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.669990 · Cited by 3
Block Contextual MDPs for Continual Learning
Sodhani, S., Meier, F., Pineau, J., & Zhang, A. (2021) · arXiv · doi:10.48550/arxiv.2110.06972 · Cited by 3
Peng, Y., Feng, G., Qin, S., Wei, F., & Zhang, L. (2024) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2024.3457525 · Cited by 2
Message Action Adapter Framework in Multi-Agent Reinforcement Learning
Park, B., & Choi, J. (2024) · Applied Sciences · doi:10.3390/app142110079 · Cited by 2
Task-Agnostic Learning to Accomplish New Tasks
Zhang, X., Wang, X., Liu, X., Wang, W., Xiaopeng Fan, B., & Zhao, D. (2025) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2025.3565515 · Cited by 2
Cai, Y., Yanbing, M., Sha, L., Cao, H., & Caccamo, M. (2025) · ACM Transactions on Cyber-Physical Systems · doi:10.1145/3744351 · Cited by 2
When Does Neuroevolution Outcompete Reinforcement Learning in Transfer Learning Tasks?
Nisioti, E., Pedersen, J., Plantec, E., Montero, M. L., & Risi, S. (2025) · Annual Conference on Genetic and Evolutionary Computation · doi:10.1145/3712256.3726475 · Cited by 2
A comparative review of decision-making approaches for realistic event-driven environments
Stoyanova, I., Museux, N., Nguyen, S. M., & Filliat, D. (2025) · Discrete Event Dynamic Systems · doi:10.1007/s10626-025-00421-w · Cited by 2
Demircioğlu, U., & Bakır, H. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.112692 · Cited by 2
XAI ‐Guided Continual Learning: Rationale, Methods, and Future Directions
Proietti, M., Ragno, A., & Capobianco, R. (2025) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70046 · Cited by 3
Continual Visual Reinforcement Learning with A Life-Long World Model
Pan, M., Zhang, W., Chen, G., Zhu, X., Gao, S., Wang, Y., & Yang, X. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-06106-5_9 · Cited by 2
An improved Q -learning algorithm integrating past and future rewards
Lou, Q., Han, D., & Sun, M. (2026) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2026.118127 · Cited by 2
Goal-oriented inference of environment from redundant observations
Takahashi, K., Fukai, T., Sakai, Y., & Takekawa, T. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106246 · Cited by 2
Imitation Game: A Model-Based and Imitation Learning Deep Reinforcement Learning Hybrid
Veith, E. M., Logemann, T., Berezin, A., Wellßow, A., & Balduin, S. (2024) · Workshop on Modeling and Simulation of Cyber-Physical Energy Systems (MSCPES) · doi:10.1109/mscpes62135.2024.10542713 · Cited by 2
High-Dimensional Controller Tuning through Latent Representations
Sarmadi, A., Krishnamurthy, P., & Khorrami, F. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610607 · Cited by 2
A new paradigm to study social and physical affordances as model-based reinforcement learning
Chartouny, A., Amini, K., Khamassi, M., & Girard, B. (2024) · Cognitive Robotics · doi:10.1016/j.cogr.2024.08.001 · Cited by 2
Adaptive Quantization of Graph Convolutional Networks with Hardware-Aware On-device Training
Nunez-Yanez, J. L. (2024) · IEEE Nordic Circuits and Systems Conference (NorCAS) · doi:10.1109/norcas64408.2024.10752483 · Cited by 2
Srisuchinnawong, A., & Manoonpong, P. (2025) · The International Journal of Robotics Research · doi:10.1177/02783649251336385 · Cited by 5
Leveraging More of Biology in Evolutionary Reinforcement Learning
Gašperov, B., Ðurasević, M., & Jakobović, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56855-8_6 · Cited by 2
Continual Driving Policy Optimization with Closed-Loop Individualized Curricula
Niu, H., Xu, Y., Jiang, X., & Hu, J. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611578 · Cited by 2
A Simple Approach to Continual Learning by Transferring Skill ParametersPreprint
Zentner, K. R., Julian, R., Puri, U., Zhang, Y., & Sukhatme, G. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.10255 · Cited by 2
Iterative improvements from feedback for language modelsPreprint
Li, Y. (2023) · doi:10.14293/pr2199.000220.v1 · Cited by 3
Rapid Open-World Adaptation by Adaptation Principles LearningPreprint
Xue, C., Nikonova, E., Zhang, P., & Renz, J. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4778438 · Cited by 2
Continual Model-Based Reinforcement Learning for Data Efficient Wireless Network Optimisation
Hasan, C., Agapitos, A., Lynch, D., Castagna, A., Cruciata, G., Wang, H., & Milenovic, A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43427-3_18 · Cited by 1
Plasticity Driven Knowledge Transfer for Continual Deep Reinforcement Learning in Financial Trading
Katsikas, D., Passalis, N., & Tefas, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78189-6_6 · Cited by 1
A Unified Sparse Training Framework for Lightweight Lifelong Deep Reinforcement Learning
Xu, M., Chen, X., Fu, W., Lin, Y. S., Li, Y., & Wang, J. (2025) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2025.3537990 · Cited by 1
Combining meta reinforcement learning with neural plasticity mechanisms for improved AI performance
Liu, L., & Xu, Z. (2025) · PLoS ONE · doi:10.1371/journal.pone.0320777 · Cited by 1
Continual Reinforcement Learning with Implicit Generative Replay for Autonomous Driving
Deng, Q., Li, R., Hu, Q., Zhang, T., & Zhang, H. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91767-7_4 · Cited by 1
Uncertainty-Aware Continual Reinforcement Learning via PPO with Graph Representation Learning
Kim, D. (2025) · Mathematics · doi:10.3390/math13162542 · Cited by 3
Rahaman, S. U. (2025) · Computational Economics · doi:10.1007/s10614-025-11057-1 · Cited by 2
A Continual Offline Reinforcement Learning Benchmark for Navigation Tasks
Kobanda, A., Maillard, O.-A., & Portelas, R. (2025) · IEEE Conference on Games (CoG) · doi:10.1109/cog64752.2025.11114167 · Cited by 1
Ji, X., Pan, C., Yu, M., Jiang, Q., & Fan, Q. (2025) · Journal of Control and Decision · doi:10.1080/23307706.2025.2556991 · Cited by 1
Clustering-Based Weight Orthogonalization for Stabilizing Deep Reinforcement Learning
Ma, G., Zhang, Y., Dai, Y., Hao, G., Chen, Y., & Yu, S. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11227331 · Cited by 1
Beam-Hopping-based Massive Random Access in Satellite-assisted 6G Metaverse Networks
Liao, Y., Deng, Z., Wang, K., Lin, W., Xia, X. T., & Zhu, B. (2025) · IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) · doi:10.1109/infocomwkshps65812.2025.11152805 · Cited by 1
Position Paper: Collectionless Artificial Intelligence
Gori, M., & Melacci, S. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228186 · Cited by 1
A Choose-Your-Own-Adventure Game Using Branching Path Implemented By Fuzzy Logic
Ruslan, M. A. A., Narawi, N. A. A. B., Diah, N. M., Rahim, N., & Kamal, N. A. M. (2025) · IEEE International Conference on Computing (ICOCO) · doi:10.1109/icoco67189.2025.11334140 · Cited by 1
Hu, P., Wen, T., Zhang, D., Chen, H., & Gong, J. (2026) · Scientific Reports · doi:10.1038/s41598-026-35391-y · Cited by 2
Domain-specific schema reuse supports flexible learning to learn in the primate brain
Tian, K., Zhao, Z., Chen, Y., Ge, N., Cao, S., Han, X., Gu, J., & Yu, S. (2026) · Nature Communications · doi:10.1038/s41467-026-68692-x · Cited by 2
Efficient Continual Imitation Learning With Online Meta-Adapters
Zhu, R., Sun, E., Huang, G., & Çeliktutan, O. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3682534 · Cited by 1
Addressing catastrophic forgetting in class-incremental learning—a survey
Bako, J., & Kalita, J. K. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11575-w · Cited by 2
Safe adaptation in multiagent competition
Shen, M., & How, J. P. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981763 · Cited by 1
An Intentional Forgetting-Driven Self-Healing Method for Deep Reinforcement Learning Systems
Yahmed, A. H., Bouchoucha, R., Braiek, H. B., & Khomh, F. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00121 · Cited by 4
Cognitive mapping and episodic memory emerge from simple associative learning rules
Gribkova, E., Chowdhary, G. V., & Gillette, R. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127812 · Cited by 2
Comparative Evaluation of Continual Learning Methods in Financial and Industrial Time-Series Data
Ramjattan, R., Atzeni, D., & Mazzei, D. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650750 · Cited by 1
Boutyour, Y., & Idrissi, A. (2024) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2024.3485482 · Cited by 1
Bi-Directional Knowledge Transfer For Continual Deep Reinforcement Learning in Financial Trading
Katsikas, D., Passalis, N., & Tefas, A. (2024) · Machine learning for signal processing ... · doi:10.1109/mlsp58920.2024.10734752 · Cited by 1
Constraining an Unconstrained Multi-agent Policy with offline data
Guan, C., Jiang, T., Li, Y.-C., Zhang, Z., Yuan, L., & Yu, Y. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107253 · Cited by 1
Improved Context-Based Offline Meta-RL with Attention and Contrastive Learning.Preprint
Li, L., Huang, Y., & Luo, D. (2021) · arXiv (Cornell University) · Cited by 1
Dusparić, I., & Cardozo, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.06908 · Cited by 1
Persistent Reinforcement Learning via Subgoal CurriculaPreprint
Sharma, A., Gupta, A., Levine, S., Hausman, K., & Finn, C. (2021) · arXiv (Cornell University) · Cited by 1
Nonlinear Filtering and Reinforcement Learning-based Smart Autonomous Multi-agent Systems
Borah, K. J. (2024) · doi:10.32920/25412566 · Cited by 1
Multi-model reinforcement learning with online retrospective change point detectionPreprint
Chartouny, A., Khamassi, M., & Girard, B. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.05.13.653727 · Cited by 1
Toward Enactive Artificial Intelligence
Rafiee, B., & Sutton, R. (2026) · arXiv · doi:10.48550/arxiv.2605.24238
Space-sampled Value Decay: Forgetting Mechanisms for Non-stationary Reinforcement Learning
Störck, F., Hartmann, P., Hinder, F., Neumann, K., & Hammer, B. (2026) · arXiv · doi:10.48550/arxiv.2606.11797
Deliberation gated by opportunity cost adapts to context with urgencyPreprint
Touzel, M. P., Cisek, P., & Lajoie, G. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.07.31.452742
Automating Staged Rollout with Reinforcement Learning
Pritchard, S., Nagaraju, V., & Fiondella, L. (2022) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier55298.2022.9793526
Opponent modeling with trajectory representation clustering
Lv, Y., Zheng, Y. P., & Hao, J. (2022) · Intelligence & Robotics · doi:10.20517/ir.2022.09
Tutorial - Continual Learning beyond classification
Gepperth, A., & Lesort, T. (2022) · European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN) · doi:10.14428/esann/2022.es2022-4
Transformed-*: A domain-incremental lifelong learning scenario generation framework
Żurek, D., Corizzo, R., Karwatowski, M., Pietroń, M., & Faber, K. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191200
SKYNET 2023 Conception of the Artificial Super Intelligence Project. A System Approach. ver. 3Preprint
Novikov, A. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/kqt9p
MEWA: A Benchmark For Meta-Learning in Collaborative Working Agents
Stoican, R., Cangelosi, A., & Weisswange, T. H. (2023) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci52147.2023.10371913
Let’s Evolve Intelligence, Not Solutions
Hussain, T. S. (2024) · Genetic and evolutionary computation · doi:10.1007/978-981-99-8413-8_16
Pourshamsaei, H., Rimaz, M. S., & Nobakhti, A. (2024) · doi:10.1109/qicar61538.2024.10496597
Situation Awareness Based Continuous Time Learning Process and Convergence Analysis
Wang, Y., Wang, D., Lian, J., Xu, X., & Lian, L. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-97-3332-3_25
Approches pédagogiques en vue de stimuler la pensée critique et créative des étudiants
FRĂSINEANU, E. S., & Ilie, V. (2024) · Analele Universității din Craiova seria Psihologie-Pedagogie/Annals of the University of Craiova Series Psychology- Pedagogy · doi:10.52846/aucpp.2024.1.21
Continual Deep Reinforcement Learning for Financial Trading Using Bi-Directional Knowledge TransferPreprint
Katsikas, D., Passalis, N., & Tefas, A. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4907420
Mutschler, C., Kontes, G. D., & Rietsch, S. (2024) · Springer eBooks · doi:10.1007/978-3-031-64832-8_3 · Cited by 1
Continual Reinforcement Learning Without Replay Buffers
Krawczyk, A., Bagus, B., Denker, Y., & Gepperth, A. (2024) · IEEE International Conference on Intelligent Systems · doi:10.1109/is61756.2024.10705256 · Cited by 1
Graceful Task Adaptation with a Bi-hemispheric RL Agent
Nicholas, G., Kuhlmann, L., & Kowadlo, G. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0351-0_3
An Incremental Learning Method for PV Generation Prediction Based on MMD Data Screening
Xiao, Y., Pang, Z., Zhan, Z., Li, X., Li, Z., Li, H., & Ye, X. (2024) · International Conference on New Trends in Computational Intelligence (NTCI) · doi:10.1109/ntci64025.2024.10776132
Informative Performance Measures for Continual Reinforcement Learning
Denker, Y., Bagus, B., Krawczyk, A., & Gepperth, A. (2024) · IEEE International Conference on Intelligent Computer Communication and Processing (ICCP) · doi:10.1109/iccp63557.2024.10793039
Ma, Z., Cui, Y., & Huang, Y. (2024) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc58415.2024.10919663
Denker, Y., & Gepperth, A. (2025) · European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN) · doi:10.14428/esann/2025.es2025-33
Open-ended Deep Reinforcement Learning for a Bitcoin Trading Bot
Altermatt, A., Dornberger, R., & Hanne, T. (2025) · International Symposium on Computational and Business Intelligence (ISCBI) · doi:10.1109/iscbi64586.2025.11015421
Leuzzi, L., Bacciu, D., Hauert, S., Jones, S., & Cossu, A. (2025) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3712256.3726384
Requirements for Recognition and Rapid Response to Unfamiliar Events Outside of Agent Design Scope
Wray, R. E., Jones, S. J., & Laird, J. E. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-00800-8_27
Kulagin, V. A., Matsukatova, A. N., Rylkov, V. V., & Demin, V. A. (2025) · Russian Microelectronics · doi:10.1134/s1063739725600475
Adiabatic Reinforcement Learning
Osinenko, P., Yaremenko, G., Belov, D., & Gepperth, A. (2025) · International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar65334.2025.11338705
Tan, W., Lv, Q., Li, X., Sun, N., & Huang, N. (2026) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-026-02360-6
Zainol, N., & Wahid, H. A. (2025) · International Journal of Modern Education · doi:10.35631/ijmoe.728051
Rodrigues, H. D. N. (2023) · theses.fr (ABES) · doi:10.70675/a0210e97ze844z4f11z93e1zb64db1b962c8
Kuramoto Object-Centric Reinforcement Learning for Robotic Manipulation Tasks
Ugadiarov, L., & Panov, A. (2026) · Technologies · doi:10.3390/technologies14050266
A Novel Framework to Evaluate Generalization Performance of Deep Reinforcement Learning
Tian, Y., Wu, Q., & Chen, L. (2025) · China Automation Congress (CAC) · doi:10.1109/cac67268.2025.11487359
Wang, X., Luo, B., Hu, T., Cui, Y., & Feng, Y. (2026) · Systems Science & Control Engineering · doi:10.1080/21642583.2026.2672169 · Cited by 3
A Novel Q-Learning Architecture for Non-Stationary Markov Decision Processes
Clancy, D. J., & Fitch, N. (2026) · IEEE Aerospace Conference (AERO) · doi:10.1109/aero66936.2026.11520163
Dynamic epidemic control with continual learning via mass testing
Yu, H., Du, M., Zhang, Y., & Kong, N. (2026) · IISE Transactions · doi:10.1080/24725854.2026.2682881
Metadata
- Authors
- Khetarpal, K., Riemer, M., Rish, I., & Precup, D.
- Affiliations
- McGill University (CA)
- Mila - Quebec Artificial Intelligence Institute (CA)
- Venue
- Journal of Artificial Intelligence Research
- Publisher
- AI Access Foundation
- Published
- 2022-12-22
- Volume
- 75
- Pages
- 1401-1476
- ISSN
- 1076-9757 (ISSN-L)
- OpenAlex
- W3117215073
- Cited by
- 224
- References
- 474
- Citations by year
- 2026: 37 · 2025: 54 · 2024: 72 · 2023: 28 · 2022: 11
- FWCI
- 13.57
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Journal of Artificial Intelligence Research (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Canadian Institute for Advanced Research
- Canada Excellence Research Chairs, Government of Canada
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Keywords
- continual reinforcement learning, lifelong learning, incremental learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
