The DOI matches its Crossref record, with a small difference in title or year; 435 of the 474 references it cites were found in a registry or an index.
Towards Continual Reinforcement Learning: A Review and Perspectives
Open accessAbstract
In this article, we aim to provide a literature review of different formulations and approaches to continual reinforcement learning (RL), also known as lifelong or non-stationary RL. We begin by discussing our perspective on why RL is a natural fit for studying continual learning. We then provide a taxonomy of different continual RL formulations by mathematically characterizing two key properties of non-stationarity, namely, the scope and driver non-stationarity. This offers a unified view of various formulations. Next, we review and present a taxonomy of continual RL approaches. We go on to discuss evaluation of continual RL agents, providing an overview of benchmarks used in the literature and important metrics for understanding agent performance. Finally, we highlight open problems and challenges in bridging the gap between the current state of continual RL and findings in neuroscience. While still in its early days, the study of continual RL has the promise to develop better incremental reinforcement learners that can function in increasingly realistic applications where non-stationarity plays a vital role. These include applications such as those in the fields of healthcare, education, logistics, and robotics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Another line develops affordance theory in RL, where the environment constrains and suggests possible actions (Graves et al., 2022; Khetarpal et al., 2020). Rather than treating all actions as always available, agents learn which are afforded in each situation, mirroring selective, skillful responses in enactive perception.”
From Toward Enactive Artificial Intelligence (2026) · cites this paper as Khetarpal et al., 2020
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“While continual learning does not necessarily emphasize agent-gathered data or sustained interaction with the environment, it aligns with enactive views in emphasizing ongoing adaptation under non-stationarity.”
From Toward Enactive Artificial Intelligence (2026) · cites this paper as Khetarpal et al., 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636
Neural Architecture Search with Reinforcement LearningPreprint
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 5,705
McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256
Li, Z., & Hoiem, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2773081 · Cited by 4,202
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2018) · Science · doi:10.1126/science.aar6404 · Cited by 3,562 · cited 1×
“, 2016) and achieving super human performance (Silver et al., 2018), it is not immediately clear if the aforementioned benchmarks have the sufficient characteristics of desired environments for continual RL.”
BackgroundSchaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05952 · Cited by 3,120
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., & Singh, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00052-1 · Cited by 3,119
Self-citationContinual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
Markov games as a framework for multi-agent reinforcement learning
Littman, M. L. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50027-1 · Cited by 2,789
Daw, N. D., Niv, Y., & Dayan, P. (2005) · Nature Neuroscience · doi:10.1038/nn1560 · Cited by 2,533
A massively parallel architecture for a self-organizing neural pattern recognition machine
Carpenter, G. A., & Grossberg, S. (1987) · Computer Vision, Graphics, and Image Processing · doi:10.1016/s0734-189x(87)80014-2 · Cited by 2,525
Internal models in the cerebellum
Wolpert, D. M., Miall, R. C., & Kawato, M. (1998) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(98)01221-2 · Cited by 2,469
The Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3912 · Cited by 2,161
A Note on Measurement of Utility
Samuelson, P. A. (1937) · The Review of Economic Studies · doi:10.2307/2967612 · Cited by 2,140
Toward an architecture for never-ending language learning
Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka, E., & Mitchell, T. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7519 · Cited by 2,000
Curiosity-Driven Exploration by Self-Supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815
Model-based influences on humans’ choices and striatal prediction errors
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011) · Neuron · doi:10.1016/j.neuron.2011.02.027 · Cited by 1,779
Deep Reinforcement Learning That Matters
Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., & Meger, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11694 · Cited by 1,685
Self-citationA Continual Learning Survey: Defying Forgetting in Classification Tasks
Delange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3057446 · Cited by 1,647
Self-improving reactive agents based on reinforcement learning, planning and teaching
Lin, L.-J. (1992) · Machine Learning · doi:10.1007/bf00992699 · Cited by 1,564
Neuroscience-Inspired Artificial Intelligence
Hassabis, D., Kumaran, D., Summerfield, C., & Botvinick, M. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.011 · Cited by 1,557
Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition
Dietterich, T. G. (2000) · Journal of Artificial Intelligence Research · doi:10.1613/jair.639 · Cited by 1,526
Transfer Learning for Reinforcement Learning Domains: A Survey
Taylor, M. E., & Stone, P. (2009) · doi:10.5555/1577069.1755839 · Cited by 1,524
An Outline of a Theory of Affordances
Chemero, A. P. (2003) · Ecological Psychology · doi:10.1207/s15326969eco1502_5 · Cited by 1,363
Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Abbeel, P., & Zaremba, W. (2017) · arXiv · doi:10.48550/arxiv.1707.01495 · Cited by 1,355
Rusu, A. A., Rabinowitz, N. C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., & Hadsell, R. (2016) · arXiv · doi:10.48550/arxiv.1606.04671 · Cited by 1,307
Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010) · Neuron · doi:10.1016/j.neuron.2010.04.016 · Cited by 1,166
Reinforcement Learning with Unsupervised Auxiliary TasksPreprint
Jaderberg, M., Mnih, V., Czarnecki, W. M., Schaul, T., Leibo, J. Z., Silver, D., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05397 · Cited by 1,117
A perspective view and survey of meta-learning
Vilalta, R., & Drissi, Y. (2002) · Artificial Intelligence Review · doi:10.1023/a:1019956318069 · Cited by 1,108
IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner ArchitecturesPreprint
Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.01561 · Cited by 1,049
Human cerebellar activity reflecting an acquired internal model of a new tool
Imamizu, H., Miyauchi, S., Tamada, T., Sasaki, Y., Takino, R., Pütz, B., Yoshioka, T., & Kawato, M. (2000) · Nature · doi:10.1038/35003194 · Cited by 1,000
PathNet: Evolution Channels Gradient Descent in Super Neural NetworksPreprint
Fernando, C., Banarse, D., Blundell, C., Zwólš, Y., Ha, D., Rusu, A. A., Pritzel, A., & Wierstra, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.08734 · Cited by 966
A Simple Neural Attentive Meta-LearnerPreprint
Mishra, N., Rohaninejad, M., Chen, X., & Abbeel, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.03141 · Cited by 961
Meta-Learning in Neural Networks: A Survey
Hospedales, T. M., Antoniou, A., Micaelli, P., & Storkey, A. J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3079209 · Cited by 886
Efficient Lifelong Learning with A-GEM
Chaudhry, A., Ranzato, M., Rohrbach, M., & Elhoseiny, M. (2018) · arXiv · doi:10.48550/arxiv.1812.00420 · Cited by 852
Learning to Navigate in Complex EnvironmentsPreprint
Mirowski, P., Pascanu, R., Viola, F., Soyer, H., Ballard, A. J., Banino, A., Denil, M., Goroshin, R., Sifre, L., Kavukcuoglu, K., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.03673 · Cited by 847
Catastrophic forgetting, rehearsal and pseudorehearsal
ROBINS, A. (1995) · Connection Science · doi:10.1080/09540099550039318 · Cited by 833
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
Levine, S., Kumar, A., Tucker, G., & Fu, J. (2020) · arXiv · doi:10.48550/arxiv.2005.01643 · Cited by 824
A survey and critique of multiagent deep reinforcement learning
Hernández-Leal, P., Kartal, B., & Taylor, M. E. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09421-1 · Cited by 799
The Option-Critic Architecture
Bacon, P., Harb, J., & Precup, D. (2017) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v31i1.10916 · Cited by 779
Self-citationDyna, an Integrated Architecture for Learning, Planning, and Reacting
Sutton, R. S. (1991) · ACM SIGART Bulletin · doi:10.1145/122344.122377 · Cited by 777
Reinforcement learning in the brain
Niv, Y. (2009) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2008.12.005 · Cited by 775
Gradient Surgery for Multi-Task Learning
Yu, T., Kumar, S., Gupta, A., Levine, S., Hausman, K., & Finn, C. (2020) · arXiv · doi:10.48550/arxiv.2001.06782 · Cited by 755
(2016) · neural information processing systems · Cited by 735
Universal Value Function Approximators
Schaul, T., Horgan, D., Gregor, K., & Silver, D. (2015) · Cited by 701
Improving generalization for temporal difference learning: The successor representation
Dayan, P. (1993) · Neural Computation · doi:10.1162/neco.1993.5.4.613 · Cited by 693
Li, Z., & Hoiem, D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_37 · Cited by 659
Reptile: a Scalable Metalearning AlgorithmPreprint
Nichol, A., & Schulman, J. (2018) · arXiv (Cornell University) · Cited by 648
Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective
Botvinick, M. M., Niv, Y., & Barto, A. G. (2009) · Cognition · doi:10.1016/j.cognition.2008.08.011 · Cited by 646
ViZDoom: A Doom-based AI research platform for visual reinforcement learning
Kempka, M., Wydmuch, M., Runc, G., Toczek, J., & Jaśkowski, W. (2016) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2016.7860433 · Cited by 624
Reinforcement Learning with Hierarchies of Machines
Parr, R., & Russell, S. (1997) · Cited by 618
Intrinsically Motivated Reinforcement Learning
Singh, S. P., Barto, A. G., & Chentanez, N. (2005) · doi:10.21236/ada440280 · Cited by 614
Curious model-building control systems
Schmidhuber, J. (1991) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1991.170605 · Cited by 601
VIME: Variational Information Maximizing ExplorationPreprint
Houthooft, R., Chen, X., Duan, Y., Schulman, J., Turck, F. D., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.09674 · Cited by 597
Prefrontal cortex as a meta-reinforcement learning system
Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., Hassabis, D., & Botvinick, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0147-8 · Cited by 596
Self-Evaluation of Decision-Making: A General Bayesian Framework for Metacognitive Computation
Fleming, S. M., & Daw, N. D. (2017) · Psychological Review · doi:10.1037/rev0000045 · Cited by 584
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016) · arXiv · doi:10.48550/arxiv.1606.01540 · Cited by 582
Learning to reinforcement learn
Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., & Botvinick, M. (2016) · arXiv · doi:10.48550/arxiv.1611.05763 · Cited by 579
Learning to Learn without Forgetting by Maximizing Transfer and Minimizing InterferencePreprint
Riemer, M., Cases, I., Ajemian, R., Liu, M., Rish, I., Tu, Y., & Tesauro, G. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.11910 · Cited by 534
Self-citationExperience Replay for Continual Learning
Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T. P., & Wayne, G. (2018) · arXiv · doi:10.48550/arxiv.1811.11682 · Cited by 529
HEFT, H. (1989) · Journal for the Theory of Social Behaviour · doi:10.1111/j.1468-5914.1989.tb00133.x · Cited by 522
RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
Duan, Y., Schulman, J., Chen, X., Bartlett, P. L., Sutskever, I., & Abbeel, P. (2016) · arXiv · doi:10.48550/arxiv.1611.02779 · Cited by 503
SMASH: One-Shot Model Architecture Search through HyperNetworksPreprint
Brock, A., Lim, T., Ritchie, J. M., & Weston, N. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05344 · Cited by 500
Playing FPS Games with Deep Reinforcement Learning
Lample, G., & Chaplot, D. S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10827 · Cited by 493
Diversity is All You Need: Learning Skills without a Reward Function
Eysenbach, B., Gupta, A., Ibarz, J., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1802.06070 · Cited by 480
Multiple model-based reinforcement learning
Doya, K., Samejima, K., Katagiri, K.-I., & Kawato, M. (2002) · Neural Computation · doi:10.1162/089976602753712972 · Cited by 476
Embracing Change: Continual Learning in Deep Neural Networks
Hadsell, R., Rao, D., Rusu, A. A., & Pascanu, R. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.09.004 · Cited by 467
Variational Information Maximisation for Intrinsically Motivated Reinforcement LearningPreprint
Mohamed, S., & Rezende, D. J. (2015) · Neural Information Processing Systems · doi:10.48550/arxiv.1509.08731 · Cited by 460
Collins, A. G. E., & Frank, M. J. (2012) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2011.07980.x · Cited by 457
Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
Yu, T., Quillen, D., He, Z., Julian, R., Narayan, A., Shively, H., Bellathur, A., Hausman, K., Finn, C., & Levine, S. (2019) · arXiv · doi:10.48550/arxiv.1910.10897 · Cited by 450
A Study on Overfitting in Deep Reinforcement LearningPreprint
Zhang, C., Vinyals, O., Munos, R., & Bengio, S. (2018) · arXiv.org · doi:10.48550/arxiv.1804.06893 · Cited by 447
Progress & Compress: A scalable framework for continual learningPreprint
Schwarz, J., Luketina, J., Czarnecki, W. M., Grabska‐Barwińska, A., Teh, Y. W., Pascanu, R., & Hadsell, R. T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.06370 · Cited by 422
Bisk, Y., Holtzman, A., Thomason, J., Andreas, J., Bengio, Y., Chai, J. Y., Lapata, M., Lazaridou, A., May, J., Nisnevich, A., et al. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.703 · Cited by 421
Navigating the Affordance Landscape: Feedback Control as a Process Model of Behavior and Cognition
Pezzulo, G., & Cisek, P. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.03.013 · Cited by 419
The successor representation in human reinforcement learning
Momennejad, I., Russek, E. M., Cheong, J. H., Botvinick, M., Daw, N. D., & Gershman, S. J. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0180-8 · Cited by 408
Friend-or-Foe Q-learning in General-Sum Games
Littman, M. L. (2001) · Cited by 403
Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context VariablesPreprint
Rakelly, K., Zhou, A., Quillen, D., Finn, C., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.08254 · Cited by 403
Learning a synaptic learning rule
Bengio, Y., Bengio, S., & Cloutier, J. (1991) · IJCNN-91-Seattle International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1991.155621 · Cited by 402
Large-Scale Study of Curiosity-Driven Learning
Burda, Y., Edwards, H., Pathak, D., Storkey, A., Darrell, T., & Efros, A. A. (2018) · arXiv · doi:10.48550/arxiv.1808.04355 · Cited by 401
Meta-Learning Representations for Continual LearningPreprint
Javed, K., & White, M. (2019) · Neural Information Processing Systems · doi:10.48550/arxiv.1905.12588 · Cited by 397
Prioritized memory access explains planning and hippocampal replay
Mattar, M. G., & Daw, N. D. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0232-z · Cited by 393
Variational Intrinsic ControlPreprint
Gregor, K., Rezende, D. J., & Wierstra, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.07507 · Cited by 386
The ubiquity of model-based reinforcement learning
Doll, B. B., Simon, D. A., & Daw, N. D. (2012) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.08.003 · Cited by 380
Reinforcement Learning in Multidimensional Environments Relies on Attention Mechanisms
Niv, Y., Daniel, R., Geana, A., Gershman, S. J., Leong, Y. C., Radulescu, A., & Wilson, R. C. (2015) · The Journal of Neuroscience · doi:10.1523/jneurosci.2978-14.2015 · Cited by 362
Stochastic dynamic programming with factored representations
Boutilier, C., Dearden, R., & Goldszmidt, M. (2000) · Artificial Intelligence · doi:10.1016/s0004-3702(00)00033-3 · Cited by 361
Machado, M. C., Bellemare, M. G., Talvitie, E., Veness, J., Hausknecht, M., & Bowling, M. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.5699 · Cited by 359
Learning to Control Fast-Weight Memories: An Alternative to Dynamic Recurrent Networks
Schmidhuber, J. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.131 · Cited by 357
Learning task-state representations
Niv, Y. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0470-8 · Cited by 356
Frank, M. J., & Badre, D. (2011) · Cerebral Cortex · doi:10.1093/cercor/bhr114 · Cited by 340
Learning modular neural network policies for multi-task and multi-robot transfer
Devin, C., Gupta, A., Darrell, T., Abbeel, P., & Levine, S. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989250 · Cited by 324
Wang, R., Lehman, J., Clune, J., & Stanley, K. O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.01753 · Cited by 311
Sutton, R. S., Modayil, J., Delp, M., Degris, T., Pilarski, P. M., White, A., & Precup, D. (2011) · doi:10.5555/2031678.2031726 · Cited by 304
Self-citationSequential replay of nonspatial task states in the human hippocampus
Schuck, N. W., & Niv, Y. (2019) · Science · doi:10.1126/science.aaw5181 · Cited by 304
Continuous Adaptation via Meta-Learning in Nonstationary and Competitive EnvironmentsPreprint
Al-Shedivat, M., Bansal, T., Burda, Y., Sutskever, I., Mordatch, I., & Abbeel, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.03641 · Cited by 303
Transfer of Learning by Composing Solutions of Elemental Sequential Tasks
Singh, S. P. (1992) · Machine Learning · doi:10.1007/bf00992700 · Cited by 301
A Deep Hierarchical Approach to Lifelong Learning in Minecraft
Tessler, C., Givony, S., Zahavy, T., Mankowitz, D. J., & Mannor, S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10744 · Cited by 300
Empowerment: A Universal Agent-Centric Measure of Control
Klyubin, A. S., Polani, D., & Nehaniv, C. L. (2005) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2005.1554676 · Cited by 285
Towards a Unified Theory of State Abstraction for MDPs.
Li, L. C., Walsh, T. J., & Littman, M. L. (2006) · Cited by 283
Model-based learning protects against forming habits
Gillan, C. M., Otto, A. R., Phelps, E. A., & Daw, N. D. (2015) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-015-0347-6 · Cited by 283
Xu, J., & Zhu, Z. (2018) · ePrints Soton (University of Southampton) · doi:10.48550/arxiv.1805.12369 · Cited by 274
Modular Multitask Reinforcement Learning with Policy Sketches
Andreas, J., Klein, D., & Levine, S. (2016) · arXiv · doi:10.48550/arxiv.1611.01796 · Cited by 274
Rusu, A. A., Colmenarejo, S. G., Gulcehre, C., Desjardins, G., Kirkpatrick, J., Pascanu, R., Mnih, V., Kavukcuoglu, K., & Hadsell, R. (2015) · arXiv · doi:10.48550/arxiv.1511.06295 · Cited by 270
Multi-task reinforcement learning
Wilson, A., Fern, A., Ray, S., & Tadepalli, P. V. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273624 · Cited by 262
Embodiment theory and education: The foundations of cognition in perception and action
Kiefer, M., & Trumpp, N. M. (2012) · Trends in Neuroscience and Education · doi:10.1016/j.tine.2012.07.002 · Cited by 254
Fast Context Adaptation via Meta-LearningPreprint
Zintgraf, L., Shiarlis, K., Kurin, V., Hofmann, K., & Whiteson, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.03642 · Cited by 253
Reinforcement Learning to Play an Optimal Nash Equilibrium in Team Markov Games
Wang, X., & Sandholm, T. (2002) · Cited by 251
Intrinsic Motivation and Reinforcement Learning
Barto, A. G. (2012) · Springer eBooks · doi:10.1007/978-3-642-32375-1_2 · Cited by 243
Optimal learning: computational procedures for bayes-adaptive markov decision processes
Duff, M. O., & Barto, A. G. (2002) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 242
Planning under Uncertainty for Robotic Tasks with Mixed Observability
Ong, S. C. W., Png, S., Hsu, D., & Lee, W. S. (2010) · The International Journal of Robotics Research · doi:10.1177/0278364910369861 · Cited by 238
Hierarchical Solution of Markov Decision Processes using Macro-actionsPreprint
Hauskrecht, M., Meuleau, N., Kaelbling, L. P., Dean, T., & Boutilier, C. E. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.7381 · Cited by 230
Multi-Task Deep Reinforcement Learning with PopArt
Hessel, M., Soyer, H., Espeholt, L., Czarnecki, W. M., Schmitt, S., & van Hasselt, H. P. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013796 · Cited by 229
Meta-Gradient Reinforcement LearningPreprint
Xu, Z., van Hasselt, H. P., & Silver, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09801 · Cited by 227
An object-oriented representation for efficient reinforcement learning
Diuk, C., Cohen, A., & Littman, M. L. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390187 · Cited by 226
Automatic Goal Generation for Reinforcement Learning AgentsPreprint
Florensa, C., Held, D., Geng, X., & Abbeel, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.06366 · Cited by 221
Online Meta-LearningPreprint
Finn, C., Rajeswaran, A., Kakade, S. M., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.08438 · Cited by 221
Meta Learning Shared HierarchiesPreprint
Frans, K., Ho, J., Chen, X., Abbeel, P., & Schulman, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.09767 · Cited by 219
Symbolic dynamic programming for first-order MDPs
Boutilier, C. E., Reiter, R., & Price, B. (2001) · Cited by 218
SMASH: One-Shot Model Architecture Search through HyperNetworks
Brock, A., Lim, T., Ritchie, J. M., & Weston, N. J. (2018) · International Conference on Learning Representations · Cited by 217
Learning with Opponent-Learning Awareness
Foerster, J., Chen, R. Y., Al-Shedivat, M., Whiteson, S., Abbeel, P., & Mordatch, I. (2018) · International Joint Conference on Autonomous Agents and Multiagent Systems · doi:10.65109/hgwa8807 · Cited by 215
Convergence and No-Regret in Multiagent Learning
Bowling, M. (2004) · doi:10.7939/r3zs2kf41 · Cited by 213
Hooker, S. (2021) · Communications of the ACM · doi:10.1145/3467017 · Cited by 211
Model-Based Active ExplorationPreprint
Shyam, P., Jaśkowski, W., & Gomez, F. (2018) · International Conference on Machine Learning · doi:10.48550/arxiv.1810.12162 · Cited by 207
Near-optimal Regret Bounds for Reinforcement Learning
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1859902 · Cited by 207
Kaelbling, L. P. (1993) · Cited by 206
Dopamine reward prediction errors reflect hidden-state inference across time
Starkweather, C. K., Babayan, B. M., Uchida, N., & Gershman, S. J. (2017) · Nature Neuroscience · doi:10.1038/nn.4520 · Cited by 206
Multi-criteria Reinforcement Learning
Gábor, Z., Kalmár, Z., & Szepesvári, C. (1998) · Cited by 205
Go-Explore: a New Approach for Hard-Exploration Problems
Ecoffet, A., Huizinga, J., Lehman, J., Stanley, K. O., & Clune, J. (2019) · arXiv · doi:10.48550/arxiv.1901.10995 · Cited by 205
Learning Action Representations for Reinforcement LearningPreprint
Chandak, Y., Theocharous, G., Kostas, J. E., Jordan, E., & Thomas, P. (2019) · International Conference on Machine Learning · doi:10.48550/arxiv.1902.00183 · Cited by 205
Thomas, L. C., White, D. J., & Puterman, M. L. (1995) · Journal of the Operational Research Society · doi:10.1057/jors.1995.110 · Cited by 204
Mechanisms of Hierarchical Reinforcement Learning in Cortico-Striatal Circuits 2: Evidence from fMRI
Badre, D., & Frank, M. (2011) · Cerebral Cortex · doi:10.1093/cercor/bhr117 · Cited by 203
Discovering Structure in Multiple Learning Tasks: The TC Algorithm.
Thrun, S., & O’Sullivan, J. A. (1996) · Cited by 201
Leveraging Procedural Generation to Benchmark Reinforcement Learning
Cobbe, K., Hesse, C., Hilton, J., & Schulman, J. (2019) · arXiv · doi:10.48550/arxiv.1912.01588 · Cited by 196
Learning an Embedding Space for Transferable Robot Skills
Hausman, K., Springenberg, J. T., Wang, Z., Heess, N., & Riedmiller, M. (2018) · International Conference on Learning Representations · Cited by 189
Zero-Shot Task Generalization with Multi-Task Deep Reinforcement LearningPreprint
Oh, J., Singh, S. P., Lee, H., & Kohli, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05064 · Cited by 182
On the Optimization of a Synaptic Learning Rule
Samy Bengio, A., Bengio, Y., Cloutier, J., & Gescei, J. (2013) · doi:10.4324/9780203773833-22 · Cited by 181
Loss is its own Reward: Self-Supervision for Reinforcement LearningPreprint
Shelhamer, E., Mahmoudieh, P., Argus, M. J., & Darrell, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.07307 · Cited by 180
Finding Structure in Reinforcement Learning
Thrun, S., & Schwartz, A. (1994) · Cited by 176
Self-Improving Reactive Agents Based on Reinforcement Learning, Planning and Teaching
Lin, L.-J. (1992) · Springer eBooks · doi:10.1007/978-1-4615-3618-5_5 · Cited by 176
Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation
Justesen, N., Torrado, R. R., Bontrager, P., Khalifa, A., Togelius, J., & Risi, S. (2018) · OAR@UM (University of Malta) · doi:10.48550/arxiv.1806.10729 · Cited by 174
Schmidhuber, J., Zhao, J., & Wiering, M. (1997) · Machine Learning · doi:10.1023/a:1007383707642 · Cited by 171
Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task LearningPreprint
Rosenbaum, C., Klinger, T., & Riemer, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.01239 · Cited by 171
Self-citationAdapting bias by gradient descent: an incremental version of delta-bar-delta
Sutton, R. S. (1992) · Cited by 169
Evolved Policy GradientsPreprint
Houthooft, R., Chen, R. Y., Isola, P., Stadie, B. C., Wolski, F., Ho, J., & Abbeel, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04821 · Cited by 168
Mundt, M., Hong, Y., Pliushch, I., & Ramesh, V. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.01.014 · Cited by 167
Conflict-Averse Gradient Descent for Multi-task LearningPreprint
Liu, B., Liu, X., Jin, X., Stone, P., & Liu, Q. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.14048 · Cited by 167
Model-based hierarchical reinforcement learning and human action control
Botvinick, M., & Weinstein, A. (2014) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2013.0480 · Cited by 166
Bayesian Model-Agnostic Meta-LearningPreprint
Kim, T., Yoon, J., Dia, O., Kim, S., Bengio, Y., & Ahn, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.03836 · Cited by 164
Model-based predictions for dopamine
Langdon, A. J., Sharpe, M. J., Schoenbaum, G., & Niv, Y. (2018) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2017.10.006 · Cited by 161
Hippocampal Contributions to Model-Based Planning and Spatial Memory
Vikbladh, O. M., Meager, M. R., King, J. A., Blackmon, K., Devinsky, O., Shohamy, D., Burgess, N., & Daw, N. D. (2019) · Neuron · doi:10.1016/j.neuron.2019.02.014 · Cited by 159
Learning structures: Predictive representations, replay, and generalization
Momennejad, I. (2020) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2020.02.017 · Cited by 159
Generalizing plans to new environments in relational MDPs
Guestrin, C., Koller, D., Gearhart, C., & Kanodia, N. (2003) · Cited by 157
Solway, A., Diuk, C., Córdova, N. I., Yee, D. M., Barto, A. G., Niv, Y., & Botvinick, M. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003779 · Cited by 154
Dealing with non-stationary environments using context detection
da Silva, B. C., Basso, E. W., Bazzan, A. L. C., & Engel, P. M. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143872 · Cited by 154
Bayesian Decision Problems and Markov Chains.
Olshen, R. A., & Martin, J. J. (1968) · Journal of the American Statistical Association · doi:10.2307/2285906 · Cited by 154
Online Multi-Task Learning for Policy Gradient Methods
Ammar, H. B., Eaton, E., Ruvolo, P., & Taylor, M. E. (2014) · Cited by 153
Dopamine, uncertainty and TD learning
Niv, Y., Duff, M. O., & Dayan, P. (2005) · Behavioral and Brain Functions · doi:10.1186/1744-9081-1-6 · Cited by 153
Discovering latent causes in reinforcement learning
Gershman, S. J., Norman, K. A., & Niv, Y. (2015) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2015.07.007 · Cited by 151
Scaling Laws for Autoregressive Generative Modeling
Henighan, T., Kaplan, J., Katz, M., Chen, M., Hesse, C., Jackson, J., Jun, H., Brown, T. B., Dhariwal, P., Gray, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.14701 · Cited by 150
Teaching on a budget: agents advising agents in reinforcement learning
Torrey, L. A., & Taylor, M. E. (2013) · doi:10.5555/2484920.2485086 · Cited by 149
Doshi-Velez, F., & Konidaris, G. (2013) · International Joint Conference on Artificial Intelligence · doi:10.48550/arxiv.1308.3513 · Cited by 149
Offline replay supports planning in human reinforcement learning
Momennejad, I., Otto, A. R., Daw, N. D., & Norman, K. A. (2018) · eLife · doi:10.7554/elife.32548 · Cited by 149
Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play
Sukhbaatar, S., Lin, Z., Kostrikov, I., Synnaeve, G., Szlam, A., & Fergus, R. (2017) · arXiv · doi:10.48550/arxiv.1703.05407 · Cited by 149
Schmidhuber, J. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02565-5_4 · Cited by 147
Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., & Farhadi, A. (2020) · arXiv · doi:10.48550/arxiv.2006.14769 · Cited by 144
CHILD: A First Step Towards Continual Learning
Ring, M. B. (1997) · Machine Learning · doi:10.1023/a:1007331723572 · Cited by 143
Reinforcement learning algorithm for non-stationary environments
Padakandla, S., J., P. K., & Bhatnagar, S. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01758-5 · Cited by 143
R-IAC: Robust Intrinsically Motivated Exploration and Active Learning
Baranès, A., & Oudeyer, P. (2009) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2009.2037513 · Cited by 142
Efficient reinforcement learning in factored MDPs
Kearns, M. J., & Koller, D. (1999) · Cited by 142
Continual learning: A comparative study on how to defy forgetting in classification tasks.Preprint
Lange, M. D., Aljundi, R., Masana, M., Parisot, S., Xu, J., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2019) · arXiv (Cornell University) · Cited by 141
Doing more with less: meta-reasoning and meta-learning in humans and machines
Griffiths, T. L., Callaway, F., Chang, M. B., Grant, E., Krueger, P. M., & Lieder, F. (2019) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2019.01.005 · Cited by 139
Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning
Achiam, J., & Sastry, S. (2017) · arXiv · doi:10.48550/arxiv.1703.01732 · Cited by 139
Positive reward prediction errors during decision-making strengthen memory encoding
Jang, A. I., Nassar, M. R., Dillon, D. G., & Frank, M. J. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0597-3 · Cited by 138
Gotta Learn Fast: A New Benchmark for Generalization in RLPreprint
Nichol, A., Pfau, V., Hesse, C., Klimov, O., & Schulman, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.03720 · Cited by 136
Schmidhuber, J. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00313 · Cited by 135
Learning to Learn: Meta-Critic Networks for Sample Efficient LearningPreprint
Sung, F., Zhang, L., Xiang, T., Hospedales, T. M., & Yang, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.09529 · Cited by 132
Acute stress selectively reduces reward sensitivity
Berghorst, L. H., Bogdan, R., Frank, M. J., & Pizzagalli, D. A. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00133 · Cited by 131
On Learning Intrinsic Rewards for Policy Gradient MethodsPreprint
Zheng, Z., Oh, J., & Singh, S. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.06459 · Cited by 131
Using Semi-Distributed Representations to Overcome Catastrophic Forgetting in Connectionist Networks
French, R. M. (1991) · Cited by 130
ProMP: Proximal Meta-Policy SearchPreprint
Rothfuss, J., Lee, D. M., Clavera, I., Asfour, T., & Abbeel, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.06784 · Cited by 129
Planning to Explore via Self-Supervised World Models
Sekar, R., Rybkin, O., Daniilidis, K., Abbeel, P., Hafner, D., & Pathak, D. (2020) · arXiv · doi:10.48550/arxiv.2005.05960 · Cited by 129
Automatic Curriculum Learning For Deep RL: A Short Survey
Portelas, R., Colas, C., Weng, L., Hofmann, K., & Oudeyer, P.-Y. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/671 · Cited by 127
Automatic Goal Generation for Reinforcement Learning Agents
Florensa, C., Held, D., Geng, X., & Abbeel, P. (2018) · International Conference on Machine Learning · Cited by 127
Behaviour Suite for Reinforcement LearningPreprint
Osband, I., Doron, Y., Hessel, M., Aslanides, J., Sezener, E., Saraiva, A., McKinney, K., Lattimore, T., Szepesvári, C., Singh, S. P., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.03568 · Cited by 126
Minimax Regret Bounds for Reinforcement Learning
Azar, M. G., Osband, I., & Munos, R. (2017) · arXiv · doi:10.48550/arxiv.1703.05449 · Cited by 124
Experience Replay for Continual LearningPreprint
Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T., & Wayne, G. (2019) · arXiv (Cornell University) · Cited by 124
Search on the Replay Buffer: Bridging Planning and Reinforcement Learning
Eysenbach, B., Salakhutdinov, R., & Levine, S. (2019) · arXiv · doi:10.48550/arxiv.1906.05253 · Cited by 123
Evolutionary architecture search for deep multitask networks
Liang, J., Meyerson, E., & Miikkulainen, R. P. (2018) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3205455.3205489 · Cited by 122
Moderate levels of activation lead to forgetting in the think/no-think paradigm
Detre, G. J., Natarajan, A., Gershman, S. J., & Norman, K. A. (2013) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2013.02.017 · Cited by 120
Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RLPreprint
Nagabandi, A., Finn, C., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.07671 · Cited by 120
Research works printed but not linked: 26
- PMLR, 2021.
- David Lopez-Paz and Marc'Aurelio Ranzato. Gradient episodic memory for continuum learning. Advances in Neural Information Processing Systems, 2017.
- Morgan Kaufmann Publishers Inc., 1994. ISBN 1-55860-335-2. URL http://dl.acm.org/citation.cfm?id=3091574.3091594.
- Qi Cai, Zhuoran Yang, and Zhaoran Wang. Sample-efficient reinforcement learning for pomdps with linear function approximations. arXiv preprint arXiv:2204.09787, 2022. · arXiv
- Weitong Zhang, Jiafan He, Dongruo Zhou, Amy Zhang, and Quanquan Gu. Provably efficient representation learning in low-rank markov decision processes. arXiv preprint arXiv:2106.11935, 2021a. · arXiv
- Biwei Huang, Fan Feng, Chaochao Lu, Sara Magliacane, and Kun Zhang. Adarl: What, where, and how to adapt in transfer reinforcement learning. arXiv preprint arXiv:2107.02729, 2021. · arXiv
- PMLR, 2022.
- Ayush Jain, Norio Kosaka, Kyung-Min Kim, and Joseph J Lim. Know your action set: Learning action relations for reinforcement learning. In International Conference on Learning Representations, 2021.
- Jürgen Schmidhuber. Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-... hook. PhD thesis, Technische Universität München, 1987.
- Pascal Klink, Carlo D'Eramo, Jan Peters, and Joni Pajarinen. Boosted curriculum reinforce- ment learning. In International Conference on Learning Representations, 2021.
- Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. Advances in Neural Information Processing Systems, 33:5824-5836, 2020b.
- Cristian Buciluǎ, Rich Caruana, and Alexandru Niculescu-Mizil. Model compression. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 535-541. ACM, 2006.
- Maximilian Igl, Gregory Farquhar, Jelena Luketina, JW Böhmer, and Shimon Whiteson. Transient non-stationarity and generalisation in deep reinforcement learning. In 9th International Conference on Learning Representations, 2021.
- Youngmin Oh, Jinwoo Shin, Eunho Yang, and Sung Ju Hwang. Model-augmented prioritized experience replay. In International Conference on Learning Representations, 2021.
- Kevin Lu, Aditya Grover, Pieter Abbeel, and Igor Mordatch. Reset-free lifelong learning with skill-space planning. In International Conference on Learning Representations, 2021.
- David Abel, Dilip Arumugam, Lucas Lehnert, and Michael L Littman. Toward good abstractions for lifelong learning. In NIPS Workshop on Hierarchical Reinforcement Learning, 2017.
- Chi Jin, Akshay Krishnamurthy, Max Simchowitz, and Tiancheng Yu. Reward-free explo- ration for reinforcement learning. In International Conference on Machine Learning, pages 4870-4879. PMLR, 2020.
- Dagui Chen, Qi Yan, Shangqi Guo, Zhile Yang, Xin Su, and Feng Chen. Learning effective subgoals with multi-task hierarchical reinforcement learning. Scaling-Up Reinforce- ment Learning (SURL) Workshop. URl: http://surl. tirl. info/proceedings/SURL- 2019 paper 10. pdf, 2019.
- Lorenzo Moro, Amarildo Likmeta, Enrico Prati, Marcello Restelli, et al. Goal-directed planning via hindsight experience replay. In International Conference on Learning Representations, pages 1-16, 2022.
- Andrew Levy, George Konidaris, Robert Platt, and Kate Saenko. Learning multi-level hierarchies with hindsight. arXiv preprint arXiv:1712.00948, 2017. · arXiv
- Curran Associates, Inc., 2019. URL http://papers.nips.cc/paper/8458-meta-learning-representations-for -continual-learning.pdf.
- Frederic Kaplan and Pierre-Yves Oudeyer. Curiosity-driven development. In Proceedings of the International Workshop on Synergistic Intelligence Dynamics. Citeseer, 2006.
- Karl Cobbe, Oleg Klimov, Chris Hesse, Taehoon Kim, and John Schulman. Quantifying generalization in reinforcement learning. In International Conference on Machine Learning, pages 1282-1289. PMLR, 2019a.
- Hiroshi Imamizu. Separated modules for visuomotor control and learning in the cerebellum: a functional mri study. In NeuroImage: Third International Conference on Functional Mapping of the Human Brain, volume 5, page S598. Academic Press, 1997.
- Morgan Kaufmann Publishers Inc. ISBN 1558607781.
- Amy Greenwald and Keith Hall. Correlated-Q learning. In ICML, page 242-249. AAAI Press, 2003. ISBN 1577351894.
Cited by
Sorted by citations
A Comprehensive Survey of Continual Learning: Theory, Method and Application
Wang, L., Zhang, X., Su, H., & Zhu, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367329 · Cited by 1,113
Organoid intelligence (OI): the new frontier in biocomputing and intelligence-in-a-dish
Smirnova, L., Caffo, B. S., Gracias, D. H., Huang, Q., Pantoja, I. E. M., Tang, B., Zack, D. J., Berlinicke, C. A., Boyd, J. L., Harris, T. D., et al. (2023) · Frontiers in Science · doi:10.3389/fsci.2023.1017235 · Cited by 329
A Survey of Deep RL and IL for Autonomous Driving Policy Learning
Zhu, Z., & Zhao, H. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3134702 · Cited by 260
A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
Kirk, R., Zhang, A., Grefenstette, E., & Rocktäschel, T. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14174 · Cited by 193
“Khetarpal et al. (2020) motivated and surveyed continual reinforcement learning (CRL), which is closely related to ZSG in RL as both settings require adaptation to unseen tasks or environments; however, they explicitly do not discuss the zero-shot setting that is the concern of this paper (for more…”
BackgroundInfluentialAutonomous Drone Racing: A Survey
Hanover, D., Loquercio, A., Bauersfeld, L., Romero, Á., Pěnička, R., Song, Y., Cioffi, G., Kaufmann, E., & Scaramuzza, D. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3400838 · Cited by 173
Metadata
- Authors
- Khetarpal, K., Riemer, M., Rish, I., & Precup, D.
- Affiliations
- McGill University (CA)
- Mila - Quebec Artificial Intelligence Institute (CA)
- Venue
- Journal of Artificial Intelligence Research
- Publisher
- AI Access Foundation
- Published
- 2022-12-22
- Volume
- 75
- Pages
- 1401-1476
- ISSN
- 1076-9757 (ISSN-L)
- OpenAlex
- W3117215073
- Cited by
- 224
- References
- 474
- Citations by year
- 2026: 37 · 2025: 54 · 2024: 72 · 2023: 28 · 2022: 11
- FWCI
- 13.57
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Journal of Artificial Intelligence Research (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Canadian Institute for Advanced Research
- Canada Excellence Research Chairs, Government of Canada
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Keywords
- continual reinforcement learning, lifelong learning, incremental learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
