The DOI matches its DataCite record; 45 of the 45 references it cites were found in a registry or an index.
Learning to reinforcement learn
Open accessAbstract
In recent years deep reinforcement learning (RL) systems have attained superhuman performance in a number of challenging task domains. However, a major limitation of such applications is their demand for massive amounts of training data. A critical present objective is thus to develop deep RL methods that can adapt rapidly to new tasks. In the present work we introduce a novel approach to this challenge, which we refer to as deep meta-reinforcement learning. Previous work has shown that recurrent networks can support meta-learning in a fully supervised context. We extend this approach to the RL setting. What emerges is a system that is trained using one RL algorithm, but whose recurrent dynamics implement a second, quite separate RL procedure. This second, learned RL algorithm can differ from the original one in arbitrary ways. Importantly, because it is learned, it is configured to exploit structure in the training domain. We unpack these points in a series of seven proof-of-concept experiments, each of which examines a key aspect of deep meta-RL. We consider prospects for extending and scaling up the approach, and also point out some potentially important implications for neuroscience.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Even though finding an analytical expression of the Bayes-optimal policy is often impossible, it is straightforward to meta-learn an approximation to it (Duan et al., 2016; Wang et al., 2016).”
From Meta-learned models of cognition (2024) · cites this paper as Wang et al., 2016
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Of particular interest due to their potential and novelty are the following methods: curriculum learning [37], incremental environment complexity [39], and continual learning and policy distillation for multiple tasks [12].”
From Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (2020) · cites this paper as [37]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Owing to the more specific environment and sensor suites that are required for different navigation tasks, custom simulators have often been used [36], [37], or simulation worlds have been created using Unity, Unreal Engine, or Gazebo [39], [42].”
From Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (2020) · cites this paper as [37]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“While our exposition in this section focused on the supervised learning case, the same ideas can also be readily extended to the reinforcement learning setting (Duan et al., 2016; Wang et al., 2016).”
From Meta-learned models of cognition (2024) · cites this paper as Wang et al., 2016
The cited source says what the claim attributes to it.
In the citing paper
“One way to address these shortcomings are meta-RL approaches, where adaptive agents are pre-trained on diverse task distributions to significantly increase sample efficiency on new problems”
From XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX (2024) · cites this paper as Wang et al., 2016
The cited source says what the claim attributes to it.
In the citing paper
“we are providing near-single-file implementations of recurrent PPO (Schulman et al., 2017) for single-task environments and its extension to RL 2 (Duan et al., 2016; Wang et al., 2016) for meta-learning as baselines.”
From XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX (2024) · cites this paper as Wang et al., 2016
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Sim-to-real transfer for DRL policies can be applied to complex navigation tasks: from six-legged robots [37] to depth-based mapless navigation [39], including robots for soccer competitions [36].”
From Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (2020) · cites this paper as [37]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Self-citationReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Finite-time Analysis of the Multiarmed Bandit Problem
Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002) · Machine Learning · doi:10.1023/a:1013689704352 · Cited by 6,033
Neural Architecture Search with Reinforcement LearningPreprint
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 5,705
THOMPSON, W. R. (1933) · Biometrika · doi:10.1093/biomet/25.3-4.285 · Cited by 2,712
Research works printed but not linked: 5
- Marcos Economides, Zeb Kurth-Nelson, Annika Lübbert, Marc Guitart-Masip, and Raymond Dolan. Modelbased reasoning in humans becomes automatic with training. PLoS Computational Biology , 11(9):e1004463, 2015.
- Emilie Kaufmann, Olivier Cappé, and Aurélien Garivier. On bayesian upper confidence bounds for bandit problems. In Proc. of Int'l Conf. on Artificial Intelligence and Statistics, AISTATS , 2012a.
- Tor Lattimore and Rémi Munos. Bounded regret for finite-armed structured bandits. In Advances in Neural Information Processing Systems 27 , pages 550-558, 2014.
- Dan Russo and Benjamin Van Roy. Learning to optimize via information-directed sampling. In Advances in Neural Information Processing Systems 27 , pages 1583-1591, 2014.
- Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta-learning with memory-augmented neural networks. In Proceedings of The 33rd International Conference on Machine Learning , pages 1842-1850, 2016.
Cited by
Sorted by citations
Learning Transferable Architectures for Scalable Image Recognition
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00907 · Cited by 6,875
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
Building machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x16001837 · Cited by 2,550
Deep reinforcement learning for autonomous driving: A survey
Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Sallab, A. A. A., Yogamani, S., & Perez, P. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3054625 · Cited by 2,342
Searching for activation functions
Ramachandran, P., Zoph, B., & Le, Q. V. (2017) · arXiv · doi:10.48550/arxiv.1710.05941 · Cited by 2,335
Neuroscience-Inspired Artificial Intelligence
Hassabis, D., Kumaran, D., Summerfield, C., & Botvinick, M. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.011 · Cited by 1,557
Self-citationHutter, F., Kotthoff, L., & Vanschoren, J. (2019) · The Springer series on challenges in machine learning · doi:10.1007/978-3-030-05318-5 · Cited by 1,691
Multi-agent deep reinforcement learning: a survey
Gronauer, S., & Diepold, K. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-09996-w · Cited by 957
Meta-Learning in Neural Networks: A Survey
Hospedales, T. M., Antoniou, A., Micaelli, P., & Storkey, A. J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3079209 · Cited by 886
Meta-SGD: Learning to Learn Quickly for Few-Shot LearningPreprint
Li, Z., Zhou, F., Chen, F., & Li, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.09835 · Cited by 1,239
Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey
Zhao, W., Queralta, J. P., & Westerlund, T. (2020) · 2020 IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci47803.2020.9308468 · Cited by 941
Deep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762
Designing neural networks through neuroevolution
Stanley, K. O., Clune, J., Lehman, J., & Miikkulainen, R. P. (2018) · Nature Machine Intelligence · doi:10.1038/s42256-018-0006-z · Cited by 745
Reinforcement learning, fast and slow
Botvinick, M., Ritter, S., Wang, J. X., Kurth-Nelson, Z., Blundell, C., & Hassabis, D. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.02.006 · Cited by 667
Self-citationEmbedded Devices for Neuromorphic Time-Series Assessment
Mazumder, A. N., Hosseini, M., & Mohsenin, T. (2022) · Research Explorer (The University of Manchester) · doi:10.13016/m2ksy0-wxr3 · Cited by 662
Unity: A General Platform for Intelligent Agents
Juliani, A., Berges, V.-P., Teng, E., Cohen, A., Harper, J., Elion, C., Goy, C., Gao, Y., Henry, H., Mattar, M., et al. (2018) · arXiv · doi:10.48550/arxiv.1809.02627 · Cited by 626
“These complex tasks might display hierarchical structure, a hallmark of human intelligence (Botvinick, 2008), or vary from instance to instance, thus requiring meta-learning or generalization to solve (Wang et al., 2016).”
BackgroundPrefrontal cortex as a meta-reinforcement learning system
Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., Hassabis, D., & Botvinick, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0147-8 · Cited by 596
Self-citationEfficient Architecture Search by Network Transformation
Cai, H., Chen, T., Zhang, W., Yu, Y. H., & Wang, Y.-X. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11709 · Cited by 616
Embracing Change: Continual Learning in Deep Neural Networks
Hadsell, R., Rao, D., Rusu, A. A., & Pascanu, R. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.09.004 · Cited by 467
Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
Yu, T., Quillen, D., He, Z., Julian, R., Narayan, A., Shively, H., Bellathur, A., Hausman, K., Finn, C., & Levine, S. (2019) · arXiv · doi:10.48550/arxiv.1910.10897 · Cited by 450
Xavier, M. S., Tawk, C., Zolfagharian, A., Pinskier, J., Howard, D., Young, T. R., Lai, J., Harrison, S. M., Yong, Y. K., Bodaghi, M., et al. (2022) · IEEE Access · doi:10.1109/access.2022.3179589 · Cited by 475
Chen, X., Qu, G., Tang, Y., Low, S. H., & Li, N. (2022) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2022.3154718 · Cited by 465
Learning task-state representations
Niv, Y. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0470-8 · Cited by 356
How to train your MAMLPreprint
Antoniou, A., Edwards, H. S., & Storkey, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.09502 · Cited by 394
Learning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement Learning
Nagabandi, A., Clavera, I., Liu, S., Fearing, R. S., Abbeel, P., Levine, S., & Finn, C. (2018) · arXiv · doi:10.48550/arxiv.1803.11347 · Cited by 370
Advances and challenges in conversational recommender systems: A survey
Gao, C., Lei, W., He, X., de Rijke, M., & Chua, T. (2021) · AI Open · doi:10.1016/j.aiopen.2021.06.002 · Cited by 390
A Survey of Transfer Learning for Convolutional Neural Networks
Ribani, R., & Marengoni, M. (2019) · SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) · doi:10.1109/sibgrapi-t.2019.00010 · Cited by 343
A survey of deep meta-learning
Huisman, M., van Rijn, J. N., & Plaat, A. (2021) · Data Archiving and Networked Services (DANS) · doi:10.1007/s10462-021-10004-4#rightslink · Cited by 316
Inductive biases for deep learning of higher-level cognition
Goyal, A., & Bengio, Y. (2022) · Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rspa.2021.0068 · Cited by 300
DMRO: A Deep Meta Reinforcement Learning-Based Task Offloading Framework for Edge-Cloud Computing
Qu, G., Wu, H., Li, R., & Jiao, P. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3087258 · Cited by 303
Garnelo, M., Rosenbaum, D., Maddison, C. J., Ramalho, T., Saxton, D., Shanahan, M., Teh, Y. W., Rezende, D. J., & Eslami, S. M. A. (2018) · arXiv · doi:10.48550/arxiv.1807.01613 · Cited by 287
Federated Meta-Learning with Fast Convergence and Efficient CommunicationPreprint
Chen, F., Luo, M., Dong, Z., Li, Z., & He, X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.07876 · Cited by 387
Deep reinforcement learning: a survey
Wang, H.-N., Liu, N., Zhang, Y., Feng, D., Huang, F., Li, D., & Zhang, Y. (2020) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1900533 · Cited by 297
Deep learning for insider threat detection: Review, challenges and opportunities
Yuan, S., & Wu, X. (2021) · Computers & Security · doi:10.1016/j.cose.2021.102221 · Cited by 271
Deep reinforcement learning and its neuroscientific implications
Botvinick, M., Wang, J. X., Dabney, W., Miller, K. J., & Kurth-Nelson, Z. (2020) · Neuron · doi:10.1016/j.neuron.2020.06.014 · Cited by 269
Self-citationDeep reinforcement learning in computer vision: a comprehensive survey
Le, N., Rathour, V. S., Yamazaki, K., Luu, K., & Savvides, M. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-10061-9 · Cited by 271
Li, C. (2019) · Chapman & Hall/CRC eBooks · doi:10.1201/9781351006620-6 · Cited by 257
A Survey of Deep RL and IL for Autonomous Driving Policy Learning
Zhu, Z., & Zhao, H. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3134702 · Cited by 260
Continuous Adaptation via Meta-Learning in Nonstationary and Competitive EnvironmentsPreprint
Al-Shedivat, M., Bansal, T., Burda, Y., Sutskever, I., Mordatch, I., & Abbeel, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.03641 · Cited by 303
Human-like systematic generalization through a meta-learning neural network
Lake, B. M., & Baroni, M. (2023) · Nature · doi:10.1038/s41586-023-06668-3 · Cited by 230
Learning Navigation Behaviors End-to-End With AutoRL
Chiang, H.-T. L., Faust, A., Fišer, M., & Francis, A. G., Jr. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2899918 · Cited by 226
Closed-loop 4D-printed soft robots
Zolfagharian, A., Kaynak, A., & Kouzani, A. Z. (2019) · Materials & Design · doi:10.1016/j.matdes.2019.108411 · Cited by 231
Towards Continual Reinforcement Learning: A Review and Perspectives
Khetarpal, K., Riemer, M., Rish, I., & Precup, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13673 · Cited by 224
Ramezanpour, K., & Jagannath, J. (2022) · Computer Networks · doi:10.1016/j.comnet.2022.109358 · Cited by 216
Deep reinforcement learning in medical imaging: A literature review
Zhou, S., Le, N., Luu, K., Nguyen, H. V., & Ayache, N. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102193 · Cited by 227
One-Shot Imitation LearningPreprint
Duan, Y., Andrychowicz, M., Stadie, B. C., Ho, O. J., Schneider, J., Sutskever, I., Abbeel, P., & Zaremba, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07326 · Cited by 473
Meta-Reinforcement Learning of Structured Exploration StrategiesPreprint
Gupta, A., Mendonca, R., Liu, Y., Abbeel, P., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.07245 · Cited by 265
A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
Kirk, R., Zhang, A., Grefenstette, E., & Rocktäschel, T. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14174 · Cited by 193
Resource Management in Wireless Networks via Multi-Agent Deep Reinforcement Learning
Naderializadeh, N., Sydir, J. J., Simsek, M., & Nikopour, H. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3051163 · Cited by 212
Deep reinforcement learning in recommender systems: A survey and new perspectives
Chen, X., Yao, L., McAuley, J., Zhou, G., & Wang, X. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110335 · Cited by 288
StarCraft Micromanagement With Reinforcement Learning and Curriculum Transfer Learning
Shao, K., Zhu, Y., & Zhao, D. (2018) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2018.2823329 · Cited by 177
Ding, Y., Ma, L., Ma, J., Suo, M., Tao, L., Cheng, Y., & Lü, C. (2019) · Advanced Engineering Informatics · doi:10.1016/j.aei.2019.100977 · Cited by 179
MetaLight: Value-Based Meta-Reinforcement Learning for Traffic Signal Control
Zang, X., Yao, H., Zheng, G., Xu, N., Xu, K. M., & Li, Z. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5467 · Cited by 172
Meta-learning in natural and artificial intelligence
Wang, J. X. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.01.002 · Cited by 171
Self-citationMeta-learning approaches for learning-to-learn in deep learning: A survey
Tian, Y., Zhao, X., & Huang, W. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.04.078 · Cited by 169
A Survey on Policy Search Algorithms for Learning Robot Controllers in a Handful of Trials
Chatzilygeroudis, K., Vassiliades, V., Stulp, F., Calinon, S., & Mouret, J.-B. (2020) · IEEE Transactions on Robotics · doi:10.1109/tro.2019.2958211 · Cited by 160
Finn, C., & Levine, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.11622 · Cited by 195
Liu, R., Gao, J., Zhang, J., Meng, D., & Lin, Z. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3132674 · Cited by 190
Reinforcement learning for robot research: A comprehensive review and open issues
Zhang, T., & Mo, H. (2021) · International Journal of Advanced Robotic Systems · doi:10.1177/17298814211007305 · Cited by 173
Evolved Policy GradientsPreprint
Houthooft, R., Chen, R. Y., Isola, P., Stadie, B. C., Wolski, F., Ho, J., & Abbeel, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04821 · Cited by 168
Learned Optimizers that Scale and GeneralizePreprint
Wichrowska, O., Maheswaranathan, N., Hoffman, M. W., Colmenarejo, S. G., Denil, M., de Freitas, N., & Sohl‐Dickstein, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04813 · Cited by 174
Reinforcement learning for angle-only intercept guidance of maneuvering targets
Gaudet, B., Furfaro, R., & Linares, R. (2020) · Aerospace Science and Technology · doi:10.1016/j.ast.2020.105746 · Cited by 147
Unsupervised Predictive Memory in a Goal-Directed AgentPreprint
Wayne, G., Hung, C.-C., Amos, D., Mirza, M., Ahuja, A., Grabska‐Barwińska, A., Rae, J. W., Mirowski, P., Leibo, J. Z., Santoro, A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.10760 · Cited by 137
Self-citationDomain Adaptive Dialog Generation via Meta Learning
Qian, K., & Yu, Z. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1253 · Cited by 134
Doing more with less: meta-reasoning and meta-learning in humans and machines
Griffiths, T. L., Callaway, F., Chang, M. B., Grant, E., Krueger, P. M., & Lieder, F. (2019) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2019.01.005 · Cited by 139
Robot Learning From Randomized Simulations: A Review
Muratore, F., Ramos, F., Turk, G., Yu, W., Gienger, M., & Peters, J. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.799893 · Cited by 128
Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RLPreprint
Nagabandi, A., Finn, C., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.07671 · Cited by 120
Assessing Generalization in Deep Reinforcement Learning
Packer, C., Gao, K., Kos, J., Krähenbühl, P., Koltun, V., & Song, D. (2018) · arXiv · doi:10.48550/arxiv.1810.12282 · Cited by 114
Federated Learning and Meta Learning: Approaches, Applications, and Directions
Liu, X., Deng, Y., Nallanathan, A., & Bennis, M. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3330910 · Cited by 116
Adaptive guidance and integrated navigation with reinforcement meta-learning
Gaudet, B., Linares, R., & Furfaro, R. (2020) · Acta Astronautica · doi:10.1016/j.actaastro.2020.01.007 · Cited by 111
“In this proposal, we use a different approach similar to that employed in [7] using a recurrent policy and value function.”
MethodA survey on deep reinforcement learning for audio-based applications
Latif, S., Cuayáhuitl, H., Pervez, F., Shamshad, F., Ali, H. S., & Cambria, E. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10224-2 · Cited by 126
MetaCleaner: Learning to Hallucinate Clean Representations for Noisy-Labeled Visual Recognition
Zhang, W., Wang, Y., & Qiao, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00755 · Cited by 106
Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey
Majid, A. Y., Majid, A. Y., Saaybi, S., van Rietbergen, T., François-Lavet, V., Prasad, R. R. V., & Verhoeven, C. J. M. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3264540 · Cited by 124
Differentiable plasticity: training plastic neural networks with backpropagationPreprint
Miconi, T., Clune, J., & Stanley, K. O. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.02464 · Cited by 124
CEM-RL: Combining evolutionary and gradient-based methods for policy searchPreprint
Pourchot, A., & Sigaud, O. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.01222 · Cited by 127
An Active Inference Approach to Modeling Structure Learning: Concept Learning as an Example Case
Smith, R. S., Schwartenbeck, P., Parr, T., & Friston, K. J. (2020) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2020.00041 · Cited by 119
Feature Control as Intrinsic Motivation for Hierarchical Reinforcement Learning
Dilokthanakul, N., Kaplanis, C., Pawlowski, N., & Shanahan, M. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2891792 · Cited by 100
Deep Meta-Learning: Learning to Learn in the Concept SpacePreprint
Zhou, F., Wu, B., & Li, Z. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03596 · Cited by 136
Compositional generalization through meta sequence-to-sequence learningPreprint
Lake, B. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.05381 · Cited by 117
Automated Reinforcement Learning (AutoRL): A Survey and Open Problems
Parker-Holder, J., Rajan, R., Song, X., Biedenkapp, A., Miao, Y., Eimer, T., Zhang, B., Nguyen, V., Calandra, R., Faust, A., et al. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13596 · Cited by 114
Obstacle Tower: A Generalization Challenge in Vision, Control, and Planning
Juliani, A., Khalifa, A., Berges, V.-P., Harper, J., Teng, E., Henry, H., Crespi, A., Togelius, J., & Lange, D. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/373 · Cited by 93
Understanding Human Intelligence through Human Limitations
Griffiths, T. L. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.09.001 · Cited by 88
Gaudet, B., Linares, R., & Furfaro, R. (2020) · Acta Astronautica · doi:10.1016/j.actaastro.2020.02.036 · Cited by 93
MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at ScalePreprint
Kalashnikov, D., Varley, J., Chebotar, Y., Swanson, B., Jonschkowski, R., Finn, C., Levine, S., & Hausman, K. (2021) · arXiv.org · doi:10.48550/arxiv.2104.08212 · Cited by 344
Reinforcement Learning on Graphs: A Survey
Nie, M., Chen, D., & Wang, D. (2023) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2022.3222545 · Cited by 99
A survey on reinforcement learning in aviation applications
Razzaghi, P., Tabrizian, A., Guo, W., Chen, S., Taye, A. G., Thompson, E. L., Bregeon, A., Baheri, A., & Wei, P. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108911 · Cited by 115
Multiagent Meta-Reinforcement Learning for Adaptive Multipath Routing Optimization
Chen, L., Hu, B., Guan, Z., Zhao, L. Y., & Shen, X. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3070584 · Cited by 84
Meta-learning, social cognition and consciousness in brains and machines
Langdon, A., Botvinick, M., Nakahara, H., Tanaka, K., Matsumoto, M., & Kanai, R. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.10.004 · Cited by 87
Self-citationClune, J. (2019) · arXiv · doi:10.48550/arxiv.1905.10985 · Cited by 82
Policy search in continuous action domains: An overview
Sigaud, O., & Stulp, F. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.011 · Cited by 89
Learning to Continually LearnPreprint
Beaulieu, S., Frati, L., Miconi, T., Lehman, J., Stanley, K. O., Clune, J., & Cheney, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.09571 · Cited by 80
Yang, S., Gao, T. Y., Wang, J., Deng, B., Azghadi, M. R., Lei, T., & Linares-Barranco, B. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.850945 · Cited by 78
Self-citationYang, N., Chen, S., Zhang, H., & Berry, R. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3405075 · Cited by 83
Pyramid Convolutional RNN for MRI Image Reconstruction
Chen, E. Z., Wang, P., Chen, X., Chen, T., & Sun, S. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3153849 · Cited by 69
Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning
Song, X., Yang, Y., Choromański, K., Caluwaerts, K., Gao, W., Finn, C., & Tan, J. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341571 · Cited by 73
Meta-Reinforcement Learning in Non-Stationary and Dynamic Environments
Bing, Z., Lerch, D., Huang, K., & Knoll, A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3185549 · Cited by 67
On data-driven modeling and control in modern power grids stability: Survey and perspective
Gong, X., Wang, X., & Cao, B. (2023) · Applied Energy · doi:10.1016/j.apenergy.2023.121740 · Cited by 67
Meta-reinforcement learning via orbitofrontal cortex
Hattori, R., Hedrick, N. G., Jain, A., Chen, S., You, H., Hattori, M., Choi, J., Lim, B. K., Yasuda, R., & Komiyama, T. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01485-3 · Cited by 75
Unsupervised Meta-Learning for Reinforcement LearningPreprint
Gupta, A., Eysenbach, B., Finn, C., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.04640 · Cited by 104
Learning to Generalize from Sparse and Underspecified RewardsPreprint
Agarwal, R., Chen, L., Schuurmans, D., & Norouzi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07198 · Cited by 66
Metadata
- Authors
- Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., & Botvinick, M.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2016
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1611.05763
- OpenAlex
- W2550182557
- Cited by
- 579
- References
- 32
- Citations by year
- 2026: 2 · 2025: 17 · 2024: 47 · 2023: 59 · 2022: 79 · 2021: 104 · 2020: 117 · 2019: 69 · 2018: 61 · 2017: 22 · 2016: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- meta-learning, recurrent neural networks, training data efficiency, learned dynamics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
