The DOI matches its DataCite record; 45 of the 45 references it cites were found in a registry or an index.
Learning to reinforcement learn
Open accessAbstract
In recent years deep reinforcement learning (RL) systems have attained superhuman performance in a number of challenging task domains. However, a major limitation of such applications is their demand for massive amounts of training data. A critical present objective is thus to develop deep RL methods that can adapt rapidly to new tasks. In the present work we introduce a novel approach to this challenge, which we refer to as deep meta-reinforcement learning. Previous work has shown that recurrent networks can support meta-learning in a fully supervised context. We extend this approach to the RL setting. What emerges is a system that is trained using one RL algorithm, but whose recurrent dynamics implement a second, quite separate RL procedure. This second, learned RL algorithm can differ from the original one in arbitrary ways. Importantly, because it is learned, it is configured to exploit structure in the training domain. We unpack these points in a series of seven proof-of-concept experiments, each of which examines a key aspect of deep meta-RL. We consider prospects for extending and scaling up the approach, and also point out some potentially important implications for neuroscience.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Even though finding an analytical expression of the Bayes-optimal policy is often impossible, it is straightforward to meta-learn an approximation to it (Duan et al., 2016; Wang et al., 2016).”
From Meta-learned models of cognition (2024) · cites this paper as Wang et al., 2016
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Of particular interest due to their potential and novelty are the following methods: curriculum learning [37], incremental environment complexity [39], and continual learning and policy distillation for multiple tasks [12].”
From Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (2020) · cites this paper as [37]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Owing to the more specific environment and sensor suites that are required for different navigation tasks, custom simulators have often been used [36], [37], or simulation worlds have been created using Unity, Unreal Engine, or Gazebo [39], [42].”
From Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (2020) · cites this paper as [37]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“While our exposition in this section focused on the supervised learning case, the same ideas can also be readily extended to the reinforcement learning setting (Duan et al., 2016; Wang et al., 2016).”
From Meta-learned models of cognition (2024) · cites this paper as Wang et al., 2016
The cited source says what the claim attributes to it.
In the citing paper
“One way to address these shortcomings are meta-RL approaches, where adaptive agents are pre-trained on diverse task distributions to significantly increase sample efficiency on new problems”
From XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX (2024) · cites this paper as Wang et al., 2016
The cited source says what the claim attributes to it.
In the citing paper
“we are providing near-single-file implementations of recurrent PPO (Schulman et al., 2017) for single-task environments and its extension to RL 2 (Duan et al., 2016; Wang et al., 2016) for meta-learning as baselines.”
From XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX (2024) · cites this paper as Wang et al., 2016
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Sim-to-real transfer for DRL policies can be applied to complex navigation tasks: from six-legged robots [37] to depth-based mapless navigation [39], including robots for soccer competitions [36].”
From Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (2020) · cites this paper as [37]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Self-citationReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Finite-time Analysis of the Multiarmed Bandit Problem
Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002) · Machine Learning · doi:10.1023/a:1013689704352 · Cited by 6,033
Neural Architecture Search with Reinforcement LearningPreprint
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 5,705
THOMPSON, W. R. (1933) · Biometrika · doi:10.1093/biomet/25.3-4.285 · Cited by 2,712
Research works printed but not linked: 5
- Marcos Economides, Zeb Kurth-Nelson, Annika Lübbert, Marc Guitart-Masip, and Raymond Dolan. Modelbased reasoning in humans becomes automatic with training. PLoS Computational Biology , 11(9):e1004463, 2015.
- Emilie Kaufmann, Olivier Cappé, and Aurélien Garivier. On bayesian upper confidence bounds for bandit problems. In Proc. of Int'l Conf. on Artificial Intelligence and Statistics, AISTATS , 2012a.
- Tor Lattimore and Rémi Munos. Bounded regret for finite-armed structured bandits. In Advances in Neural Information Processing Systems 27 , pages 550-558, 2014.
- Dan Russo and Benjamin Van Roy. Learning to optimize via information-directed sampling. In Advances in Neural Information Processing Systems 27 , pages 1583-1591, 2014.
- Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta-learning with memory-augmented neural networks. In Proceedings of The 33rd International Conference on Machine Learning , pages 1842-1850, 2016.
Cited by
Sorted by citations
Items 201–300 of 592
Adaptive Learning Knowledge Networks for Few-Shot Learning
Yan, M. (2019) · IEEE Access · doi:10.1109/access.2019.2934694 · Cited by 24
Learning to Explore with Meta-Policy GradientPreprint
Xu, T.-B., Liu, Q., Zhao, L. L., & Peng, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.05044 · Cited by 27
Auto-Meta: Automated Gradient Based Meta Learner SearchPreprint
Kim, J., Lee, S., Kim, S., Cha, M., Lee, J. K., Choi, Y., Choi, Y.-S., Cho, D.-Y., & Kim, J. W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.06927 · Cited by 24
Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task LearningPreprint
Huang, W., Mordatch, I., Abbeel, P., & Pathak, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.03062 · Cited by 24
Tal, A., Bloch, A., Cohen-Dallal, H., Aviv, O., Ashkenazi, S. S., Bar, M., & Vakil, E. (2021) · Scientific Reports · doi:10.1038/s41598-021-85842-x · Cited by 23
Adaptive Approach Phase Guidance for a Hypersonic Glider via Reinforcement Meta Learning
Gaudet, B., Drozd, K. M., & Furfaro, R. (2022) · AIAA SciTech Forum · doi:10.2514/6.2022-2214 · Cited by 23
Chen, X., Yao, L., McAuley, J., Zhou, G., & Wang, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.03540 · Cited by 23
Meta-Reinforcement Learning in Nonstationary and Nonparametric Environments
Bing, Z., Knak, L., Cheng, L., Morin, F. O., Huang, K., & Knoll, A. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3270298 · Cited by 28
Understanding the development of reward learning through the lens of meta-learning
Nussenbaum, K., & Hartley, C. A. (2024) · Nature Reviews Psychology · doi:10.1038/s44159-024-00304-1 · Cited by 35
AutoLoss: Learning Discrete Schedules for Alternate OptimizationPreprint
Xu, H., Zhang, H., Hu, Z., Liang, X., Salakhutdinov, R., & Xing, E. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.02442 · Cited by 27
Shaking the foundations: delusions in sequence models for interaction and controlPreprint
Ortega, P. A., Kunesch, M., Delétang, G., Genewein, T., Grau-Moya, J., Veness, J., Buchli, J., Degrave, J., Piot, B., Pérolat, J., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.10819 · Cited by 79
Sequential Mastery of Multiple Visual Tasks: Networks Naturally Learn to Learn and Forget to Forget
Davidson, G., & Mozer, M. C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00930 · Cited by 22
Summerfield, C. (2022) · doi:10.1093/oso/9780192843883.001.0001 · Cited by 24
Learning to Learn: Hierarchical Meta-Critic Networks
Xu, Z., Cao, L., & Chen, X. (2019) · IEEE Access · doi:10.1109/access.2019.2914469 · Cited by 21
Exploration by Maximizing Renyi Entropy for Reward-Free RL Framework
Zhang, C., Cai, Y., Huang, L., & Li, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17297 · Cited by 27
Learning to Learn Variational Semantic MemoryPreprint
Zhen, X., Du, Y., Xiong, H., Qiu, Q., Snoek, C. G. M., & Shao, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.10341 · Cited by 24
Adaptive meta-reinforcement learning for AUVs 3D guidance and control under unknown ocean currents
Jiang, Y. L., Zhang, K., Zhao, M., & Qin, H. (2024) · Ocean Engineering · doi:10.1016/j.oceaneng.2024.118498 · Cited by 20
Robust interplanetary trajectory design under multiple uncertainties via meta-reinforcement learning
Federici, L., & Zavoli, A. (2023) · Acta Astronautica · doi:10.1016/j.actaastro.2023.10.018 · Cited by 23
Jointly Training and Pruning CNNs via Learnable Agent Guidance and Alignment
Ganjdanesh, A., Gao, S., & Huang, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01520 · Cited by 22
Subramoney, A., Scherr, F., & Maass, W. (2021) · Natural computing series · doi:10.1007/978-981-13-1687-6_3 · Cited by 21
EgoMap: Projective Mapping and Structured Egocentric Memory for Deep RL
Beeching, E., Dibangoye, J., Simonin, O., & Wolf, C. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-67661-2_31 · Cited by 27
A neural circuit architecture for rapid behavioral flexibility in goal-directed navigation
Dan, C., Hulse, B. K., Kappagantula, R., Jayaraman, V., & Hermundstad, A. M. (2021) · doi:10.1101/2021.08.18.456004 · Cited by 51
“Recent RL studies have explored how such inductive biases might be constructed by learning common features across different learning tasks [115, 116], a process known as learning to learn [117].”
BackgroundHuman-Timescale Adaptation in an Open-Ended Task Space
Adaptive Agent Team, Bauer, J., Baumli, K., Baveja, S., Behbahani, F., Bhoopchand, A., Bradley-Schmieg, N., Chang, M., Clay, N., Collister, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2301.07608 · Cited by 19
“We focus on memory-based meta-RL, building upon Duan et al. (2017) and Wang et al. (2016), who showed that if an agent observes rewards and terminations, and the memory does not reset, a memory-based policy can implement a learning algorithm.”
MethodOverview of Meta-Reinforcement Learning Research
Shengguang, P. (2020) · International Conference on Information Technology and Computer Application (ITCA) · doi:10.1109/itca52113.2020.00019 · Cited by 20
Multi-agent policy transfer via task relationship modeling
Qin, R., Chen, F., Wang, T., Yuan, L., Wu, X., Kang, Y., Zhang, Z., Zhang, C., & Yu, Y. (2024) · Science China Information Sciences · doi:10.1007/s11432-023-3862-1 · Cited by 28
Lansdell, B., & Kording, K. P. (2019) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2019.04.005 · Cited by 22
Lifelong Learning with a Changing Action Set
Chandak, Y., Theocharous, G., Nota, C., & Thomas, P. S. (2020) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v34i04.5739 · Cited by 20
The Faults in Our Pi Stars: Security Issues and Open Challenges in Deep Reinforcement LearningPreprint
Behzadan, V., & Munir, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.10369 · Cited by 25
Toward Multimodal Model-Agnostic Meta-LearningPreprint
Vuorio, R., Sun, S.-H., Hu, H., & Lim, J. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.07172 · Cited by 22
Sample-Efficient Reinforcement Learning via Counterfactual-Based Data AugmentationPreprint
Lu, C., Huang, B., Wang, K., Hernández-Lobato, J. M., Zhang, K., & Schölkopf, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.09092 · Cited by 58
Meta Learning Backpropagation And Improving It
Kirsch, L., & Schmidhuber, J. (2020) · arXiv · doi:10.48550/arxiv.2012.14905 · Cited by 19
Robertazzi, F., Vissani, M., Schillaci, G., & Falotico, E. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.06.020 · Cited by 19
Wu, J., Liu, X., & Chen, S. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110160 · Cited by 19
Deep Reinforcement Learning on a Budget: 3D Control and Reasoning Without a Supercomputer
Beeching, E., Debangoye, J., Simonin, O., & Wolf, C. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412212 · Cited by 23
Learning to Learn Cropping Models for Different Aspect Ratio Requirements
Li, D., Zhang, J., & Huang, K. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01270 · Cited by 21
Exploring the Task Cooperation in Multi-goal Visual Navigation
Wu, Y., Rao, Z., Zhang, W., Lu, S., Lu, W., & Zha, Z.-J. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/86 · Cited by 18
Zhang, S., Wen, L., Peng, H., & Tseng, H. E. (2021) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc48978.2021.9564972 · Cited by 18
Fleet planning under demand and fuel price uncertainty using actor–critic reinforcement learning
Geursen, I. L., Santos, B. F., & Yorke‐Smith, N. (2023) · Journal of Air Transport Management · doi:10.1016/j.jairtraman.2023.102397 · Cited by 18
Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian ProcessesPreprint
Xu, M., Ding, W., Zhu, J., Liu, Z., Chen, B., & Zhao, D. X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.11441 · Cited by 18
Offline Meta Learning of ExplorationPreprint
Dorfman, R., Shenfeld, I., & Tamar, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.02598 · Cited by 41
Asymmetric self-play for automatic goal discovery in robotic manipulation
OpenAI, O., Plappert, M., Sampedro, R., Xu, T., Akkaya, I., Kosaraju, V., Welinder, P., D'Sa, R., Petron, A., Pinto, H. P. D. O., et al. (2021) · arXiv · doi:10.48550/arxiv.2101.04882 · Cited by 18
Agent Modelling under Partial Observability for Deep Reinforcement LearningPreprint
Papoudakis, G., Christianos, F., & Albrecht, S. V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.09447 · Cited by 22
Context-Based Meta-Reinforcement Learning With Bayesian Nonparametric Models
Bing, Z., Yun, Y., Huang, K., & Knoll, A. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3386780 · Cited by 17
Transfer Reinforcement Learning Across Homotopy Classes
Cao, Z., Kwon, M., & Sadigh, D. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3057050 · Cited by 17
Rectified Meta-learning from Noisy Labels for Robust Image-based Plant Disease Classification
Zhai, D., Shi, R., Jiang, J., & Liu, X. (2022) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3472809 · Cited by 23
Guided Feature Transformation (GFT): A Neural Language Grounding Module for Embodied AgentsPreprint
Yu, H., Lian, X., Zhang, H., & Xu, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.08329 · Cited by 17
Unsupervised Learning of Neural Networks to Explain Neural NetworksPreprint
Zhang, Q., Yu, Y., Liu, Y., Wu, Y., & Zhu, S. C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.07468 · Cited by 24
Variational Autoencoders for Opponent Modeling in Multi-Agent SystemsPreprint
Papoudakis, G., & Albrecht, S. V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.10829 · Cited by 36
Rethinking the hippocampal cognitive map as a meta-learning computational module
Ambrogioni, L., & Ólafsdóttir, H. F. (2023) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2023.05.011 · Cited by 16
Transforming Software Testing in the US: Generative AI Models for Realistic User Simulation
Islam, S. A. M., Bari, M. S., & Sarkar, A. (2024) · Journal of Artificial Intelligence General science (JAIGS) · doi:10.60087/jaigs.v6i1.292 · Cited by 17
Introducing Symmetries to Black Box Meta Reinforcement Learning
Kirsch, L., Flennerhag, S., van Hasselt, H. P., Friesen, A. L., Oh, J., & Chen, Y. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20681 · Cited by 22
Mallozzi, P., Castellano, E., Pelliccione, P., Schneider, G., & Tei, K. (2019) · IEEE/ACM International Workshop on Robotics Software Engineering (RoSE) · doi:10.1109/rose.2019.00011 · Cited by 16
ReinforceWalk: Learning to Walk in Graph with Monte Carlo Tree SearchPreprint
Shen, Y., Chen, J., Huang, P.-S., Guo, Y., & Gao, J. (2018) · arXiv (Cornell University) · Cited by 15
Revisiting Meta-Learning as Supervised LearningPreprint
Chao, W., Ye, H.-J., Zhan, D., Campbell, M., & Weinberger, K. Q. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.00573 · Cited by 18
Mendonca, R., Geng, X., Finn, C., & Levine, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07178 · Cited by 36
MHER: Model-based Hindsight Experience ReplayPreprint
Yang, R., Fang, M., Han, L., Du, Y., Luo, F., & Li, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.00306 · Cited by 16
Improved Q-learning for Energy Management in a Grid-tied PV Microgrid
Arwa, E. O., & Folly, K. A. (2021) · SAIEE Africa Research Journal · doi:10.23919/saiee.2021.9432896 · Cited by 15
Meta-Reinforcement Learning With Dynamic Adaptiveness Distillation
Hu, H., Huang, G. M., Li, X., & Song, S. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3105407 · Cited by 16
Learning to Optimize in Model Predictive Control
Sacks, J., & Boots, B. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812369 · Cited by 15
Zhang, H., Wang, H., & Kan, Z. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3290511 · Cited by 15
Meta-Reinforcement Learning via Language Instructions
Bing, Z.-S., Koch, A., Yao, X.-T., Huang, K., & Knoll, A. C. (2022) · IEEE International Conference on Robotics and Automation · doi:10.1109/icra48891.2023.10160626 · Cited by 29
Grigorescu, S. M. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461174 · Cited by 15
Self-Attentional Credit Assignment for Transfer in Reinforcement Learning
Ferret, J., Marinier, R., Geist, M., & Pietquin, O. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/368 · Cited by 27
Meta-Active Learning in Probabilistically Safe Optimization
Schrum, M., Connolly, M. J., Cole, E. R., Ghetiya, M., Gross, R. E., & Gombolay, M. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3193497 · Cited by 15
Learning Task-Distribution Reward Shaping with Meta-Learning
Zou, H., Ren, T., Yan, D., Su, H., & Zhu, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17337 · Cited by 17
M$^3$RL: Mind-aware Multi-agent Management Reinforcement LearningPreprint
Shu, T., & Tian, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.00147 · Cited by 15
The effects of negative adaptation in Model-Agnostic Meta-LearningPreprint
Deleu, T., & Bengio, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.02159 · Cited by 15
Learning to Learn with Feedback and Local PlasticityPreprint
Lindsey, J. W., & Litwin-Kumar, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.09549 · Cited by 17
Physics-Informed Neural Networks (PINNs) for Parameterized PDEs: A Metalearning ApproachPreprint
Penwarden, M., Zhe, S., Narayan, A., & Kirby, R. M. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3965238 · Cited by 15
Generalized Decision Transformer for Offline Hindsight Information MatchingPreprint
Furuta, H., Matsuo, Y., & Gu, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.10364 · Cited by 66
Learning few-shot imitation as cultural transmission
Bhoopchand, A., Brownfield, B., Collister, A., Dal Lago, A., Edwards, A., Everett, R., Fréchette, A., Oliveira, Y. G., Hughes, E., Mathewson, K. W., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-42875-2 · Cited by 14
In-context Reinforcement Learning with Algorithm Distillation
Laskin, M., Wang, L., Oh, J., Parisotto, E., Spencer, S., Steigerwald, R., Strouse, D., Hansen, S., Filos, A., Brooks, E., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.14215 · Cited by 14
“In-context deep meta-RL methods introduced by Wang et al. (2016) and Duan et al. (2016) are usually trained in the online setting by maximizing multi-episodic value functions with memory-based architectures through environment interactions (Ni et al., 2022).”
MethodGraph Convolutional Network Based Multi-Objective Meta-Deep Q-Learning for Eco-Routing
Ma, X., Xie, Y., & Chigan, C. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3348034 · Cited by 14
Neural mechanisms of relational learning and fast knowledge reassembly in plastic neural networks
Miconi, T., & Kay, K. (2025) · Nature Neuroscience · doi:10.1038/s41593-024-01852-8 · Cited by 19
Learning How to Self-Learn: Enhancing Self-Training Using Neural Reinforcement Learning
Chen, C., Zhang, Y., & Gao, Y. (2018) · International Conference on Asian Language Processing · doi:10.1109/ialp.2018.8629107 · Cited by 17
Automated Curriculum Learning by Rewarding Temporally Rare Events
Justesen, N., & Risi, S. (2018) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2018.8490448 · Cited by 15
Watch, Try, Learn: Meta-Learning from Demonstrations and RewardPreprint
Zhou, A., Jang, E., Kappler, D., Herzog, A., Khansari, M., Wohlhart, P., Bai, Y., Kalakrishnan, M., Levine, S., & Finn, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.03352 · Cited by 31
Meta Reinforcement Learning with Autonomous Inference of Subtask DependenciesPreprint
Sohn, S., Woo, H., Choi, J., & Lee, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.00248 · Cited by 20
ARLPE: A meta reinforcement learning framework for glucose regulation in type 1 diabetics
Yu, X., Guan, Y., Yan, L., Li, S., Fu, X., & Jiang, J. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120156 · Cited by 13
Metzger, A., Kley, T., Rothweiler, A., & Pohl, K. (2023) · Information Systems · doi:10.1016/j.is.2023.102254 · Cited by 13
A Survey of Algorithmic Methods for Competency Self-Assessments in Human-Autonomy Teaming
Conlon, N., Ahmed, N. R., & Szafir, D. J. (2023) · ACM Computing Surveys · doi:10.1145/3616010 · Cited by 13
L2E: Learning to Exploit Your Opponent
Wu, Z., Li, K., Xu, H., Zang, Y., An, B., & Xing, J. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892077 · Cited by 18
Memory, Show the Way: Memory Based Few Shot Word Representation Learning
Sun, J., Wang, S., & Zong, C. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1173 · Cited by 13
Meta-Learning via Weighted Gradient Update
Xu, Z., Cao, L., & Chen, X. (2019) · IEEE Access · doi:10.1109/access.2019.2933988 · Cited by 13
Context-Aware Autonomous Driving Using Meta-Reinforcement Learning
Jaafra, Y., Deruyver, A., Laurent, J. L., & Naceur, M. S. (2019) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2019.00084 · Cited by 14
Kim, T., & Lee, J.-H. (2020) · IEEE Access · doi:10.1109/access.2020.3015245 · Cited by 14
Concurrent Meta Reinforcement LearningPreprint
Parisotto, E., Ghosh, S., Yalamanchi, S., Chinnaobireddy, V., Wu, Y., & Salakhutdinov, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.02710 · Cited by 24
Improving Generalization in Meta Reinforcement Learning using Neural ObjectivesPreprint
Kirsch, L., van Steenkiste, S., & Schmidhuber, J. (2020) · arXiv (Cornell University) · Cited by 14
Provably Convergent Policy Gradient Methods for Model-Agnostic Meta-Reinforcement LearningPreprint
Fallah, A., Mokhtari, A., & Ozdaglar, A. (2020) · arXiv (Cornell University) · Cited by 13
A Meta-Reinforcement Learning Approach to Process Control
McClement, D. G., Lawrence, N. P., Loewen, P. D., Forbes, M. G., Backström, J. U., & Gopaluni, R. B. (2021) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2021.08.321 · Cited by 13
Omni-Training: Bridging Pre-Training and Meta-Training for Few-Shot Learning
Shu, Y., Cao, Z., Gao, J., Wang, J., Yu, P. S., & Long, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3319517 · Cited by 14
Self-citationFew-Shot Quality-Diversity Optimization
Salehi, A., Coninx, A., & Doncieux, S. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3148438 · Cited by 12
Learning to reinforcement learn for Neural Architecture SearchPreprint
Gomez-Robles, J., & Vanschoren, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.03769 · Cited by 12
Watch, Try, Learn: Meta-Learning from Demonstrations and RewardsPreprint
Zhou, A., Jang, E., Kappler, D., Herzog, A., Khansari, M., Wohlhart, P., Bai, Y., Kalakrishnan, M., Levine, S., & Finn, C. (2020) · arXiv (Cornell University) · Cited by 12
Meta-Adversarial Inverse Reinforcement Learning for Decision-making Tasks
Wang, P., Li, H., & Chan, C. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561330 · Cited by 11
Meta-Scheduling Framework With Cooperative Learning Toward Beyond 5G
Min, K., Kim, Y., & Lee, H.-S. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2023.3273698 · Cited by 11
Learning Context-aware Task Reasoning for Efficient Meta Reinforcement Learning
Wang, H., Zhou, J., & He, X. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/rveo2535 · Cited by 16
Visual Navigation with Actor-Critic Deep Reinforcement Learning
Shao, K., Zhao, D., Zhu, Y., & Zhang, Q. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489185 · Cited by 12
Meta-learning autoencoders for few-shot predictionPreprint
Wu, T., Peurifoy, J., Chuang, I. L., & Tegmark, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.09912 · Cited by 11
Meta Learning MPC using Finite-Dimensional Gaussian Process ApproximationsPreprint
Arcari, E., Carron, A., & Zeilinger, M. N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.05984 · Cited by 11
Metadata
- Authors
- Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., & Botvinick, M.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2016
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1611.05763
- OpenAlex
- W2550182557
- Cited by
- 579
- References
- 32
- Citations by year
- 2026: 2 · 2025: 17 · 2024: 47 · 2023: 59 · 2022: 79 · 2021: 104 · 2020: 117 · 2019: 69 · 2018: 61 · 2017: 22 · 2016: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- meta-learning, recurrent neural networks, training data efficiency, learned dynamics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
