In-context Reinforcement Learning with Algorithm Distillation
Open accessAbstract
We propose Algorithm Distillation (AD), a method for distilling reinforcement learning (RL) algorithms into neural networks by modeling their training histories with a causal sequence model. Algorithm Distillation treats learning to reinforcement learn as an across-episode sequential prediction problem. A dataset of learning histories is generated by a source RL algorithm, and then a causal transformer is trained by autoregressively predicting actions given their preceding learning histories as context. Unlike sequential policy prediction architectures that distill post-learning or expert sequences, AD is able to improve its policy entirely in-context without updating its network parameters. We demonstrate that AD can reinforcement learn in-context in a variety of environments with sparse rewards, combinatorial task structure, and pixel-based observations, and find that AD learns a more data-efficient RL algorithm than the one that generated the source data.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most recent works on adaptive agents avoid complex environments in favor of more simplistic ones.
“most recent works on adaptive agents (Laskin et al., 2022; Lee et al., 2023; Lu et al., 2023; Norman & Clune, 2023) avoid complex environments in favor of more simplistic ones”
From XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX (2024) · cites this paper as Laskin et al., 2022
Section 4.1 (pp. 5–6) describes evaluations in a 10-arm bandit and small 2D grid environments, supporting the point that this work uses relatively simple settings. But it also evaluates a partially observable 3D visual DMLab Watermaze, so the paper does not support saying it avoids complex environments altogether. It cannot establish what most recent works do. (LLM, conf=0.84)
Experimental — accuracy not yet measuredThe BALROG codebase supports the study of In-Context Reinforcement Learning, where agents learn to improve from mistakes during inference.
“The benchmark codebase also supports the study of In-Context Reinforcement Learning (Lee et al., 2024; Laskin et al., 2022; Lin et al., 2023), where agents learn to improve from mistakes during inference.”
From BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games (2024) · cites this paper as Laskin et al., 2022
In-context reinforcement learning (ICRL) extends the concept of inter-test-time in-context learning by maintaining histories of observations and actions within the agent's context window.
“The field of in-context reinforcement learning (ICRL) (Moeini et al., 2025; Laskin et al., 2022; Lee et al., 2023) extends this concept by maintaining histories of observations and actions within the agent's context window.”
From A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence (2025) · cites this paper as Laskin et al., 2022
Recent works have developed systems to store and retrieve episodes of action to help guide future decision making in a reinforcement learning context.
“Partially instantiating our speculation about the utility of past episodes for planning, recent works have developed systems to store and retrieve episodes of action to help guide future decision making in a reinforcement learning context [43], [44].”
From Episodic Memory in AI Agents Poses Risks that Should be Studied and Mitigated (2025) · cites this paper as [43]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 3× · see citing works
“Transformers have emerged as powerful neural network architectures for sequence modeling (Vaswani et al., 2017).”
BackgroundMethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2× · see citing works
“Here we consider the importance of the Transformer architecture to the success of algorithm distillation (AD) by comparing to AD based off of an LSTM (Hochreiter & Schmidhuber, 1997).”
BackgroundMethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · cited 3× · see citing works
“After pre-training on large offline datasets, large transformers have been shown to generalize to downstream tasks in text completion (Brown et al., 2020), language understanding (Devlin et al., 2018), and image generation (Yu et al., 2022).”
BackgroundRethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138 · cited 2× · see citing works
“For the harder exploration task of Dark Room (Hard), we found that adding label smoothing regularization (Szegedy et al., 2016; Müller et al., 2019) improved the in-context learning ability of AD .”
BackgroundHuman-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 2× · see citing works
“To date, deep RL research has mostly focused on powerful single-task agents Mnih et al. (2015); Schrittwieser et al. (2019); Hessel et al. (2021).”
BackgroundSelf-citationLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 5× · see citing works
“Transformers (Vaswani et al., 2017; Devlin et al., 2018; Brown et al., 2020) utilize self-attention as a core part of the architecture to process sequential data such as text sequences.”
BackgroundMethodInfluentialPlaying Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471 · cited 7× · see citing works
“In the Dark Room and Dark Key-to-Door environments we collect 2000 learning histories with an Asynchronous Advantage Actor-Critic (A3C) (Mnih et al., 2016) with 100 actors, while in DMLab Watermaze we collect 4000 learning histories with a distributed DQN with 16 parallel actors (see Appendix F for asymptotic learning…”
BackgroundMethodInfluentialSelf-citationModel-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512 · cited 1× · see citing works
“Another common approach to online meta-RL includes optimization-based methods that find good network parameter initializations for meta-RL (Hochreiter et al., 2001; Finn et al., 2017; Nichol et al., 2018) and adapt by taking additional gradient steps.”
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 7,936 · see citing works
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · cited 9× · see citing works
“In the Dark Room and Dark Key-to-Door environments we collect 2000 learning histories with an Asynchronous Advantage Actor-Critic (A3C) (Mnih et al., 2016) with 100 actors, while in DMLab Watermaze we collect 4000 learning histories with a distributed DQN with 16 parallel actors (see Appendix F for asymptotic learning…”
BackgroundMethodInfluentialSelf-citationA Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369 · cited 1× · see citing works
“We utilize the GPT (Radford et al., 2018) causal transformer model for sequential action prediction, but AD is compatible with any sequence model including RNNs (Williams & Zipser, 1989).”
MethodSpatial localization does not require the presence of local cues
Morris, R. G. M. (1981) · Learning and Motivation · doi:10.1016/0023-9690(81)90020-5 · Cited by 2,699 · cited 8× · see citing works
“DMLab Watermaze: a partially observable 3D visual DMLab environment based on the classic Morris Watermaze (Morris, 1981).”
MethodInfluentialAsymptotically efficient adaptive allocation rules
Lai, T. L., & Robbins, H. (1985) · Advances in Applied Mathematics · doi:10.1016/0196-8858(85)90002-8 · Cited by 2,406 · see citing works
On First-Order Meta-Learning Algorithms
Nichol, A., Achiam, J., & Schulman, J. (2018) · arXiv · doi:10.48550/arxiv.1803.02999 · Cited by 1,438 · cited 1× · see citing works
“Another common approach to online meta-RL includes optimization-based methods that find good network parameter initializations for meta-RL (Hochreiter et al., 2001; Finn et al., 2017; Nichol et al., 2018) and adapt by taking additional gradient steps.”
Mastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · Nature · doi:10.1038/s41586-020-03051-4 · Cited by 1,214 · cited 2× · see citing works
“3Indeed, current state-of-the-art RL algorithms such as MuZero (Schrittwieser et al., 2019) and Muesli (Hessel et al., 2021) rely on distributed actors.”
BackgroundMethodWhen does label smoothing help?
Müller, R., Kornblith, S., & Hinton, G. (2019) · arXiv · doi:10.48550/arxiv.1906.02629 · Cited by 859 · see citing works
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
Levine, S., Kumar, A., Tucker, G., & Fu, J. (2020) · arXiv · doi:10.48550/arxiv.2005.01643 · Cited by 824 · cited 1× · see citing works
“However, offline RL (Levine et al., 2020) has recently emerged as an alternate and often complementary paradigm for RL where an agent aims to extract return maximizing policies from offline data gathered by another agent.”
BackgroundSimple statistical gradient-following algorithms for connectionist reinforcement learning
Williams, R. J. (1992) · Machine Learning · doi:10.1023/a:1022672621406 · Cited by 817 · cited 2× · see citing works
“RL algorithms broadly fall into two categories: on-policy algorithms (Williams, 1992) where the agent directly maximizes a Monte-Carlo estimate of the total returns or off-policy (Mnih et al.”
BackgroundDecision Transformer: Reinforcement Learning via Sequence ModelingPreprint
Chen, L.-L., Lu, K., Rajeswaran, A., Lee, K., Grover, A., Laskin, M., Abbeel, P., Srinivas, A., & Mordatch, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.01345 · Cited by 677 · cited 11× · see citing works
“Prior related works (Chen et al., 2021; Lee et al., 2022; Janner et al., 2021; Reed et al., 2022) evaluated on Atari, OpenAI gym, and as well as other environments.”
BackgroundMethodInfluentialSelf-citationLearning to reinforcement learn
Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., & Botvinick, M. (2016) · arXiv · doi:10.48550/arxiv.1611.05763 · Cited by 579 · cited 1× · see citing works
“In-context deep meta-RL methods introduced by Wang et al. (2016) and Duan et al. (2016) are usually trained in the online setting by maximizing multi-episodic value functions with memory-based architectures through environment interactions (Ni et al., 2022).”
MethodRL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
Duan, Y., Schulman, J., Chen, X., Bartlett, P. L., Sutskever, I., & Abbeel, P. (2016) · arXiv · doi:10.48550/arxiv.1611.02779 · Cited by 503 · cited 5× · see citing works
“RL2 (Duan et al., 2016): an online meta-RL algorithm where exploration and fast in-context adaptation are learned jointly by maximizing a multi-episodic value function.”
BackgroundMethodInfluentialScaling Autoregressive Models for Content-Rich Text-to-Image Generation
Yu, J., Xu, Y., Koh, J. Y., Luong, T., Baid, G., Wang, Z., Vasudevan, V., Ku, A., Yang, Y., Ayan, B. K., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.10789 · Cited by 371 · cited 2× · see citing works
“After pre-training on large offline datasets, large transformers have been shown to generalize to downstream tasks in text completion (Brown et al., 2020), language understanding (Devlin et al., 2018), and image generation (Yu et al., 2022).”
BackgroundMethodEfficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context VariablesPreprint
Rakelly, K., Zhou, A., Quillen, D., Finn, C., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.08254 · Cited by 371 · cited 1× · see citing works
“Meta-RL through multi-episodic value functions has been done in both on-policy (Duan et al., 2016) and off-policy (Rakelly et al., 2019; Fakoor et al., 2020) settings.”
BackgroundZheng, Q., Zhang, A., & Grover, A. (2022) · International Conference on Machine Learning · doi:10.48550/arxiv.2202.05607 · Cited by 297
Rusu, A. A., Colmenarejo, S. G., Gulcehre, C., Desjardins, G., Kirkpatrick, J., Pascanu, R., Mnih, V., Kavukcuoglu, K., & Hadsell, R. (2015) · arXiv · doi:10.48550/arxiv.1511.06295 · Cited by 270 · see citing works
Self-citationScaling Language Models: Methods, Analysis & Insights from Training Gopher
Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.11446 · Cited by 235 · cited 1× · see citing works
“For example, large language models like GPT-3 (Brown et al., 2020) or Gopher (Rae et al., 2021) can be directed at solving tasks such as text completion, code generation, and text summarization by specifying the task through language as a prompt.”
BackgroundPrompting Decision Transformer for Few-Shot Policy Generalization
Xu, M., Shen, Y., Zhang, S., et al. (2022) · International Conference on Machine Learning · doi:10.48550/arxiv.2206.13499 · Cited by 216
Control of exploitation–exploration meta-parameter in reinforcement learning
Ishii, S., Yoshida, W., & Yoshimoto, J. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00056-4 · Cited by 209 · cited 1× · see citing works
“This general idea of learning the policy improvement process has a long history in reinforcement learning, but has been limited to meta-learning hyper-parameters4 until recently (Ishii et al., 2002).”
BackgroundTowards Learning Universal Hyperparameter Optimizers with Transformers
Chen, Y., Song, X.-Y., Lee, C., Wang, Z., Zhang, Q., Dohan, D., Kawakami, K., Kochanski, G., Doucet, A., Ranzato, M., et al. (2022) · Neural Information Processing Systems · doi:10.48550/arxiv.2205.13320 · Cited by 111 · cited 1×
“While the aforementioned approaches were examples of in-context learning, a recent work (Chen et al., 2022) demonstrated incremental in-context learning for hyperparameter optimization by treating hyperparameter optimization as a sequential prediction problem with a score function.”
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes
Garg, S., Tsipras, D., Liang, P., & Valiant, G. (2022) · arXiv · doi:10.48550/arxiv.2208.01066 · Cited by 110 · see citing works
Data Distributional Properties Drive Emergent In-Context Learning in Transformers
Chan, S. C. Y., Santoro, A., Lampinen, A. K., Wang, J. X., Singh, A., Richemond, P. H., McClelland, J., & Hill, F. (2022) · arXiv · doi:10.48550/arxiv.2205.05055 · Cited by 69 · cited 1× · see citing works
“We use the terms ‘in-weights learning’ and ‘in-context learning’ from prior work on sequence models (Brown et al., 2020; Chan et al., 2022) to distinguish between gradient-based learning with parameter updates and gradient-free learning from context, respectively.”
MethodTraining Agents using Upside-Down Reinforcement LearningPreprint
Srivastava, R. K., Shyam, P., Mutz, F. W., Jaśkowski, W., & Schmidhuber, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02877 · Cited by 52 · cited 1× · see citing works
“Some PD architectures such as DT and MGDT are instantiations of Upside Down RL (UDRL) Schmidhuber (2019); Srivastava et al. (2019) where rather than learning a value function, a policy is conditioned directly on the desired return.”
BackgroundOlson, D. L., & Wu, D. (2019) · ACM SIGSPATIAL International Workshop on Advances in Geographic Information Systems · doi:10.1007/978-0-387-35973-1_90 · Cited by 49
Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to ActionsPreprint
Schmidhuber, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02875 · Cited by 46 · cited 2× · see citing works
“However, AD is not explicitly doing UDRL since it is not conditioned on returns.”
BackgroundMeta-Q-LearningPreprint
Fakoor, R., Chaudhari, P., Soatto, S., & Smola, A. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.00125 · Cited by 42 · cited 1× · see citing works
“Meta-RL through multi-episodic value functions has been done in both on-policy (Duan et al., 2016) and off-policy (Rakelly et al., 2019; Fakoor et al., 2020) settings.”
BackgroundRecurrent Model-Free RL is a Strong Baseline for Many POMDPs
Ni, T., Eysenbach, B., & Salakhutdinov, R. (2021) · arXiv.org · doi:10.48550/arxiv.2110.05038 · Cited by 30 · cited 1×
“(2016) are usually trained in the online setting by maximizing multi-episodic value functions with memory-based architectures through environment interactions (Ni et al., 2022).”
BackgroundOffline Meta-Reinforcement Learning with Advantage WeightingPreprint
Mitchell, E., Rafailov, R., Peng, X. B., Levine, S., & Finn, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.06043 · Cited by 26 · cited 1× · see citing works
“Recent works have proposed learning to reinforcement learn from offline datasets, or offline meta-RL, using Bayesian RL (Dorfman et al., 2021) and optimization-based meta-RL (Mitchell et al., 2021).”
BackgroundOffline Meta-Reinforcement Learning with Online Self-Supervision
Pong, V. H., Nair, A., Smith, L., Huang, C., & Levine, S. (2021) · arXiv · doi:10.48550/arxiv.2107.03974 · Cited by 25 · see citing works
Generalized Decision Transformer for Offline Hindsight Information MatchingPreprint
Furuta, H., Matsuo, Y., & Gu, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.10364 · Cited by 20 · cited 2× · see citing works
“A number of recent works have explored more generalized PD architectures (Furuta et al., 2021), prompt conditioning (Xu et al., 2022), and online gradient-based RL (Zheng et al., 2022).”
BackgroundMuesli: Combining Improvements in Policy OptimizationPreprint
Hessel, M., Danihelka, I., Viola, F., Guez, A., Schmitt, S., Sifre, L., Weber, T., Silver, D., & van Hasselt, H. P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.06159 · Cited by 18 · cited 3× · see citing works
“3Indeed, current state-of-the-art RL algorithms such as MuZero (Schrittwieser et al., 2019) and Muesli (Hessel et al., 2021) rely on distributed actors.”
BackgroundMethodBootstrapped Meta-LearningPreprint
Flennerhag, S., Schroecker, Y., Zahavy, T., van Hasselt, H. P., Silver, D., & Singh, S. P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.04504 · Cited by 15 · see citing works
Self-citation
Flagged first, then by citations
6 of the 47 references couldn’t be matched to a known paper.
Research works printed but not linked: 7
- Charles Beattie, Joel Z. Leibo, Denis Teplyashin, Tom Ward, Marcus Wainwright, Heinrich K¨ uttler, Andrew Lefrancq, Simon Green, V´ ıctor Vald´ es, Amir Sadik, Julian Schrittwieser, Keith Anderson, Sarah York, Max Cant, Adam Cain, Adrian Bolton, Stephen Gaffney, Helen King, Demis Hassabis, Shane Legg, and Stig Petersen. DeepMind Lab. CoRR , abs/1612.03801, 2016. · arXiv
- Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. Generative pretraining from pixels. In International Conference on Machine Learning , pp. 16911703. PMLR, 2020.
- Ron Dorfman, Idan Shenfeld, and Aviv Tamar. Offline Meta Reinforcement Learning-Identifiability Challenges and Effective Data Collection Strategies. Advances in Neural Information Processing Systems , 34:4607-4618, 2021.
- Sepp Hochreiter, A Steven Younger, and Peter R Conwell. Learning to learn using gradient descent. In International conference on artificial neural networks , pp. 87-94. Springer, 2001.
- Shin Ishii, Wako Yoshida, and Junichiro Yoshimoto. Control of exploitation-exploration metaparameter in reinforcement learning. Neural networks , 15(4-6):665-687, 2002.
- Michael Janner, Qiyang Li, and Sergey Levine. Reinforcement Learning as One Big Sequence Modeling Problem. arXiv preprint arXiv:2106.02039 , 2021. · arXiv
- Eric Mitchell, Rafael Rafailov, Xue Bin Peng, Sergey Levine, and Chelsea Finn. Offline metareinforcement learning with advantage weighting. In International Conference on Machine Learning , pp. 7780-7791. PMLR, 2021.
Cited by
Sorted by citations
A review of large language models and autonomous agents in chemistry
Ramos, M. C., Collison, C. J., & White, A. D. (2025) · Chemical Science · doi:10.1039/d4sc03921a · Cited by 274 · see citing works
Images Speak in Images: A Generalist Painter for In-Context Visual Learning
Wang, X., Wang, W., Cao, Y., Shen, C., & Huang, T. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00660 · Cited by 201 · see citing works
On Transforming Reinforcement Learning With Transformers: The Development Trajectory
Hu, S., Shen, L., Zhang, Y., Chen, Y., & Tao, D. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3408271 · Cited by 81 · see citing works
“Furthermore, the literature [202] suggests that transformers have the capability to emulate all RL algorithms given a sufficiently expansive and diverse training dataset.”
BackgroundAIGC-empowered smart manufacturing: Prospects and challenges
Leng, J., Zheng, K., Li, R. C., Chen, C., Wang, B., Liu, Q., Chen, X., & Shen, W. (2025) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2025.103076 · Cited by 70 · see citing works
Causal reasoning: Charting a revolutionary course for next-generation AI-native wireless networks
Thomas, C. K., Chaccour, C., Saad, W., Debbah, M., & Hong, C. S. (2024) · IEEE Vehicular Technology Magazine · doi:10.1109/mvt.2024.3359357 · Cited by 46 · see citing works
“Next, by algorithm distillation (AD) [29], we distill the behavior of the source algorithm into a sequence model that maps long histories to probabilities over actions.”
MethodHuman-Timescale Adaptation in an Open-Ended Task Space
Adaptive Agent Team, Bauer, J., Baumli, K., Baveja, S., Behbahani, F., Bhoopchand, A., Bradley-Schmieg, N., Chang, M., Clay, N., Collister, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2301.07608 · Cited by 19 · see citing works
“Transformers have also been shown to be effective for meta-RL on simple domains (Melo, 2022) and for learning RL algorithms (Laskin et al., 2022) from offline data.”
BackgroundInfluentialThe Solar AI Nexus: Reinforcement Learning Shaping the Future of Energy Management
Hanif, M. F., Ahmad, R., Farooq, A. B., et al. (2025) · Wiley Interdisciplinary Reviews Energy and Environment · doi:10.1002/wene.70012 · Cited by 15 · see citing works
Contextual reinforcement learning for supply chain management
Batsis, A., & Samothrakis, S. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123541 · Cited by 13 · see citing works
Attention-Based Policy Distillation for UAV Simultaneous Target Tracking and Obstacle Avoidance
Xu, L., Wang, T., Wang, J., et al. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3342174 · Cited by 10 · see citing works
“[38] introduces an approach to distill DRL algorithms into NN by representing the progress of training through the utilization of a causal sequential framework.”
MethodZheng, C., Xie, F., Ji, T., Zhou, H., & Zheng, Y. (2024) · Journal of Fluid Mechanics · doi:10.1017/jfm.2024.1133 · Cited by 10 · see citing works
Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identifications
Zhang, X., Liu, S., Huang, P., Han, W. J., Lyu, Y., Xu, M., & Zhao, D. X. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3540391 · Cited by 7 · see citing works
“In-context learning has gained traction as a method for adjusting model behavior without gradient updates, widely used in Natural Language Processing (NLP) [14] and recently applied in robotics to improve generalization [15], [16], [17].”
InfluentialCan Transformers Learn Optimal Filtering for Unknown Systems?
Du, Z., Balim, H., Oymak, S., & Özay, N. (2023) · IEEE Control Systems Letters · doi:10.1109/lcsys.2023.3335318 · Cited by 7 · see citing works
Episodic Memory in AI Agents Poses Risks that Should be Studied and Mitigated
DeChant, C. (2025) · 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml64287.2025.00024 · Cited by 5 · see citing works
“Partially instantiating our speculation about the utility of past episodes for planning, recent works have developed systems to store and retrieve episodes of action to help guide future decision making in a reinforcement learning context [43], [44].”
XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX
Nikulin, A., Kurenkov, V., Zisman, I., Agarkov, A., Sinii, V., & Kolesnikov, S. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-1390 · Cited by 3 · see citing works
“…and this also has been pointed out by Wang et al. (2021), that such demanding requirements are the reason why most recent works on adaptive agents (Laskin et al., 2022; Lee et al., 2023; Lu et al., 2023; Norman & Clune, 2023) avoid complex environments in favor of more simplistic ones (e.g.,…”
BackgroundGao, H.-A., Geng, J., Hua, W., Hu, M., Juan, X., Liu, H., Liu, S., Qiu, J., Qi, X., Wu, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2507.21046 · Cited by 3 · see citing works
BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games
Paglieri, D., Cupiał, B., Coward, S., Piterbarg, U., Wolczyk, M., Khan, A., Pignatelli, E., Kuciński, Ł., Pinto, L., Fergus, R., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.13543 · Cited by 2 · see citing works
“The benchmark codebase also supports the study of In-Context Reinforcement Learning (Lee et al., 2024; Laskin et al., 2022; Lin et al., 2023), where agents learn to improve from mistakes during inference.”
DuoJoule: Accurate On-Device Deep Reinforcement Learning for Energy and Timeliness
Shirvani, S., Samanta, A., Li, Z., et al. (2024) · Proceedings/Proceedings - Real-Time Systems Symposium · doi:10.1109/rtss62706.2024.00019 · Cited by 2 · see citing works
MesaNet: Sequence Modeling by Locally Optimal Test-Time Training
von Oswald, J., Scherrer, N., Kobayashi, S., Versari, L., Yang, S., Mittal, S., Schlegel, M., Maile, K., Schimpf, Y., Sieberling, O., et al. (2025) · arXiv
“Therefore the Mesa layer might excel in domains which require some form of in-context (control or reinforcement learning) algorithm distillation [61].”
Artificial Generational Intelligence: Cultural Accumulation in Reinforcement Learning
Cook, J., Lu, C., Hughes, E., Leibo, J. Z., & Foerster, J. (2024) · arXiv · doi:10.48550/arxiv.2406.00392 · Cited by 6 · see citing works
“This setting has also been called in-context RL [Laskin et al., 2022].”
MethodUnified Image-to-Image Generation for Diverse Medical Vision TasksPreprint
Veluru, A. (2025) · doi:10.31224/5017 · Cited by 2 · see citing works
- Authors
- Laskin, M., Wang, L., Oh, J., Parisotto, E., Spencer, S., Steigerwald, R., Strouse, D., Hansen, S., Filos, A., Brooks, E., Gazeau, M., Sahni, H., Singh, S., & Mnih, V.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2210.14215
- OpenAlex
- W4307413493
- Cited by
- 14
- References
- 47
- Citations by year
- 2025: 5 · 2024: 6 · 2023: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Keywords
- in-context learning, reinforcement learning, causal transformer, policy prediction, sparse rewards, data efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
