Meta reinforcement learning for sim-to-real domain adaptation
Open accessAbstract
Modern reinforcement learning methods suffer from low sample efficiency and unsafe exploration, making it infeasible to train robotic policies entirely on real hardware. In this work, we propose to address the problem of sim-to-real domain transfer by using meta learning to train a policy that can adapt to a variety of dynamic conditions, and using a task-specific trajectory generation model to provide an action space that facilitates quick exploration. We evaluate the method by performing domain adaptation in simulation and analyzing the structure of the latent space during adaptation. We then deploy this policy on a KUKA LBR 4+ robot and evaluate its performance on a task of hitting a hockey puck to a target. Our method shows more consistent and stable domain adaptation than the baseline, resulting in better overall performance.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In robotic vision tasks, training data from simulators always have different textures, lighting, and camera positions from realistic environments.
“In robotic vision tasks including object localization [53], object detection [54], pose estimation [55], and semantic segmentation [56], the training data from simulator always have different textures, lighting, and camera positions from the realistic environments.”
From Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (2020) · cites this paper as [54]
Cites
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · see citing works
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471 · see citing works
Domain randomization for transferring deep neural networks from simulation to the real world
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., & Abbeel, P. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202133 · Cited by 3,124 · cited 1× · see citing works
“The idea can be applied both to perception [8], [22]– [24] and to the dynamics of the system [25], [26].”
BackgroundLearning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement Learning
Nagabandi, A., Clavera, I., Liu, S., Fearing, R. S., Abbeel, P., Levine, S., & Finn, C. (2018) · arXiv · doi:10.48550/arxiv.1803.11347 · Cited by 370 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey
Zhao, W., Queralta, J. P., & Westerlund, T. (2020) · 2020 IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci47803.2020.9308468 · Cited by 941 · see citing works
Zhao, W., Queralta, J. P., Qingqing, L., & Westerlund, T. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.10.043 · Cited by 9 · see citing works
- Authors
- Arndt, K., Hazara, M., Ghadirzadeh, A., & Kyrki, V.
- Affiliations
- Aalto University (FI)
- Venue
- 2020 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2020-05
- Pages
- 2725-2731
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2976047560
- Cited by
- 97
- References
- 4
- Citations by year
- 2026: 7 · 2025: 23 · 2024: 11 · 2023: 20 · 2022: 13 · 2021: 11 · 2020: 11 · 2012: 1
- FWCI
- 5.25
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Keywords
- meta-reinforcement learning, sim-to-real transfer, domain adaptation, trajectory generation, latent space structure, sample efficiency, safe exploration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
