Continual Reinforcement Learning deployed in Real-life using Policy Distillation and Sim2Real Transfer
Open accessAbstract
We focus on the problem of teaching a robot to solve tasks presented sequentially, i.e., in a continual learning scenario. The robot should be able to solve all tasks it has encountered, without forgetting past tasks. We provide preliminary work on applying Reinforcement Learning to such setting, on 2D navigation tasks for a 3 wheel omni-directional robot. Our approach takes advantage of state representation learning and policy distillation. Policies are trained using learned features as input, rather than raw observations, allowing better sample efficiency. Policy distillation is used to combine multiple policies into a single one that solves all encountered tasks.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Some approaches to bridge the gap rely on meta learning or continual learning.
“Some of the approaches to bridge the gap in this direction rely on meta learning [11] or continual learning [12], among others.”
From Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (2020) · cites this paper as [12]
DisCoRL has been successfully applied to multiple tasks learned by different teachers, with their knowledge being distilled to a single student network.
“DisCoRL has been succesfully applied to multiple tasks learned by different teachers, with their knowledge being distilled to a single student network.”
From Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (2020) · cites this paper as [12]
In [12], the authors presented DisCoRL, a modular, effective and scalable pipeline for continual DRL.
“In [12], the authors presented DisCoRL, a modular, effective and scalable pipeline for continual DRL.”
From Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (2020) · cites this paper as [12]
Curriculum learning, incremental environment complexity, and continual learning and policy distillation for multiple tasks are methods of particular interest for achieving successful sim-to-real transfer in navigation.
“Of particular interest due to their potential and novelty are the following methods: curriculum learning [37], incremental environment complexity [39], and continual learning and policy distillation for multiple tasks [12].”
From Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (2020) · cites this paper as [12]
Cited by
Sorted by citations
Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey
Zhao, W., Queralta, J. P., & Westerlund, T. (2020) · 2020 IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci47803.2020.9308468 · Cited by 941 · see citing works
Procedural Content Generation: Better Benchmarks for Transfer Reinforcement Learning
Muller-Brockhausen, M., Preuss, M., & Plaat, A. (2021) · 2021 IEEE Conference on Games (CoG) · doi:10.1109/cog52621.2021.9619000 · Cited by 11 · see citing works
- Authors
- Traoré, R., Caselles-Dupré, H., Lesort, T., Sun, T., Díaz-Rodríguez, N., & Filliat, D.
- Affiliations
- Institut national de recherche en sciences et technologies du numérique (FR)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- arXiv
- 1906.04452
- OpenAlex
- W4297740831
- Cited by
- 24
- References
- 0
- Citations by year
- 2024: 3 · 2023: 6 · 2022: 6 · 2021: 3 · 2020: 3 · 2019: 2 · 2012: 1
- FWCI
- 1.31
- Type
- conference paper
- Language
- en
- Open access
- green
- doi.org
- Locations
- HAL (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Keywords
- continual reinforcement learning, policy distillation, sim-to-real transfer, robot navigation, omni-directional robot, state representation learning, forgetting, sample efficiency, multi-task learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
