Sim-to-Real Transfer with Incremental Environment Complexity for Reinforcement Learning of Depth-based Robot Navigation
Open accessAbstract
Transferring learning-based models to the real world remains one of the hardest problems in model-free control theory. Due to the cost of data collection on a real robot and the limited sample efficiency of Deep Reinforcement Learning algorithms, models are usually trained in a simulator which theoretically provides an infinite amount of data. Despite offering unbounded trial and error runs, the reality gap between simulation and the physical world brings little guarantee about the policy behavior in real operation. Depending on the problem, expensive real fine-tuning and/or a complex domain randomization strategy may be required to produce a relevant policy. In this paper, a Soft-Actor Critic (SAC) training strategy using incremental environment complexity is proposed to drastically reduce the need for additional training in the real world. The application addressed is depth-based mapless navigation, where a mobile robot should reach a given waypoint in a cluttered environment with no prior mapping information. Experimental results in simulated and real environments are presented to assess quantitatively the efficiency of the proposed approach, which demonstrated a success rate twice higher than a naive strategy.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 38 · Cited by 38
Research works printed but not linked: 6
- Badnava, B. and Mozayani, N. (2019). A new potentialbased reward shaping for reinforcement learning agent. ArXiv , abs/1902.06239. · arXiv
- Juozapaitis, Z., Koul, A., Fern, A., Erwig, M., and DoshiVelez, F. (2019). Explainable reinforcement learning via reward decomposition. In Proceedings of the IJCAI 2019 Workshop on Explainable Artificial Intelligence , pages 47-53.
- Kulhanek, J., Derner, E., de Bruin, T., and Babuska, R. (2019). Vision-based navigation using deep reinforcement learning. In 9th European Conference on Mobile Robots .
- Ruiz, N., Schulter, S., and Chandraker, M. (2019). Learning to simulate. ICLR .
- Sampedro, C., Rodriguez-Ramos, A., Bavle, H., Carrio, A., de la Puente, P., and Campoy, P. (2019). A fullyautonomous aerial robot for search and rescue applications in indoor environments using learning-based techniques. Journal of Intelligent & Robotic Systems , 95(2):601-627.
- Wurm, K. M., Hornung, A., Bennewitz, M., Stachniss, C., and Burgard, W. (2010). Octomap: A probabilistic, flexible, and compact 3d map representation for robotic systems. In ICRA 2010 workshop on best practice in 3D perception and modeling for mobile manipulation , volume 2.
- Authors
- Chaffre, T., Moras, J., Chan-Hon-Tong, A., & Marzat, J.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- Université Paris-Saclay (FR)
- Venue
- Proceedings of the 17th International Conference on Informatics in Control, Automation and Robotics
- Published
- 2020
- Pages
- 314-323
- OpenAlex
- W3023661178
- Cited by
- 38
- References
- 38
- Citations by year
- 2026: 2 · 2025: 8 · 2024: 5 · 2023: 7 · 2022: 11 · 2021: 3 · 2020: 2
- FWCI
- 2.19
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- International Conference on Informatics in Control, Automation and Robotics (ICINCO) (publishedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion, other-oa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Aerospace Engineering › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- sim-to-real transfer, mapless navigation, mobile robot navigation, soft actor-critic, domain randomization, cluttered environments
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
