Reinforced Imitation: Sample Efficient Deep Reinforcement Learning for Mapless Navigation by Leveraging Prior Demonstrations
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
This letter presents a case study of a learning-based approach for target-driven mapless navigation. The underlying navigation model is an end-to-end neural network, which is trained using a combination of expert demonstrations, imitation learning (IL) and reinforcement learning (RL). While RL and IL suffer from a large sample complexity and the distribution mismatch problem, respectively, we show that leveraging prior expert demonstrations for pretraining can reduce the training time to reach at least the same level of the performance compared to plain RL by a factor of 5. We present a thorough evaluation of different combinations of expert demonstrations, different RL algorithms, and reward functions, both in simulation and on a real robotic platform. Our results show that the final model outperforms both standalone approaches in the amount of successful navigation tasks. In addition, the RL reward function can be significantly simplified when using pretraining, e.g., by using a sparse reward only. The learned navigation policy is able to generalize to unseen and real-world environments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578
Generative adversarial imitation learning
Ho, J., & Ermon, S. (2016) · arXiv · doi:10.48550/arxiv.1606.03476 · Cited by 1,942
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941
Target-driven visual navigation in indoor scenes using deep reinforcement learning
Zhu, Y., Mottaghi, R., Kolve, E., Lim, J. J., Gupta, A., Fei-Fei, L., & Farhadi, A. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989381 · Cited by 1,533
Constrained Policy Optimization
Achiam, J., Held, D., Tamar, A., & Abbeel, P. (2017) · arXiv · doi:10.48550/arxiv.1705.10528 · Cited by 804
Watch this: Scalable cost-function learning for path planning in urban environments
Wulfmeier, M., Wang, D. Z., & Posner, I. (2016) · 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759328 · Cited by 116
Cited by
Deep reinforcement learning based mobile robot navigation: A review
Zhu, K., & Zhang, T. (2021) · Tsinghua Science and Technology · doi:10.26599/tst.2021.9010012 · Cited by 499
Motion planning and control for mobile robot navigation using machine learning: a survey
Xiao, X., Liu, B., Warnell, G., & Stone, P. (2022) · Autonomous Robots · doi:10.1007/s10514-022-10039-8 · Cited by 254
Towards Target-Driven Visual Navigation in Indoor Scenes via Generative Imitation Learning
Wu, Q., Gong, X., Xu, K., Manocha, D., Dong, J., & Wang, J. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3036597 · Cited by 47
Cui, K., Song, J., Zhang, L., Tao, Y., Liu, W., & Shi, D. (2023) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2022.3231239 · Cited by 34
Comparing Task Simplifications to Learn Closed-Loop Object Picking Using Deep Reinforcement Learning
Breyer, M., Furrer, F., Novkovic, T., Siegwart, R., & Nieto, J. (2018) · arXiv · doi:10.48550/arxiv.1803.04996 · Cited by 1
Self-citation
Metadata
- Authors
- Pfeiffer, M., Shukla, S., Turchetta, M., Cadena, C., Krause, A., Siegwart, R., & Nieto, J.
- Venue
- IEEE Robotics and Automation Letters
- Published
- 2018-10
- Volume
- 3
- Issue
- 4
- Pages
- 4423-4430
- ISSN
- 2377-3766 (ISSN-L)
- 2377-3774
- OpenAlex
- W2963129648
- Cited by
- 199
- References
- 7
- FWCI
- 11.03
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.