Deep reinforcement learning based mobile robot navigation: A review
Open accessThe DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
Abstract
Navigation is a fundamental problem of mobile robots, for which Deep Reinforcement Learning (DRL) has received significant attention because of its strong representation and experience learning abilities. There is a growing trend of applying DRL to mobile robot navigation. In this paper, we review DRL methods and DRL-based navigation frameworks. Then we systematically compare and analyze the relationship and differences between four typical application scenarios: local obstacle avoidance, indoor navigation, multi-robot navigation, and social navigation. Next, we describe the development of DRL-based navigation. Last, we discuss the challenges and some possible solutions regarding DRL-based navigation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · cited 1×
“Schulman et al. [19] proposed the PPO algorithm, which can perform multiple epochs of minibatch updates, thereby improving the sample utilization efficiency.”
MethodPlaying Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · cited 1×
“The A3C algorithm proposed by Mnih et al. [18] is a representative Actor-Critic (AC) method.”
MethodORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2463671 · Cited by 6,778 · cited 1×
“Classical visual SLAM methods, such as LSD-SLAM [2] and ORB-SLAM [3] , face two main challenges: (1) designing effective image features to express image information and (2) possible failure of the algorithm in cases of object movement, camera parameter change, illumination change, and single…”
MethodWatkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297
LSD-SLAM: Large-scale direct monocular SLAM
Engel, J., Schöps, T., & Cremers, D. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_54 · Cited by 3,851 · cited 2×
“Classical visual SLAM methods, such as LSD-SLAM [2] and ORB-SLAM [3] , face two main challenges: (1) designing effective image features to express image information and (2) possible failure of the algorithm in cases of object movement, camera parameter change, illumination change, and single environments that lack tex…”
MethodThe dynamic window approach to collision avoidance
Fox, D., Burgard, W., & Thrun, S. (1997) · IEEE Robotics & Automation Magazine · doi:10.1109/100.580977 · Cited by 3,771
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
ORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · Zaguan (Universidad de Zaragoza) · Cited by 3,153
Improved Techniques for Grid Mapping With Rao-Blackwellized Particle Filters
Grisetti, G., Stachniss, C., & Burgard, W. (2007) · IEEE Transactions on Robotics · doi:10.1109/tro.2006.889486 · Cited by 2,439
Target-driven visual navigation in indoor scenes using deep reinforcement learning
Zhu, Y., Mottaghi, R., Kolve, E., Lim, J. J., Gupta, A., Fei-Fei, L., & Farhadi, A. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989381 · Cited by 1,533
Nguyen, T. T., Nguyen, N. D., & Nahavandi, S. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2977374 · Cited by 1,306
Semantic Scene Completion from a Single Depth Image
Song, S., Yu, F., Zeng, A., Chang, A. X., Savva, M., & Funkhouser, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.28 · Cited by 1,209
A flexible and scalable SLAM system with full 3D motion estimation
Kohlbrecher, S., von Stryk, O., Meyer, J., & Klingauf, U. (2011) · 2011 IEEE International Symposium on Safety, Security, and Rescue Robotics · doi:10.1109/ssrr.2011.6106777 · Cited by 1,086
Tai, L., Paolo, G., & Liu, M. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202134 · Cited by 834
Sampling-Based Robot Motion Planning: A Review
Elbanhawi, M., & Simic, M. (2014) · IEEE Access · doi:10.1109/access.2014.2302442 · Cited by 786
Socially aware motion planning with deep reinforcement learning
Chen, Y. F., Everett, M., Liu, M., & How, J. P. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202312 · Cited by 719
Vector-based navigation using grid-like representations in artificial agents
Banino, A., Barry, C., Uria, B., Blundell, C., Lillicrap, T., Mirowski, P., Pritzel, A., Chadwick, M. J., Degris, T., Modayil, J., et al. (2018) · Nature · doi:10.1038/s41586-018-0102-6 · Cited by 677
Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning
Chen, Y. F., Liu, M., Everett, M., & How, J. P. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989037 · Cited by 645
Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning
Everett, M., Chen, Y. F., & How, J. P. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593871 · Cited by 614
Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning
Long, P., Fanl, T., Liao, X., Liu, W., Zhang, H., & Pan, J. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8461113 · Cited by 538
Wang, C., Wang, J., Shen, Y., & Zhang, X. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2890773 · Cited by 443
PRIMAL: Pathfinding via Reinforcement and Imitation Multi-Agent Learning
Sartoretti, G., Kerr, J., Shi, Y., Wagner, G., Kumar, T. K. S., Koenig, S., & Choset, H. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2903261 · Cited by 417
Fan, T., Long, P., Liu, W., & Pan, J. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364920916531 · Cited by 332
Faust, A., Oslund, K., Ramirez, O., Francis, A., Tapia, L., Fiser, M., & Davidson, J. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8461096 · Cited by 327
Deep reinforcement learning with successor features for navigation across similar environments
Zhang, J., Springenberg, J. T., Boedecker, J., & Burgard, W. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206049 · Cited by 278
Collision avoidance for an unmanned surface vehicle using deep reinforcement learning
Woo, J., & Kim, N. (2020) · Ocean Engineering · doi:10.1016/j.oceaneng.2020.107001 · Cited by 264
Building Generalizable Agents with a Realistic and Rich 3D Environment
Wu, Y., Wu, Y., Gkioxari, G., & Tian, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.02209 · Cited by 226
Learning Navigation Behaviors End-to-End With AutoRL
Chiang, H.-T. L., Faust, A., Fiser, M., & Francis, A. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2899918 · Cited by 225
Deep-Reinforcement-Learning-Based Autonomous UAV Navigation With Sparse Rewards
Wang, C., Wang, J., Wang, J., & Zhang, X. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2973193 · Cited by 217
Uncertainty-Aware Reinforcement Learning for Collision Avoidance
Kahn, G., Villaflor, A., Pong, V. H., Abbeel, P., & Levine, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.01182 · Cited by 209
Pfeiffer, M., Shukla, S., Turchetta, M., Cadena, C., Krause, A., Siegwart, R., & Nieto, J. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2869644 · Cited by 199
Neural networks based reinforcement learning for mobile robots obstacle avoidance
Duguleana, M., & Mogan, G. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.06.021 · Cited by 197
Neural Map: Structured Memory for Deep Reinforcement Learning
Parisotto, E., & Salakhutdinov, R. (2017) · arXiv · doi:10.48550/arxiv.1702.08360 · Cited by 178
Deep reinforcement learning for drone navigation using sensor data
Hodge, V. J., Hawkins, R., & Alexander, R. (2021) · Neural Computing and Applications · doi:10.1007/s00521-020-05097-x · Cited by 178
End-to-End Navigation Strategy With Deep Reinforcement Learning for Mobile Robots
Shi, H., Shi, L., Xu, M., & Hwang, K.-S. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2936167 · Cited by 176
Wu, X., Chen, H., Chen, C., Zhong, M., Xie, S., Guo, Y., & Fujita, H. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2019.105201 · Cited by 167
Dynamic Path Planning of Unknown Environment Based on Deep Reinforcement Learning
Lei, X., Zhang, Z., & Dong, P. (2018) · Journal of Robotics · doi:10.1155/2018/5781591 · Cited by 164
Wang, D., Fan, T., Han, T., & Pan, J. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2974648 · Cited by 155
Robot Navigation in Crowds by Graph Convolutional Networks With Attention Learned From Human Gaze
Chen, Y., Liu, C., Shi, B. E., & Liu, M. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2972868 · Cited by 151
Visual Navigation for Biped Humanoid Robots Using Deep Reinforcement Learning
Lobos-Tsunekawa, K., Leiva, F., & Ruíz-del-Solar, J. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2851148 · Cited by 111
Towards Generalization in Target-Driven Visual Navigation by Using Deep Reinforcement Learning
Devo, A., Mezzetti, G., Costante, G., Fravolini, M. L., & Valigi, P. (2020) · IEEE Transactions on Robotics · doi:10.1109/tro.2020.2994002 · Cited by 107
Wang, C., Wang, J., Zhang, X., & Zhang, X. (2017) · 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP) · doi:10.1109/globalsip.2017.8309082 · Cited by 106
Learning to Navigate Through Complex Dynamic Environment With Modular Deep Reinforcement Learning
Wang, Y., He, H., & Sun, C. (2018) · IEEE Transactions on Games · doi:10.1109/tg.2018.2849942 · Cited by 106
A survey on visual navigation for artificial agents with deep reinforcement learning
Zeng, F., Wang, C., & Ge, S. S. (2020) · IEEE Access · doi:10.1109/access.2020.3011438 · Cited by 98
Choi, J., Park, K.-S., Kim, M., & Seok, S. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793979 · Cited by 91
Anchor self-localization algorithm based on UWB ranging and inertial measurements
Shi, Q., Zhao, S., Cui, X., Lu, M., & Jia, M. (2019) · Tsinghua Science & Technology · doi:10.26599/tst.2018.9010102 · Cited by 88
Survey of pedestrian action recognition techniques for autonomous driving
Chen, L., Ma, N., Wang, P. S. P., Li, J., Wang, P., Pang, G., & Shi, X. (2020) · Tsinghua Science & Technology · doi:10.26599/tst.2019.9010018 · Cited by 79
Navigation in Unknown Dynamic Environments Based on Deep Reinforcement Learning
Zeng, J., Ju, R., Qin, L., Hu, Y., Yin, Q., & Hu, C. (2019) · Sensors · doi:10.3390/s19183837 · Cited by 75
Mapping, localization and motion planning in mobile multi-robotic systems
Rone, W. S., & Ben‐Tzvi, P. (2012) · Robotica · doi:10.1017/s0263574712000021 · Cited by 68
Fan, T., Long, P., Liu, W., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.03841 · Cited by 68
Multi-robot Target Encirclement Control with Collision Avoidance via Deep Reinforcement Learning
Ma, J., Lu, H., Xiao, J., Zeng, Z., & Zheng, Z. (2019) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-019-01106-x · Cited by 66
Real-Time Object Navigation With Deep Neural Networks and Hierarchical Reinforcement Learning
Staroverov, A., Yudin, D. A., Belkin, I., Adeshkin, V., Solomentsev, Y., & Panov, A. I. (2020) · IEEE Access · doi:10.1109/access.2020.3034524 · Cited by 62
Crowd Navigation in an Unknown and Dynamic Environment Based on Deep Reinforcement Learning
Sun, L., Zhai, J., & Qin, W. (2019) · IEEE Access · doi:10.1109/access.2019.2933492 · Cited by 61
Deep imitation learning for 3D navigation tasks
Hussein, A. M., Elyan, E., Gaber, M. M., & Jayne, C. (2017) · Neural Computing and Applications · doi:10.1007/s00521-017-3241-z · Cited by 58
Laser-Based Reactive Navigation for Multirotor Aerial Robots using Deep Reinforcement Learning
Sampedro, C., Bavle, H., Rodríguez-Ramos, A., de la Puente, P., & Campoy, P. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593706 · Cited by 52
Chen, G., Yao, S., Ma, J., Pan, L., Chen, Y., Xu, P., Ji, J., & Chen, X. (2020) · Sensors · doi:10.3390/s20174836 · Cited by 47
Teaching a Machine to Read Maps With Deep Reinforcement Learning
Brunner, G., Richter, O., Wang, Y., & Wattenhofer, R. P. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11645 · Cited by 45
Lin, J., Yang, X., Zheng, P., & Cheng, H. (2019) · IEEE International Conference on Mechatronics and Automation (ICMA) · doi:10.1109/icma.2019.8816208 · Cited by 45
Deep Reinforcement Learning for Instruction Following Visual Navigation in 3D Maze-Like Environments
Devo, A., Costante, G., & Valigi, P. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2965857 · Cited by 43
Robust RL-Based Map-Less Local Planning: Using 2D Point Clouds as Observations
Leiva, F., & Ruíz-del-Solar, J. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3010732 · Cited by 43
Hierarchical Reinforcement Learning Framework Towards Multi-Agent Navigation
Ding, W., Li, S., Qian, H., & Chen, Y. (2018) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio.2018.8664803 · Cited by 41
Lu, Y., Chen, Y., Zhao, D., & Li, D. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.07.091 · Cited by 39
Mapless Collaborative Navigation for a Multi-Robot System Based on the Deep Reinforcement Learning
Chen, W., Zhou, S., Pan, Z., Zheng, H., & Liu, Y. (2019) · Applied Sciences · doi:10.3390/app9204198 · Cited by 37
Yokoyama, K., & Morioka, K. (2020) · IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/sii46433.2020.9025987 · Cited by 35
Composite Reinforcement Learning for Social Robot Navigation
Ciou, P.-H., Hsiao, Y., Wu, Z.-Z., Tseng, S.-H., & Fu, L. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593410 · Cited by 25
Distributed Deep Reinforcement Learning based Indoor Visual Navigation
Hsu, S.-H., Chan, S.-H., Wu, P.-T., Xiao, K., & Fu, L. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594352 · Cited by 25
Kato, Y., & Morioka, K. (2019) · IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/sii.2019.8700426 · Cited by 22
Learning Reward Function with Matching Network for Mapless Navigation
Zhang, Q., Zhu, M., Zou, L., Li, M., & Zhang, Y. (2020) · Sensors · doi:10.3390/s20133664 · Cited by 18
Obtaining fault tolerance avoidance behavior using deep reinforcement learning
Aznar, F., López, M. P., & Aldeguer, R. R. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.11.090 · Cited by 15
A3C Based Motion Learning for an Autonomous Mobile Robot in Crowds
Sasaki, Y., Matsuo, S., Kanezaki, A., et al. (2019) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2019.8914201 · Cited by 15
Visual Navigation with Asynchronous Proximal Policy Optimization in Artificial Agents
Zeng, F., & Wang, C. (2020) · Journal of Robotics · doi:10.1155/2020/8702962 · Cited by 14
Cited by
Reinforcement learning algorithms: A brief survey
Shakya, A. K., Pillai, G., & Chakrabarty, S. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120495 · Cited by 656
A Survey on Active Simultaneous Localization and Mapping: State of the Art and New Frontiers
Placed, J. A., Strader, J., Carrillo, H., Atanasov, N., Indelman, V., Carlone, L., & Castellanos, J. A. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3248510 · Cited by 332
Deep reinforcement learning in smart manufacturing: A review and prospects
Li, C., Zheng, P., Yin, Y., Wang, B., & Wang, L. (2023) · CIRP Journal of Manufacturing Science and Technology · doi:10.1016/j.cirpj.2022.11.003 · Cited by 327
A Hybrid Pure Pursuit–PID and TD3 Control Framework for Differential-Drive Mobile Robots
Demircioglu, U., Bakir, H., Almarri, B., & Abdul Hafez, A. H. (2026) · IEEE Access · doi:10.1109/access.2026.3691502
“A key advantage of RL lies in its ability to learn control policies directly through interaction with the environment, without requiring explicit mathematical models of system dynamics [25], [26].”
Metadata
- Authors
- Zhu, K., & Zhang, T.
- Venue
- Tsinghua Science and Technology
- Publisher
- Tsinghua University Press
- Published
- 2021-10
- Volume
- 26
- Issue
- 5
- Pages
- 674-691
- ISSN
- 1007-0214 (ISSN-L)
- OpenAlex
- W3158253560
- Cited by
- 499
- References
- 74
- FWCI
- 24.26
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.