Rule-Based Reinforcement Learning for Efficient Robot Navigation With Space Reduction
Open accessAbstract
For real-world deployments, it is critical to allow robots to navigate in complex environments autonomously. Traditional methods usually maintain an internal map of the environment, and then design several simple rules, in conjunction with a localization and planning approach, to navigate through the internal map. These approaches often involve a variety of assumptions and prior knowledge. In contrast, recent reinforcement learning (RL) methods can provide a model-free, self-learning mechanism as the robot interacts with an initially unknown environment, but are expensive to deploy in real-world scenarios due to inefficient exploration. In this article, we focus on efficient navigation with the RL technique and combine the advantages of these two kinds of methods into a rule-based RL (RuRL) algorithm for reducing the sample complexity and cost of time. First, we use the rule of wall-following to generate a closed-loop trajectory. Second, we employ a reduction rule to shrink the trajectory, which in turn effectively reduces the redundant exploration space. Besides, we give the detailed theoretical guarantee that the optimal navigation path is still in the reduced space. Third, in the reduced space, we utilize the Pledge rule to guide the exploration strategy for accelerating the RL process at the early stage. Experiments conducted on real robot navigation problems in hex-grid environments demonstrate that RuRL can achieve improved navigation performance.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Microstructure of a spatial map in the entorhinal cortex
Hafting, T., Fyhn, M., Molden, S., Moser, M.-B., & Moser, E. I. (2005) · Nature · doi:10.1038/nature03721 · Cited by 4,068 · see citing works
Real-time loop closure in 2D LIDAR SLAM
Hess, W., Kohler, D., Rapp, H., & Andor-Ardó, D. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487258 · Cited by 1,945 · see citing works
Path Planning with Modified a Star Algorithm for a Mobile Robot
Duchoň, F., Babinec, A., Kajan, M., et al. (2014) · Procedia Engineering · doi:10.1016/j.proeng.2014.12.098 · Cited by 877 · see citing works
(2016) · neural information processing systems · Cited by 735 · see citing works
Rectangular and hexagonal grids used for observation, experiment and simulation in ecology
Birch, C., Oom, S. P., & Beecham, J. A. (2007) · Ecological Modelling · doi:10.1016/j.ecolmodel.2007.03.041 · Cited by 667 · see citing works
Abelson, H., & diSessa, A. A. (1981) · The MIT Press eBooks · doi:10.7551/mitpress/6933.001.0001 · Cited by 434 · see citing works
Rothman, D. H., & Zaleski, S. (1997) · Cambridge University Press eBooks · doi:10.1017/cbo9780511524714 · Cited by 402 · see citing works
Count-Based Exploration with Neural Density Models
Ostrovski, G., Bellemare, M. G., Oord, A. V. D., & Munos, R. (2017) · arXiv · doi:10.48550/arxiv.1703.01310 · Cited by 377 · see citing works
Is Q-learning Provably Efficient?Preprint
Jin, C., Allen-Zhu, Z., Bubeck, S., & Jordan, M. I. (2018) · arXiv (Cornell University) · Cited by 368 · see citing works
Faust, A., Oslund, K., Ramírez, O. E. M., Francis, A. G., Jr., Tapia, L., Fišer, M., & Davidson, J. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461096 · Cited by 335 · see citing works
Lattice-gas cellular automata: Simple models of complex hydrodynamics
Rothman, D. H., Zaleski, S., & Powell, A. (1998) · Computers in Physics · doi:10.1063/1.168762 · Cited by 332 · see citing works
Path Smoothing Techniques in Robot Navigation: State-of-the-Art, Current and Future Challenges
Ravankar, A. A., Ravankar, A. A., Kobayashi, Y., HOSHINO, Y., & Peng, C.-C. (2018) · Sensors · doi:10.3390/s18093170 · Cited by 320 · see citing works
Learning Classifier Systems: A Complete Introduction, Review, and Roadmap
Urbanowicz, R. J., & Moore, J. H. (2009) · Journal of Artificial Evolution and Applications · doi:10.1155/2009/736398 · Cited by 306 · see citing works
A knowledge based genetic algorithm for path planning of a mobile robot
Hu, Y., & Yang, S. X. (2004) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2004.1302402 · Cited by 299 · see citing works
Li, Z., Zhao, T., Chen, F., Hu, Y., Su, C., & Fukuda, T. (2017) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2017.2717461 · Cited by 200 · see citing works
Provably Efficient Reinforcement Learning with Linear Function ApproximationPreprint
Jin, C., Yang, Z., Wang, Z., & Jordan, M. I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.05388 · Cited by 193 · see citing works
Self-citationThomason, J., Murray, M., Cakmak, M., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1907.04957 · Cited by 173 · see citing works
Generalization and Exploration via Randomized Value FunctionsPreprint
Osband, I., Roy, B. V., & Wen, Z. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.0635 · Cited by 169 · see citing works
Li, J., Wang, J., Peng, H. H., Zhang, L., Hu, Y., & Su, H. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.05.091 · Cited by 140 · see citing works
Hierarchical A *: searching abstraction hierarchies efficiently
Holte, R. C., Perez, M. B., Zimmer, R. M., et al. (1996) · Cited by 135 · see citing works
Bassel, G. W., Glaab, E., Marquez, J., et al. (2011) · The Plant Cell · doi:10.1105/tpc.111.088153 · Cited by 107 · see citing works
TURTLE GEOMETRY The Computer as a Medium for Exploring Mathematics
Ross, P. (1987) · Bulletin of the London Mathematical Society · doi:10.1112/blms/19.5.507b · Cited by 101 · see citing works
Robust Quantum-Inspired Reinforcement Learning for Robot Navigation
Dong, D., Chen, C., Chu, J., & Tarn, T. (2010) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2010.2090896 · Cited by 95 · see citing works
Self-citationCuriosity Driven Exploration of Learned Disentangled Goal SpacesPreprint
Laversanne-Finot, A., Péré, A., & Oudeyer, P. (2018) · Conference on Robot Learning · doi:10.48550/arxiv.1807.01521 · Cited by 92 · see citing works
ITERATIVE FEEDBACK AND LEARNING CONTROL. SERVO SYSTEMS APPLICATIONS
Preitl, Ș., Precup, R., Preitl, Z., et al. (2007) · IFAC Proceedings Volumes · doi:10.3182/20070709-3-ro-4910.00004 · Cited by 91 · see citing works
Curiosity-driven Exploration in Deep Reinforcement Learning via Bayesian Neural NetworksPreprint
Houthooft, R., Chen, X., Duan, Y., Schulman, J. D., de Turck, F., & Abbeel, P. (2016) · arXiv.org · doi:10.48550/arxiv.1605.09674 · Cited by 81 · see citing works
UCB Exploration via Q-EnsemblesPreprint
Chen, R. Y., Sidor, S., Abbeel, P., & Schulman, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.01502 · Cited by 74 · see citing works
Fuzzy Rule-Based Reinforcement Learning for Load Balancing Techniques in Enterprise LTE Femtocells
Muñoz, P., Barco, R., Ruiz-Avilés, J. M., de-la-Bandera, I., & Aguilar-García, A. (2013) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2012.2234156 · Cited by 73 · see citing works
Incremental Reinforcement Learning With Prioritized Sweeping for Dynamic Environments
Wang, Z., Chen, C., Li, H., Dong, D., & Tarn, T. (2019) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2019.2899365 · Cited by 73 · see citing works
Self-citationHybrid Control for Robot Navigation - A Hierarchical Q-Learning Algorithm
Chen, C., Li, H., & Dong, D. (2008) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2008.921541 · Cited by 70 · see citing works
Self-citationLi, J., Wang, J., Wang, S., Peng, H. H., Wang, B., Qi, W., Zhang, L., & Su, H. (2020) · Assembly Automation · doi:10.1108/aa-08-2019-0148 · Cited by 68 · see citing works
Random curiosity-driven exploration in deep reinforcement learning
Li, J., Shi, X., Li, J., Zhang, X., & Wang, J.-Z. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.08.024 · Cited by 68 · see citing works
Fu, C., Sarabakha, A., Kayacan, E., Wagner, C., John, R. I., & Garibaldi, J. M. (2018) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2018.2810947 · Cited by 65 · see citing works
Watters, N., Matthey, L., Bošnjak, M., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.09275 · Cited by 65 · see citing works
smartPATH: A hybrid ACO-GA algorithm for robot path planning
Chaari, I., Koubâa, A., Bennaceur, H., et al. (2012) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2012.6256142 · Cited by 63 · see citing works
Lifelong Incremental Reinforcement Learning With Online Bayesian Inference
Wang, Z., Chen, C., & Dong, D. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3055499 · Cited by 62 · see citing works
Self-citationEvolution of a Negative-Rule Fuzzy Obstacle Avoidance Controller for an Autonomous Vehicle
Lilly, J. H. (2007) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2006.889918 · Cited by 58 · see citing works
Reinforcement learning in a rule-based navigator for robotic manipulators
Althoefer, K. A., Krekelberg, B., Husmeier, D., et al. (2001) · Neurocomputing · doi:10.1016/s0925-2312(00)00307-6 · Cited by 45 · see citing works
Wang, Z., Li, H.-X., & Chen, C. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2927320 · Cited by 39 · see citing works
Self-citationHong, Y.-D., & Lee, B. (2019) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2019.2955701 · Cited by 37 · see citing works
Multi-goal feasible path planning using ant colony optimization
Englot, B., & Hover, F. S. (2011) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2011.5980555 · Cited by 35 · see citing works
Efficient Reinforcement Learning via Probabilistic Trajectory Optimization
Pan, Y., Boutselis, G. I., & Theodorou, E. A. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2764499 · Cited by 27 · see citing works
A Machine Learning Approach to Classify Pedestrians’ Events based on IMU and GPS
Ahmed, M. U., Brickman, S., Dengg, A., et al. (2019) · Cited by 27 · see citing works
A Ruled Based System for Ear Problem Diagnosis and Treatment
Abu-Naser, S. S., & Al-Nakhal, M. A. (2016) · Cited by 27 · see citing works
Discounted UCB1-tuned for Q-learning
Saito, K., Notsu, A., & Honda, K. (2014) · Joint International Conference on Soft Computing and Intelligent Systems and International Symposium on Advanced Intelligent Systems (SCIS&ISIS) · doi:10.1109/scis-isis.2014.7044672 · Cited by 10 · see citing works
Evolutionary fuzzy navigation for security robots
Ju, H., Zhang, J., Pei, X., et al. (2008) · World Congress on Intelligent Control and Automation · doi:10.1109/wcica.2008.4592803 · Cited by 1 · see citing works
Design and Validation of Force Control Loops for a Parallel Manipulator
Carbone, G., Villegas, E., & Ceccarelli, M. (2013) · IGI Global eBooks · doi:10.4018/978-1-4666-3634-7.ch015 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Salvato, E., Fenu, G., Medvet, E., & Pellegrino, F. A. (2021) · IEEE Access · doi:10.1109/access.2021.3126658 · Cited by 200 · see citing works
“Navigation [39], [40], [41], [42] Manipulation [43], [44], [45], [46], [47], [3], [8], [48] Motion control [49], [50], [51], [52], [53], [54], [55], [56], [57], [58], [59] Locomotion [49], [50], [51], [60], [61], [62] When tackling a robot control problem with RL, according to the formulation and…”
- Authors
- Zhu, Y., Wang, Z., Chen, C., & Dong, D.
- Affiliations
- Nanjing University (CN)
- UNSW Canberra (AU)
- Venue
- IEEE/ASME Transactions on Mechatronics
- Published
- 2022-04
- Volume
- 27
- Issue
- 2
- Pages
- 846-857
- ISSN
- 1083-4435 (ISSN-L)
- 1941-014X
- OpenAlex
- W3154227779
- Cited by
- 57
- References
- 48
- Citations by year
- 2026: 8 · 2025: 12 · 2024: 20 · 2023: 12 · 2022: 3 · 2021: 2
- FWCI
- 3.27
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/ASME Transactions on Mechatronics (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (71732003, 62073160, 62006111)
- National Key Research and Development Program of China (2018AAA0101100)
- Synergistic Innovation Center of Jiangsu Modern Agricultural Equipment and Technology (4091600002)
- Australian Research Councils’s Discovery Projects (DP190101566)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotic Path Planning Algorithms
- Physical Sciences › Engineering › Aerospace Engineering › Robotics and Sensor-Based Localization
- Keywords
- robot navigation, wall following, sample complexity reduction, autonomous robots
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
