A Hybrid Pure Pursuit–PID and TD3 Control Framework for Differential-Drive Mobile Robots
Open accessThe DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
In this study, a hybrid control structure that combines classical control methods with reinforcement learning is proposed for target tracking of a differential-drive mobile robot. In the proposed framework, Pure Pursuit–based reference generation and an inner-loop Proportional–Integral–Derivative (PID) controller are integrated with the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. While the classical Pure Pursuit–PID structure ensures baseline stability and safety, the TD3-based learning component enhances performance by generating corrective torque contributions when classical control becomes insufficient. First, experiments are conducted under nominal system conditions to demonstrate that the proposed approach preserves the fundamental behavior of the classical controller and achieves comparable trajectory-tracking performance. Subsequently, the robustness of the control schemes is evaluated under parametric uncertainties applied to the chassis and wheel masses. Experimental results show that the classical PID-based control structure fails to achieve stable convergence under these uncertainties, whereas the proposed reinforcement learning–enhanced controller consistently reaches the target in a stable and reliable manner across all tested scenarios, achieving a 100% success rate. Quantitative analysis further indicates that the proposed method significantly improves convergence behavior and tracking stability by reducing trajectory deviations and eliminating oscillatory behavior observed in the classical controller. Furthermore, the influence of key learning components—such as minibatch size, replay buffer capacity, optimization algorithm, and network architecture—on control performance is systematically analyzed through a comprehensive ablation study. The results demonstrate that the proposed residual learning approach provides superior stability and robustness, effectively overcoming the practical limitations of classical control methods. Accordingly, this study presents a reliable and generalizable mobile robot control architecture capable of coping with model uncertainties and dynamic variations that are inevitable in real-world applications.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Soft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594 · cited 1×
“…algorithms such as Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO), and Soft Actor–Critic (SAC) have been successfully applied to continuous-state and continuous-action control problems, including mobile robot navigation and path-following tasks [27], [28], [29].”
MuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767 · cited 1×
“These concerns have motivated the development of hybrid control frameworks that combine classical control strategies with learning-based methods [35].”
Addressing Function Approximation Error in Actor-Critic Methods
Fujimoto, S., van Hoof, H., & Meger, D. (2018) · arXiv · doi:10.48550/arxiv.1802.09477 · Cited by 3,650 · cited 3×
“The Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm was introduced to mitigate several limitations observed in earlier deterministic policy gradient methods, particularly value overestimation and training instability [12].”
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05952 · Cited by 3,120
Deep reinforcement learning based mobile robot navigation: A review
Zhu, K., & Zhang, T. (2021) · Tsinghua Science and Technology · doi:10.26599/tst.2021.9010012 · Cited by 499 · cited 1×
“A key advantage of RL lies in its ability to learn control policies directly through interaction with the environment, without requiring explicit mathematical models of system dynamics [25], [26].”
A Review of some Pure-Pursuit based Path Tracking Techniques for Control of Autonomous Vehicle
Samuel, M., Hussein, M., & Binti, M. (2016) · International Journal of Computer Applications · doi:10.5120/ijca2016908314 · Cited by 179 · cited 1×
“Among geometric-based path-following meth-ods, the Pure Pursuit algorithm has received considerable attention [17], [18].”
Review of Wheeled Mobile Robots’ Navigation Problems and Application Prospects in Agriculture
Gao, X., Li, J., Fan, L., Zhou, Q., Yin, K., Wang, J., Song, C., Huang, L., & Wang, Z. (2018) · IEEE Access · doi:10.1109/access.2018.2868848 · Cited by 173 · cited 2×
“Autonomous target tracking and path-following problems for mobile robots have long been central topics in the robotics literature [2], [14].”
Deep Reinforcement Learning of Mobile Robot Navigation in Dynamic Environment: A Review
Zhu, Y., Hasan, W. Z. W., Ramli, H. R., Norsahperi, N. M. H., Kassim, M. S. M., & Yao, Y. (2025) · Sensors · doi:10.3390/s25113394 · Cited by 117 · cited 1×
“In recent years, reinforcement learning (RL)-based control methods have emerged as promising alternatives to classical approaches in mobile robot con-trol [9], [24].”
Regulated pure pursuit for robot path tracking
Macenski, S., Singh, S., Martín, F., et al. (2023) · Autonomous Robots · doi:10.1007/s10514-023-10097-6 · Cited by 100 · cited 1×
“Its widespread adoption is largely due to its simple geometric formulation, ease of implementation, and low computational complexity, which make it well suited for real-time mobile robot and autonomous vehicle applications [19], [20].”
Vehicle Path Tracking Control Using Pure Pursuit With MPC-Based Look-Ahead Distance Optimization
Kim, S., Lee, J., Han, K., & Choi, S. B. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3304427 · Cited by 89 · cited 1×
“Fixed parameter selection limits the adaptability of the algorithm and can result in suboptimal performance under varying operating conditions, including sharp turns, velocity changes, and external disturbances [21], [22].”
Mobile Robot Path Planning Based on Improved DDPG Reinforcement Learning Algorithm
Dong, Y., & Zou, X. (2020) · IEEE International Conference on Software Engineering and Service Science (ICSESS) · doi:10.1109/icsess49938.2020.9237641 · Cited by 84 · cited 1×
“A key advantage of RL lies in its ability to learn control policies directly through interaction with the environment, without requiring explicit mathematical models of system dynamics [25], [26].”
Path Planning of a Mobile Robot for a Dynamic Indoor Environment Based on an SAC-LSTM Algorithm
Zhang, Y., & Chen, P. (2023) · Sensors · doi:10.3390/s23249802 · Cited by 56 · cited 1×
“…algorithms such as Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO), and Soft Actor–Critic (SAC) have been successfully applied to continuous-state and continuous-action control problems, including mobile robot navigation and path-following tasks [27], [28], [29].”
A Constrained Fuzzy Control for Robotic Systems
Xue, W., Zhou, B., Chen, F., et al. (2024) · IEEE Access · doi:10.1109/access.2024.3352129 · Cited by 42 · cited 1×
“Recent studies, such as the constrained type-3 fuzzy control framework proposed by Xue et al. [6], demonstrate enhanced robustness by modeling higher-order uncertainties.”
Arega, T. B., Tesfa, Y. M., & Abdissa, C. M. (2025) · IEEE Access · doi:10.1109/access.2025.3577269 · Cited by 37 · cited 2×
“Early studies on differential-drive mobile robots primarily relied on classical control techniques derived from kinematic or simplified dynamic models [5].”
Shen, Y., Shen, Y., Zhang, Y., et al. (2025) · Agriculture · doi:10.3390/agriculture15181917 · Cited by 37 · cited 1×
“Its widespread adoption is largely due to its simple geometric formulation, ease of implementation, and low computational complexity, which make it well suited for real-time mobile robot and autonomous vehicle applications [19], [20].”
Hazem, Z. B., Saidi, F., Güler, N., et al. (2025) · Automation · doi:10.3390/automation6040056 · Cited by 34 · cited 1×
“By incorporating a twin-critic architecture and delayed policy updates, TD3 achieves improved learning stability and reliability, which has been demonstrated in various mobile robot control applications [32], [33].”
Inspection Robot Navigation Based on Improved TD3 Algorithm
Huang, B., Xie, J., & Yan, J. (2024) · Sensors · doi:10.3390/s24082525 · Cited by 32 · cited 1×
“By incorporating a twin-critic architecture and delayed policy updates, TD3 achieves improved learning stability and reliability, which has been demonstrated in various mobile robot control applications [32], [33].”
Dang, S. T., Dinh, X. M., Kim, T. D., et al. (2023) · Electronics · doi:10.3390/electronics12112345 · Cited by 29 · cited 1×
“Within this framework, controllers such as PID, nonlinear feed-back control, and sliding mode control have demonstrated acceptable tracking performance under ideal assumptions, including accurate model parameters and negligible external disturbances [15], [16].”
Challenges for Reinforcement Learning in Quantum Circuit Design
Altmann, P. J., Stein, J., Kölle, M., Bärligea, A., Zorn, M., Gabor, T., Phan, T., Feld, S., & Linnhoff–Popien, C. (2024) · IEEE International Conference on Quantum Computing and Engineering (QCE) · doi:10.1109/qce60285.2024.00187 · Cited by 27
Jiang, X., Kuroiwa, T., Cao, Y., Sun, L., Zhang, H., Kawaguchi, T., & Hashimoto, S. (2025) · Sensors · doi:10.3390/s25030745 · Cited by 23 · cited 1×
“Among geometric-based path-following meth-ods, the Pure Pursuit algorithm has received considerable attention [17], [18].”
Ding, N., Peng, C., Lin, M., & Wu, C. (2022) · Journal of Circuits Systems and Computers · doi:10.1142/s0218126622501535 · Cited by 20 · cited 1×
“Autonomous target tracking and safe navigation of mobile robots have become significant research topics in recent years, attracting considerable attention from both academia and industry [1], [2].”
The Impact of Parametric Uncertainties on Mobile Robots Velocities and Pose Estimation
Filho, J. G. N. D. C., Carvalho, E. Á. N., Molina, L., & Freire, E. O. (2019) · IEEE Access · doi:10.1109/access.2019.2919335 · Cited by 19 · cited 3×
“However, their nonlinear dynamics and sensitivity to environmental uncertainties substantially complicate the control problem [3], [4].”
Chen, R. (2024) · Highlights in Science Engineering and Technology · doi:10.54097/6q4xxg69 · Cited by 16 · cited 1×
“Within this framework, controllers such as PID, nonlinear feed-back control, and sliding mode control have demonstrated acceptable tracking performance under ideal assumptions, including accurate model parameters and negligible external disturbances [15], [16].”
Deep Reinforcement Learning for Motion Planning of Mobile RobotsPreprint
Butyrev, L., Edelhäußer, T., & Mutschler, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.09260 · Cited by 16
Martínez, D., Riazuelo, L., & Montano, L. (2025) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2025.105020 · Cited by 15
Mobile rolling robots designed to overcome obstacles: A review
García, J. M., & Duarte, F. (2024) · Forces in Mechanics · doi:10.1016/j.finmec.2024.100283 · Cited by 14
Deep Reinforcement Learning Based Collision Avoidance Algorithm for Differential Drive Robot
Lu, X., Cao, Y., Zhao, Z., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-97586-3_17 · Cited by 7 · cited 1×
“In particular, Deep Reinforcement Learning (DRL) enable the control of complex and nonlinear systems without requiring explicit system modeling [9], [10], [11].”
RL-DWA Omnidirectional Motion Planning for Person Following in Domestic Assistance and Monitoring
Eirale, A., Martini, M., & Chiaberge, M. (2023) · International Conference on Automation, Robotics and Applications · doi:10.1109/icara56516.2023.10125630 · Cited by 6 · cited 1×
“In recent years, reinforcement learning (RL)-based approaches have emerged as promising alternatives for mobile robot control [8].”
A comparison study of passive, active, and semi-active suspension systems
Demircioğlu, U., & Çakir, M. T. (2024) · International Journal of Vehicle Noise and Vibration · doi:10.1504/ijvnv.2024.142742 · Cited by 6
Self-citationIntegrating Classical Control into Reinforcement Learning Policy
Huang, Y., Gu, C., & Guan, X. (2019) · Neural Processing Letters · doi:10.1007/s11063-019-10127-4 · Cited by 5 · cited 1×
“Despite these improvements, studies have reported that purely TD3-based controllers may still exhibit unsafe or oscillatory behavior, especially during early training stages, due to the exploration requirements inherent in continuous action spaces [34].”
Mobile robot navigation based on intrinsic reward mechanism with TD3 algorithm
Yang, J., Liu, Y., Zhang, J., Guan, Y., & Shao, Z. (2024) · International Journal of Advanced Robotic Systems · doi:10.1177/17298806241292893 · Cited by 5 · cited 1×
“Nevertheless, these methods often face practical challenges, such as unstable learning dynamics, sensitivity to hyperparameter selection, overfitting to specific environments, and long training durations [30], [31].”
Model-Based Hybrid Control of Pure Pursuit and Stanley Methods for Vehicle Path Tracking
Jung, H. (2025) · Sensors · doi:10.3390/s25206491 · Cited by 4
Pham, V. T., Štefek, A., Křivánek, V., et al. (2023) · Applied Sciences · doi:10.3390/app13020997 · Cited by 3 · cited 2×
“Similarly, Pham et al. [7] developed a genetic algorithm–based fuzzy logic controller for differential-drive robots, achieving improved energy efficiency and trajectory tracking performance.”
Budak, S., Sungur, C., & Durdu, A. (2026) · Physica Scripta · doi:10.1088/1402-4896/ae2c22 · Cited by 3 · cited 1×
“In particular, Deep Reinforcement Learning (DRL) enable the control of complex and nonlinear systems without requiring explicit system modeling [9], [10], [11].”
Segmenting Action-Value Functions over Time Scales in SARSA via TD(Δ)
Humayoo, M., Zheng, G., Dong, X., et al. (2025) · Algorithms · doi:10.3390/a18110729 · Cited by 1 · cited 1×
“Actor–critic-based algorithms, which are well suited to deterministic and continuous action spaces, have therefore been widely applied in mobile robot control tasks [12], [13].”
Deep Reinforcement Learning with Enhanced PPO for Safe Mobile Robot Navigation
Taheri, H., Hosseini, S. R., & Nekoui, M. A. (2024) · arXiv · doi:10.48550/arxiv.2405.16266
Metadata
- Authors
- Demircioglu, U., Bakir, H., Almarri, B., & Abdul Hafez, A. H.
- Affiliations
- Faculty of Engineering and Natural Sciences, Sivas University of Science and Technology, Sivas, Türkiye
- King Faisal University (SA)
- Department of Computer Science, College of Computer Sciences and Information Technology, King Faisal University, Al-Ahsa, Saudi Arabia
- Venue
- IEEE Access
- Published
- 2026
- Volume
- 14
- Pages
- 80817-80838
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W7160621918
- Cited by
- 0
- References
- 31
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- IEEE Access (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Deanship of Scientific Research, Vice Presidency for Graduate Studies and Scientific Research, King Faisal University, Saudi Arabia, (KFU262319)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Control and Dynamics of Mobile Robots
- Physical Sciences › Engineering › Control and Systems Engineering › Adaptive Dynamic Programming Control
- Keywords
- differential drive robot, target tracking, pure pursuit, PID control, TD3, trajectory tracking, parameter uncertainty, robustness, residual learning, ablation study
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.