TC-driver: A trajectory conditioned reinforcement learning approach to zero-shot autonomous racing

Abstract

Autonomous racing challenges perception, planning, and control algorithms, serving as a testbed for general autonomous driving. While traditional methods, such as model predictive control (MPC), can generate optimal control sequences, they are sensitive to modeling parameter accuracy. This article introduces trajectory-conditioned RL controller (TC-Driver), a reinforcement learning (RL) approach for robust control in autonomous racing, addressing tire parameter modeling inaccuracies. TC-Driver is conditioned by a trajectory from any high-level planner, combining RL’s learning capabilities with the reliability of traditional planning. Trained under varying tire conditions, it aims to generalize across different model parameters, enhancing real-world racing performance. Experimental results show TC-Driver improves generalization robustness compared with a state-of-the-art (SotA) end-to-end architecture. It achieves a 29-fold improvement in crash ratio when facing model mismatch and successfully transfers to unseen tracks with new features, while the baseline fails. In physical deployment, TC-Driver demonstrates zero-shot Sim2Real capabilities, outperforming end-to-end agents tenfold in crash ratio while maintaining similar driving characteristics in reality as in simulation. This hybrid RL architecture leverages traditional planning methods’ reliability while exploiting RL’s ability to handle model uncertainties, offering a robust solution for autonomous racing challenges.