Real-Time Reinforcement Learning Optimized Energy Management for a 48V Mild Hybrid Electric Vehicle

Abstract

Energy management of hybrid vehicle has been a widely researched area. Strategies like dynamic programming (DP), equivalent consumption minimization strategy (ECMS), Pontryagin’s minimum principle (PMP) are well analyzed in literatures. However, the adaptive optimization work is still lacking, especially for reinforcement learning (RL). In this paper, Q-learning, as one of the model-free reinforcement learning method, is implemented in a mid-size 48V mild parallel hybrid electric vehicle (HEV) framework to optimize the fuel economy. Different from other RL work in HEV, this paper only considers vehicle speed and vehicle torque demand as the Q-learning states. SOC is not included for the reduction of state dimension. This paper focuses on showing that the EMS with non-SOC state vectors are capable of controlling the vehicle and outputting satisfactory results. Electric motor torque demand is chosen as action. In the reward function definition, the fuel consumption contains engine fuel consumption and equivalent battery fuel consumption. The Q-learning table is executed over Worldwide Harmonized Light Vehicle driving cycle. The learning process shows fast convergence and fuel economy improvement of 5%. The Q-learning strategy is compared with baseline thermostatic rule-based EMS and ECMS. The Q-learning result shows 8.89% fuel economy improvement than the rule-based EMS and 0.88% than the ECMS.