Imitation reinforcement learning energy management for electric vehicles with hybrid energy storage system

Abstract

Deep reinforcement learning has become a promising method for the energy management of electric vehicles. However, deep reinforcement learning relies on a large amount of trial-and-error training to acquire near-optimal performance. An adversarial imitation reinforcement learning energy management strategy is proposed for electric vehicles with hybrid energy storage system to minimize the cost of battery capacity loss. Firstly, the reinforcement learning exploration is guided by expert knowledge, which is generated by dynamic programming under various standard driving conditions. The expert knowledge is represented as the optimal power allocation mapping. Secondly, at the early imitation stage, the action of the reinforcement learning agent approaches the optimal power allocation mapping rapidly by using adversarial networks. Thirdly, a dynamic imitation weight is developed according to the Discriminator of adversarial networks, making the agent transit to self-explore the near-optimal power allocation under online driving conditions. Results demonstrate that the proposed strategy can accelerate the training by 42.60% while enhancing the reward by 15.79% compared with traditional reinforcement learning. Under different test driving cycles, the proposed method can further reduce the battery capacity loss cost by 5.1%–12.4%.