Reinforcement Learning Driving Strategy based on Auxiliary Task for Multi-Scenarios Autonomous Driving

Abstract

Reinforcement learning (RL) has made great progress in autonomous driving applications. However, using one RL based driving policy for multi-scenarios autonomous driving is still challenging for RL in autonomous driving. There are different observations and reward measurements in different scenarios. At the same time, there is also the problem of multi-source heterogeneous observation in autonomous driving. To address the problems above, we propose a reinforcement learning framework based on the auxiliary task. Firstly, we designed a reward function to enable vehicles to learn safe and efficient strategies. Further, an auxiliary task is designed to learn the characteristics of different scenarios so that the ego agent can adopt different strategies for different scenarios. Finally, in order to handle the driving problem in multiple scenarios, we propose a representation network based on Multi-layer perceptron (MLP), Convolutional neural network (CNN), and Transformer networks to learn multi-source heterogeneous observation. The multi-source heterogeneous observation consists of the ego vehicle state, the bird's eye view (BEV) state and neighbour vehicle states. Experiments show that our method achieves a higher success rate compared to a popular reinforcement learning algorithm.