An improved Dueling Deep Q-network with optimizing reward functions for driving decision method

Abstract

Aiming at poor effects and single consideration factors of traditional driving decision-making algorithm in high-speed and complex environment, a method based on improved deep reinforcement learning (DRL) is proposed in this paper. We innovatively design and optimize the reward function of the Dueling Deep Q network (Dueling DQN), and the factors such as safety, comfort, traffic efficiency and altruism are taken into account. The weight of each influencing factor is determined by the Analytic Hierarchy Process (AHP), which makes the influence of each factor on driving behavior decision-making more acceptable. Subsequently, a decision-making model of autonomous vehicles (AVs) is built by using improved Dueling DQN. Furthermore, the action space is enriched and combined with the trajectory planner, so that AVs can take appropriate behaviors in the longitudinal and lateral directions according to the environment. The output of the decision model can be combined with the underlying controller with a view to make the AVs maneuver reasonably. The driving decision-making method in two different traffic scenarios is simulated. Moreover, the improved method compares with other methods. The results illustrate that the improved Dueling DQN can make the AVs take safe, comfortable, efficient, and altruistic behavior.