Simulation of Decision-making Method for Vehicle Longitudinal Automatic Driving Based on Deep Q Neural Network

Abstract

Decision-making is one of the key parts of automatic driving, which merges multi-sensor information and then makes task decisions based on driving needs. In this research, first of all, the longitudinal car following model of adaptive cruise control system (Abbreviated as ACC, the same below) has been created. Then, the state set and action set of the Deep Q Neural Network algorithm(Abbreviated as DQN, the same below) were designed based on the machine learning theory after analyzing the decision-making behavior of intelligent driving. And the expected Time Headway that is the Convergence condition of learning process iteration was used as the evaluation function related to the ego vehicle action. Furthermore, the three-dimensional simulation test environment has been built utilizing the Unreal Engine which allows to run industry standard CarSim math models as long as the VehicleSim Dynamics plugin from Mechanical Simulation is enabled. And the distance of car-following could be detected with the millimeter wave radar. When acquiring experimental data, we could complete intelligent driving decision control by using socket to communicate between Unreal Engine and neural network. Ultimately, through the unique collision detection experiment results in simulation environment, the validity and reliability of the decision-making based on DQN for ACC system could be verified.