Road Intersection Model Based Reward Function Design in Deep Q-Learning Network for Traffic Light Control

Reinforcement learning based control is an effective approach to traffic light control. In terms of the Deep Q-learning Network (DQN) based traffic light control method, the reward function should be carefully designed, as it affects the performance of the control method and it also brings the challenge in standardized design. Thus, the control method would fail to perform well if the reward function was established improperly. If the reward function was too complicated, it would be time-consuming for the algorithm. To solve this problem, we proposed an intersection-model based reward function in the DQN algorithm. We investigated the different types of urban road intersection structures and analyzed their features. Then we used those features to formulate the reward function. Our model was evaluated via the simulation dataset under different road situations and got less emergency stop with 14%~32% improvement while the number of passing vehicles was also a bit more than the baseline. There were also 22% and 7% speedup in the training process under the tow training process.