Control of traffic network signals based on deep deterministic policy gradients

Abstract

The centralized control of traffic signals is a challenging problem due to the high randomness and complexity of traffic flow on urban road networks and the interaction between intersections. Centralized control leads to high spatial dimensionality of joint actions for traffic road network signal control. However, the decisive action output can solve the problem of “dimensional explosion” caused by joint actions. In this paper, we propose a deep deterministic policy gradient-based algorithm for centralized control of urban traffic road network signals. We simplify the traffic signal control to a four-phase green signal ratio, and the deep deterministic policy gradient-based algorithm deterministically outputs the control signal for each intersection based on the information of the whole traffic network, thus avoiding the problem of “dimensional explosion”. In particular, a new normalization function is proposed to generate the green rate of traffic signals and constrain it to a range of maximum and minimum sustained green time by linear transformation, which makes the generated traffic signals more realistic. Our proposed algorithm is shown to be optimal and robust compared to Deep Q-Network(DQN) based and fixed time control for 7-hour SUMO simulation of a single-peak traffic network with three intersections.