Stochastic traffic control based on regional state transition probability model

This paper proposes a state transition probability model for an elementary traffic network with four intersections, which is substantially the extension of the state transition probability model for a link based on a queue dynamic model. The state of this model is the combination of states of roads between these four intersections, so as the reward of each state. For the links between elementary traffic networks, some constraints are added to revise the proposed model with the aim of alleviating traffic pressure on them. Based on the proposed model, traffic control problem is formulated as a Markov Decision Process(MDP). A sensitivity-based policy iteration(PI) algorithm is introduced to effectively solve the MDP. The numerical experiments of a subnetwork with 16 intersections show that this stochastic control scheme is capable of reducing the number of vehicles substantially compared with the isolated intersection control and the fixed-time control, especially under the unbalanced scenario.