State Complexity Reduction in Reinforcement Learning based Adaptive Traffic Signal Control

Abstract

The throughput of a signalized intersection can be increased by appropriate adjustment of the signal program using Adaptive Traffic Signal Control (ATSC). One possible approach is to use Reinforcement Learning (RL). It enables model-free learning of the control law for the reduction of the negative impacts of traffic congestion. RL based ATSC achieves good results but requires many learning iterations to train optimal control policy due to high state-action complexity. In this paper, a novel approach for state complexity reduction in RL by using Self-Organizing Maps (SOM) is presented. With SOM, the convergence rate of RL and system stability in the later stages of learning is increased. The proposed approach is evaluated against the traditional RL approach that uses Q-Learning on a simulated isolated intersection calibrated according to realistic traffic data. Presented simulation results prove the effectiveness of the proposed approach regarding learning stability and traffic measures of effectiveness.