Combining Neural Gas and Reinforcement Learning for Adaptive Traffic Signal Control

Travel time of vehicles in urban traffic networks can be reduced by using Adaptive Traffic Signal Control (ATSC) to change the signal program according to the current traffic situation. Modern ATSC approaches based on Reinforcement Learning (RL) can learn the optimal signal control policy. While there are multiple RL based ATSC implementations available, most suffer from high state-action complexity leading to slow convergence and long training time. In this paper, the state-action complexity of ATSC based RL is reduced by implementing Growing Neural Gas learning structure as an integral part of RL, leading to high convergence rate and system stability. The presented approach is evaluated on a simulated signalized intersection, and compared with self-organizing map RL-based ATSC systems. Obtained results prove that the reduction of state-action complexity in this manner improves the effectiveness of RL based ATSC not needing to have an a priory analysis of needed number of neurons for state representation.