HGAT and Multi-Agent RL-Based Method for Multi-Intersection Traffic Signal Control

Abstract

In the field of multi-intersection signal control, Reinforcement Learning (RL) has demonstrated significant technical benefits in terms of optimization speed, stability, and scalability. Moreover, Graph Neural Networks (GNN) exhibit strong functional abilities in capturing multi-agent and multi-task relationships within non-Euclidean spaces, surpassing other models in these regards. However, existing algorithms based on homogeneous graph neural networks struggle to encapsulate the heterogeneous nature of traffic object. Therefore, this paper proposes a novel algorithm, termed the Heterogeneous Graph Attention Network (HGAT) and Multi-Agent RL (MARL)-based Method for Multi-Intersection Traffic Signal Control. HGAT-MARL establishes a heterogeneous graph model to explicitly represent the states of various traffic objects in the road network. Leveraging a graph attention model, it learns and aggregates traffic state data within intersections and their proximate adjacent intersections, thereby capturing the mechanism of information transfer among diverse object such as vehicles and intersections. Subsequently, within the multi-agent RL framework, agents collaborate by sharing global states and rewards among multi-intersection, actively exploring and refining the optimal control policy to achieve optimized control of global traffic flow. Experimental results on real-world traffic datasets demonstrate that HGAT-MARL exhibits significant advantages in reducing vehicle travel time.