Multi-agent neighborhood coordinated and holistic optimized actor-critic framework for adaptive traffic signal control

Adaptive Traffic Signal Control (ATSC) is a pivotal research area within intelligent transportation systems, aiming to enhance transportation efficiency and alleviate traffic congestion at signalized intersections. While multi-agent deep reinforcement learning has been extensively applied to ATSC, existing approaches commonly frame it as a fully cooperative problem, presupposing that all agents are committed to pursuing a collective optimal solution. However, achieving such altruistic cooperation is often impractical. Furthermore, as the number of agents escalates, challenges such as the curse of dimensionality and non-stationarity arise, complicating the learning process. To address these issues, we propose a novel perspective by framing ATSC as a competitive-cooperative game trade-off scenario and design a multi-agent framework, termed Neighborhood Coordinated and Holistic Optimized Actor-Critic (NcHo-AC). Specifically, we introduce a novel traffic state representation, design a sophisticated feature extraction network, develop a robust training algorithm, and leverage mean field approximation to model population-level agent interactions. These designs foster neighborhood-level cooperation and communication, facilitate the learning of the desired Nash equilibrium, and mitigate the noise caused by agents’ exploratory behaviors, thereby alleviating non-stationarity and the curse of dimensionality, while enhancing scalability to large-scale traffic networks. Comprehensive experiments conducted on both synthetic and real-world datasets demonstrate that NcHo-AC significantly outperforms state-of-the-art baselines across four key metrics: average travel time, average queue length, delay, and throughput, along with improved convergence, robustness, and interpretability.