Single-Agent Reinforcement Learning Model for Adaptive Traffic Signal Control in Urban Corridors

Abstract

For multi-intersection control, existing research mainly adopts multiagent frameworks to tackle scalability issues. However, the traffic signal control (TSC) problem necessitates a single-agent framework, as a single control center monitors traffic conditions across all roads in the study area and coordinates the control of all intersections. This work proposes a novel single-agent RL-based urban corridor ATSC model: It abandons complex multiagent coordination and uses a single agent to centrally orchestrate signal timings across multiple intersections. Notably, the model is highly applicable to real-world settings. It defines state and reward functions based on a queue length metric—one that correlates with congestion and can be reliably estimated using probe vehicle data. Since probe vehicle data have become highly prevalent, this feature enables rapid, large-scale deployment. The single-agent framework primarily relies on a unique design of state, action, and reward. To facilitate learning and manage congestion, both state and reward functions are defined based on queue length, with actions designed to modulate queue dynamics. The queue length definition used in this study deviates slightly from conventional definitions but is closely correlated with congestion states. The method was comprehensively evaluated using the SUMO simulation platform under various traffic patterns. Experimental results show that the PPO algorithm demonstrates significantly faster learning than the DQN algorithm. The model effectively alleviates urban corridor congestion through coordinated multi-intersection control: During the simulation period of the entire scenario, the queue length did not exceed 50 vehicles, and instances where it exceeded 30 vehicles were relatively rare. Compared with the baseline scenario where queue lengths exceeded 150 vehicles, the proposed method significantly reduces road congestion. The work in this paper demonstrates the feasibility of controlling multiple intersections under a single-agent framework, and the control scope will be further expanded in the future.