Integrating Multimodality and Partial Observability Solutions Into Decentralized Multiagent Reinforcement Learning Adaptive Traffic Signal Control

Abstract

Adaptive Traffic Signal Control (ATSC) systems leverage sensor data to dynamically adjust signal timings based on real-time traffic conditions but they often suffer from partial observability (PO) due to sensor limitations and restricted detection ranges. This study addresses PO in fully decentralized ATSC systems by introducing eMARLIN-T, a controller designed to enhance performance by incorporating historical information in the decision-making process. Additionally, ATSC systems are commonly optimized to improve the performance of the general traffic, ignoring the impact on transit. On the other hand, traditional transit signal priority (TSP) strategies, which overlay preferential strategies for transit vehicles onto general traffic fixed signal plans, often lead to negative impacts on the general traffic. Thus, this paper tackles the challenge of optimizing traffic signals to benefit both public transit and general vehicular traffic. To address this, a novel decentralized multimodal multiagent reinforcement learning (RL) signal controller, eMARLIN-T-MM, is developed. This controller integrates a transformer-based encoder for transforming the state observations into a latent space and an executor Q-network for decision-making. Tested on a simulation of five intersections in North York, Toronto, eMARLIN-T-MM significantly reduces the total person delays by 58% to 74% across various bus occupancy levels compared to pre-timed signals, outperforming the other decentralized RL-based ATSCs. In addition, eMARLIN-T-MM can automatically adapt to changes in the levels of occupancy, allowing it to optimize the intersection performance in response to varying transit and traffic demands.