Analyzing communication policies in cooperative multi-agent reinforcement learning for traffic signal control: A simulation-based study

Abstract

Traffic signal control (TSC) poses a significant challenge in intelligent transportation systems and has been addressed using multi-agent reinforcement learning (MARL). While centralized approaches are often impractical for large-scale TSC problems, decentralized approaches offer scalability but introduce new challenges, such as partial observability. Communication plays a crucial role in decentralized MARL, as agents must exchange information through messages to understand the system better and achieve effective coordination. Deep MARL has been applied, where multiple interacting agents share a common environment. However, many proposed deep MARL communication policies for TSC allow agents to communicate with all other agents and share global state. This can contribute to system noise and degrade overall performance since real-time global information sharing is impractical due to communication latency. This paper employs simulation-based approaches to assess the effectiveness of diverse information-sharing strategies to enhance overall system performance based on Cooperative Deep Q-Network (Co-DQN). Simulation experiment results suggest that the lack of a suitable sharing policy to provide a representative observation of the real state appears to affect performance more drastically than changes to the underlying traffic patterns.