MADDPG-GST for coordinated variable speed limit and ramp metering: A hybrid action deep reinforcement learning approach to bottleneck congestion mitigation

Abstract

Expressway merging bottlenecks are major sources of traffic congestion, where insufficient coordination among multiple traffic streams leads to severe flow disruptions. Although variable speed limits (VSL) and ramp metering (RM) are commonly used to mitigate congestion, their independent operation and mismatched control scopes often result in suboptimal outcomes. To address this, this study proposes a coordinated VSL–RM strategy based on multi-agent deep reinforcement learning. The control task is modeled as a Markov Decision Process (MDP), allowing joint policy learning between decentralized VSL and RM agents. A customized Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm is employed to dynamically optimize coordination policies. To bridge the gap between discrete VSL and continuous RM action spaces, the Gumbel-Softmax Trick (GST) is integrated into the learning process for differentiable hybrid action optimization. Additionally, a transfer learning mechanism is incorporated to ensure efficient policy adaptation across diverse traffic scenarios. Simulation results under varying demand levels show that the proposed strategy achieves 7.3 %–34.1 % improvements in traffic efficiency and stability compared to traditional methods. It also demonstrates strong transferability, reducing retraining time by up to 63.7 % and traffic delays by up to 62.7 %, while maintaining robust control under overspeed disturbances and control lag conditions.