Congestion pricing in multi-modal networks: An application of deep reinforcement learning

Abstract

Developing a real-time dynamic pricing mechanism that proactively generates toll profiles while incorporating demand elasticity and travellers’ heterogeneity remains a significant challenge. Many existing approaches suffer from low transferability and rely heavily on precise estimation of network parameters such as critical accumulation. This study introduces a data-driven, cordon-based pricing framework using reinforcement learning to optimise traffic flow and address these limitations. A multi-modal, trip-based Macroscopic Fundamental Diagram (MFD) simulation has been developed, capable of capturing individual mode choice decisions. Traveller heterogeneity is addressed through variations in origin and destination, trip length, departure time, and value of time (VoT). To establish a tolling strategy that maximises network outflow (minimises total travel time) and proactively addresses traffic congestion, a Double Deep Q-Network (DDQN) agent has been introduced. Remarkably, without prior knowledge of network parameters, the agent successfully regulates car accumulation at critical levels to maximise network outflow. Sensitivity analysis reveals that even with a 20% margin of error in input data, the agent remains effective in mitigating congestion. Additionally, the agent’s transferability has been evaluated under various traffic conditions and dynamics by introducing different demand profiles and MFD coefficients, demonstrating robust performance. Benchmark comparisons with a feedback controller across all scenarios further confirm that the DDQN agent consistently outperforms.