Optimizing Traffic Routes With Enhanced Double Q-Learning

Abstract

Traffic management has become a major issue in urban planning due to the increasing number of vehicles on urban roads. In this study, we introduce a novel approach using the Reinforcement Learning (RL) technique to address the vehicle routing problem (VRP). We explored the effectiveness of Double Q-Learning enhanced by Prioritized Experience Replay (DQL-PER) in optimizing vehicle routing to shorten travel times and reduce congestion. Using the Simulation of Urban Mobility (SUMO), this method manipulates traffic flow during peak hours to improve urban mobility. DQL-PER stands out due to its superior performance in managing complex traffic systems characterized by multiple interconnected variables and dynamic conditions inherent in urban traffic networks. Compared to standard Q-learning, DQL-PER reduces overestimation bias and facilitates faster convergence toward optimal solutions. This paper includes a comparison between DQL-PER and other RL methods, namely Q-learning, Double Q-learning (DQL), and deep Q-network (DQN), demonstrating its benefits through simulations and analysis. We also perform a scalability analysis to evaluate the algorithm's performance across network sizes, with node counts $N=39,545,1672,3236,\text{and}9652$, showing that DQL-PER performs exhaustively over larger networks, demonstrating its scalability potential. DQL-PER offers a scalable solution with the potential to transform urban transportation systems.