Urban Path Planning Based on Improved Model-based Reinforcement Learning Algorithm

Abstract

With the development of the urban economy, and the continuous expansion of vehicle scale, traffic congestion has become the most serious problem affecting contemporary urban development. Using advanced road network information perception and transmission technologies, path planning under real-time road conditions has become an important means to solve this problem. Previously, our proposed model-based reinforcement learning multipath planning algorithm realized the rapid response of the path planning result, alleviating congestion drift to a certain extent. However, further research shows that the model performs poorly in extreme road network environments (the road network traffic pressure is 0) and cannot explore the complete path, the main reason is that the effect of model hyperparameters on the convergence of the algorithm was ignored. to solve this problems, we explore the hyperparameters in detail, especially discuss the discount factor γ and the finalReward to the model convergence by using Shenzhen road network data. the results show that when the discount factor γ and the finalReward value satisfy certain conditions, which is obtained in this study, the improved model-based method can guarantee the convergence stability of the algorithm under extreme road network environments. This paper reveals the importance of the design of hyperparameters γ and finalReward as well as their interrelationship on the convergence of reinforcement learning algorithms and we hope to give some insights in the field which explore hyperparameters of reinforcement learning algorithm.