Optimization of three-dimensional urban underground logistics system alignment: a deep reinforcement learning approach

The three-dimensional (3D) alignment design of the underground logistics system (ULS) is a key factor in determining the rationality of its underground infrastructure layout. However, existing research mostly focuses on the two-dimensional horizontal alignment optimization, while neglecting the alignment design of vertical space scales, which makes it difficult for research results to effectively support projects implementation. Furthermore, traditional operations research methods struggle to handle the massive underground space data processing tasks required for detailed design of ULS alignment. Therefore, this study aims to consider the dual attributes of underground infrastructure and logistics infrastructure of ULS, proposing innovative deep reinforcement learning (DRL) method to achieve 3D alignment planning. Firstly, a DRL model was developed considering the key design factors of underground infrastructure alignment such as construction, cost, space suitability, and underground space resources. Secondly, given the large and complex optimization search space of the problem, a curriculum learning-based proximal policy optimization (CL-PPO) algorithm was proposed to efficiently solve the model. Finally, based on the Suzhou ULS case, simulations of alignment optimization results under different planning orientations were conducted to demonstrate the effectiveness of the model and algorithm. Results show that CL-PPO has significant advantages over PPO in terms of computational efficiency and global optimization capabilities. Additionally, planning orientations have a significant impact on the ULS alignment layout and project construction cost. The innovative optimization method not only enriches the infrastructure planning theory of ULS, but also provides space layout guidance for the utilization of urban underground space.