Trajectory optimization of train cooperative energy-saving operation using a safe deep reinforcement learning approach

Abstract

Energy-efficient optimization of train speed profiles can effectively reduce the traction energy consumption of urban rail transit systems. Existing reinforcement learning (RL) optimization models for optimizing train operation profiles do not proactively handle the utilization constraints of regenerative braking energy (RBE). For this reason, this paper proposes an optimization model of train energy-saving profiles under multi-train cooperative operations. A novel safe deep reinforcement learning algorithm, guided by heuristic rules, is developed to optimize energy-saving train driving strategies in various scenarios. To ensure safety during the agent’s learning processes, a two-layer protection mechanism with soft constraint and truncation penalties is employed. Dynamic energy constraints are also introduced to enable the RBE utilization between trains. The simulation experiments using a real metro line data show that the proposed model and algorithm not only generate safe and energy-efficient profiles that meet metro operational constraints but also maximize the RBE utilization between trains, significantly reducing traction energy consumption.