Q-learning for the speed trajectory optimization of the freight train

The train speed trajectory optimization (TSTO) aims at finding the optimal speed trajectory considering the safety, energy efficiency, punctuality and stopping accuracy. From the perspective of mitigating the greenhouse effect, it’s quite significant to study the TSTO problem. This paper proposed an optimization algorithm based on Reinforcement Learning (RL). Firstly, a global optimization model using RL was established. In the model, the control sequence including the control regimes and their switching points was taken as the state. The optimization objectives were taken as the reward function. The adjustment of the position of the switching points in the control sequence was taken as the decision space of the agent. Secondly, an adjustment method of the control sequence based on the deep Q-learning and embedding matrix was proposed. The training data was sampled using the experience replay. The optimal control sequence was obtained through the iterative training of the neural network. Finally, the optimization algorithm based on RL was compared with the driving strategies based on the Pontryagin’s Maximum Principle (PMP) and the field test data. The results show that the energy consumption of the proposed algorithm is reduced by 0.16% in comparison with that of the PMP, which proves that the proposed method can be applied to the multi-objective optimization of the train operation. Comparing with the field test data, the energy consumption of the optimization algorithm is reduced by 4.89%, which demonstrates that the proposed method can be used to guide the drivers to drive the freight train energy-efficiently.