A Train Cooperative Operation Optimization Method based on Improved Reinforcement Learning Algorithm*

Abstract

This paper mainly focuses on the high-speed train cooperative operation problem. To solve this problem, this paper presents a speed curve optimization method based on improved reinforcement learning algorithm. First, according to the train dynamics system, we build the speed curve optimization object. In order to realize the cooperative operation of trains, we use the artificial potential field method to establish the reward function for train spacing. At the same time, to ensure passenger comfort, train jerk rate also needs to be added into the reward function. And then, agent of improved reinforcement learning is established. The improved reinforcement learning algorithm is different from the general reinforcement learning algorithm in that the observation dimension of policy network is manually reduced compared with that of the Q value network to improve the learning speed of the algorithm. At the same time, in order to reduce the agent's attempts to perform useless actions in some states, a reference controller is added to the system to further accelerate the learning process. In addition, training parameters need to be set, such as training termination conditions, maximum number of steps, desired global reward value, and so on. After the training. The Agent can generate a desirable speed curve of train based on constraints of vehicle output and jerk rate under cooperative operation.