IRS-guided policy search in model based reinforcement learning for virtually coupled train sets control

Abstract

Virtually Coupled Train Set (VCTS) is a railway operation concept that allows shorter train intervals. However, the complex dynamic model of train convoy in urban rail transit systems (URTS) makes it challenging to achieve efficient cooperative controller for VCTS. Existing methods for VCTS control assume accurate train dynamics models are available, which is difficult to achieve in real-world VCTS scenarios. In this paper, we present a framework that employs model-based reinforcement learning (MBRL) to control VCTS. In comparison with classical methods and model-free reinforcement learning (MFRL), MBRL can learn more efficiently with limited data, making it particularly useful in situations where interactions with the real environment are costly or dangerous, such as in VCTS. Guided Policy Search (GPS) is used in the MBRL framework to derive the VCTS control policy while ensuring stability, punctuality, and safe distance protection of trains, due to its remarkable ability to quickly learn from the environment. We construct a deep neural network (DNN) based train convoy model to approximate train convoy dynamics. An iterative random shooting (IRS) based optimization method is applied to generate a set of candidate policies, which are then used to train the GPS to obtain the target policy for VCTS. Our experiments show that the proposed IRS-guided policy search (IRS-GPS) in MBRL can provide effective VCTS cooperative control. Our proposed IRS-GPS in MBRL for VCTS cooperative control ensures that virtually coupled trains operate safely, stably, and on time at short intervals.