Path following control of under-actuated autonomous surface vehicle based on random motion trajectory dataset and offline reinforcement learning

Abstract

To solve the path following problem in navigation tasks for under-actuated autonomous surface vehicles (ASVs), this paper proposed a path following control method which combines trajectory dataset of random ship motion and offline reinforcement learning (RM-ORL). The method does not require the reinforcement learning (RL) agent to interact with the environment while training the policy, and it can obtain training datasets with a lower cost. In RM-ORL, the irregular motion data of the ASV in open water is first collected. Then the desired path is reconstructed using the B-spline function and the path points along the motion trajectories. Thus the offline dataset will be enhanced with the motion data and the new path. Finally, the conservative Q-learning algorithm is utilized to train the path following controller. The path deviation in simulation maps, rudder data and ship motion parameters of RM-ORL, online RL and other offline RL policies trained on different datasets are compared. The simulation results illustrate that the RM-ORL achieves comparable path following accuracy to that of online RL agent and offline RL agent trained on expert data, while surpassing the one trained on online agent replay buffer data. The rudder steering amplitude of RM-ORL is also smaller than that of other policies, which verifies the effectiveness of our method applied to the path following control of under-actuated ASV.