Predictive Obstacle Avoidance Algorithm for Under-Actuated Unmanned Surface Vehicle Under Disturbances via Reinforcement Learning

Abstract

Due to the growing complexity of diverse maritime tasks, underactuated unmanned surface vehicle (USV) has become a research hotspot. The rapid development of deep reinforcement learning (DRL) technology has brought forth a novel approach for the USV autonomous control, rendering unnecessary the dynamical modeling of the target USV. To further improve the USV collision avoidance performance against maritime disturbances, this paper presents a predictive reinforcement learning method for USV obstacle avoidance control. A prediction module is designed to generate latent features that depict environmental states. After that, the prediction feature is provided for a DRL-based policy module to produce an action distribution for the underactuated unmanned surface vehicle. The proposed method in this paper can enable the USV avoid obstacle and reach the destination solely based on its local observational information, without relying on prior global information. Simulation and physical experiments have demonstrated that, compared to general DRL methods, the proposed method exhibits stronger robustness to environmental disturbances, enabling the USV to reach the destination while avoid the obstacle.