Robust Model-based Reinforcement Learning USV System Guided by Lyapunov Neural Networks

Abstract

This paper explores the potential of Lyapunov function approximated by neural networks in unmanned surface vehicles (USV) control problem. A novel model-based reinforcement learning method, Lyapunov filtered probabilistic model predictive control (LFPMPC) is proposed to explore the USV control policy under the guidance of Lyapunov neural networks. The USV system based on LFPMPC is developed and evaluated by a USV simulator driven by real boat data in position-keeping task with various environmental disturbances. Taking the output of Lyapunov neural networks as one metric of the system robustness in the cost function, the proposed approach demonstrated significant superiorities in not only control stability against disturbances but also learning capabilities of the system model compared with the baseline approach without Lyapunov neural networks.