A path planning approach for unmanned surface vehicles based on dynamic and fast Q-learning

Abstract

Path planning is a critical issue for unmanned surface vehicles (USVs), and an effective path-planning algorithm enables USVs to accomplish the mission. In this paper, a novel algorithm called dynamic and fast Q-learning (DFQL) to solve the path planning problem for USV in partially known maritime environments is proposed, which combines Q-learning with artificial potential field (APF) to initialize the Q-table to provide a priori knowledge from the environment to USV. To accelerate the convergence of Q-learning to the optimal solution and avoid USV's behavior of walking randomly in the early stage of exploration, the static and dynamic rewards are proposed to motivate the USV to move toward the target. Moreover, the performance of the proposed algorithm is verified with offline and online modes for USV in different environmental conditions. By comparing with the existing methods, it shows that the proposed approach is effective for path planning of USV.