The USV Path Planning of Dueling DQN Algorithm Based on Tree Sampling Mechanism

Abstract

The path planning and obstacle avoidance of USV (unmanned surface vessel) has become a research hotspot in recent years. Among them, the DQN algorithm has achieved good results in the obstacle avoidance and path planning problems of unmanned surface vessel. However, the algorithm suffers from the problems that the sampling method does not make full use of the stored information and the randomness of action selection during the training process is too large and the convergence is too slow. In this paper, we propose a Dueling DQN algorithm to optimize obstacle avoidance and path planning, which based on tree sampling mechanism. The Dueling DQN algorithm will decomposes the value function Q into a state-value function (V) and a dominance function (A). Meanwhile, the absolute value of TD-error is directly used as a priority indicator for priority sampling in the sampling process. Subsequently, the network model is built and experiments are conducted on each of the four maps. As a result, the convergence steps and loss values of the proposed algorithm on the four paths are better than those of the DQN algorithm. It shows that the dueling DQN algorithm can effectively use the stored information for optimal path planning.