Distributed multiple unmanned surface vehicles path planning integrated control framework in complex scenarios

Abstract

In complex interactive environments, distributed multiple unmanned surface vehicles (multi-USV) systems need to make efficient and safe collision avoidance decisions based on limited sensor information, while complying with the International Regulations for Preventing Collisions at Sea (COLREGs), which is a challenging task. In this paper, based on the Optimal Reciprocal Collision Avoidance (ORCA) algorithm, we propose a new concept: the cost of escaping velocity obstacles (VO) and effectively integrate it into the Deep Reinforcement Learning (DRL) framework. In addition, this paper considers the problem of dynamic change in the number of USVs and credit assignment, and trains a distributed multi-USV path planning policy DRL-Evoc by combining the mechanisms of curriculum learning and frame-skipping decision. To address the partial observability limitations of the DRL framework, the traditional global path planning algorithm A* is further integrated with the DRL-Evoc policy to construct the integrated control framework (ICF) for multi-USV path planning. Performance testing and comparative analysis were conducted in the Unity3D environment, demonstrating that the ICF exhibits strong collision avoidance capabilities, generalization, and compliance with COLREGs. This study provides a new approach for multi-USV path planning.