Formation control of multiple AUVs using decentralized self-attention based soft actor–critic model

Abstract

One of most important topics studied in the field of multiple robots is cooperative formation control. Formation structure is a combination in which agents maintain the desired form and at the same time execute the assigned commands. This article deals with the formation control of multiple Autonomous Underwater Vehicles (AUVs) based on a distributed reinforcement learning algorithm. The proposed Decentralized Self-Attention based Soft Actor–Critic (DEC-ASAC in short) method uses an attention mechanism and maximum entropy reinforcement learning control, so as to enable AUVs to learn formation control independently. The corresponding environmental states, action space and reward schemes are designed for leader and follower AUVs. Simulations and lake test verify that the proposed DEC-ASAC algorithm can stably and effectively learn control policies during training process, achieving effective control of multiple AUVs to keep different formation shapes.