Potential-Game-Driven Formation Control of AUVs: An Inverse-Reinforcement-Learning-Based Solution

Abstract

The formation control of autonomous underwater vehicles (AUVs) is widely used to perform complex maritime missions. However, most works simply add all individual interests of AUVs together to achieve the formation task, which cannot reveal their interaction relationship and guarantee the full exploitation of formation interests. In this article, we are concerned with a potential-game-driven formation issue for AUVs. Specifically, a potential game-driven formation framework is first developed to formulate the decision-making procedure of AUVs, where the cost function not only reflects the common formation objective but also captures the self-interest. Based on this, the formation optimization problem of AUVs is constructed. To solve the aforementioned problem, an inverse reinforcement learning (IRL)-based formation controller is designed, whose aim is to maximize the formation benefits and drive AUVs to the desired formation shape. Besides that, the theory analyses, including the satisfiability of the potential game, the accessibility of the Nash equilibrium, and the convergence of the formation controller, are provided. The main innovations of this article are as follows: 1) the potential-game-driven framework can fully exploit individual AUV interests and enhance formation robustness over traditional cooperative frameworks and 2) the IRL-based formation controller has better environmental adaptability to the unknown underwater environment compared with traditional reinforcement learning controllers. Finally, simulations and experiments are conducted to validate the effectiveness of our solution.