An invulnerable leader–follower collision-free unmanned aerial vehicle flocking system with attention-based Multi-Agent Reinforcement Learning

Abstract

Deep reinforcement learning has been proved useful for the flocking control of Unmanned Aerial Vehicle (UAV) swarm with the leader–followers topology. However, it remains unclear how to fully utilize the spatial information among the followers to alleviate the problems of sparse reward and policy convergence. In this article, we propose a novel multi-agent reinforcement learning-based fixed-wing UAV flocking approach named Attention Based Cucker–Smale (ABCS) Flocking to learn collision-free leader–follower flocking by utilizing the information among followers with the attention mechanism. Specifically, we design an explainable Cucker–Smale criterion-based flocking reward named ABCS reward to associate the followers with high efficiency in flocking. Then, a leader-guide attention mechanism is proposed by transferring the difference between leader and follower as the weights to support the follower selectively utilizing the followers’ information. As a result, we prove that an optimal state can be achieved so that each follower can keep an optimal distance from the other followers when the ABCS reward is maximized. In addition, we prove the ABCS reward is bounded, which can be used to indicate learning convergence. To improve the invulnerability, we proposes a leader selection method based on ABCS flocking, which can effectively select a new leader when the old leader is destroyed. Finally, we demonstrate the effectiveness of ABCS Flocking over the Multi-Agent Deep Deterministic Policy Gradient approach using various reward functions with various numbers of followers in the obstacles and leader-destroyed scenarios. The code is published on github https://github.com/YunxiaoGuo/ABCS-Flocking.