UAV formation control based on ensemble reinforcement learning

Abstract

Based on the frameworks of Multi-Agent Deep Deterministic Policy Gradient (MADDPG) and Deep Deterministic Policy Gradient (DDPG) algorithms, this paper investigates the UAV formation control problem. To address the convergence difficulties inherent in multi-agent algorithms, curriculum reinforcement learning is applied during the training phase to decompose the task into incremental stages. A progressively hierarchical reward function tailored for each stage is designed, significantly reducing the training complexity of MADDPG. In the inference phase, an ensemble reinforcement learning strategy is adopted to enhance the accuracy of UAV formation control. When the UAVs approach their target positions, the control strategy switches from MADDPG to the DDPG algorithm, thus achieving more efficient and precise control. Through ablation and comparative experiments in a self-developed Software in the Loop (SITL) simulation environment, the effectiveness and stability of the ensemble reinforcement learning algorithm in multi-agent scenarios are validated. Finally, real-world experiments further verify the practical applicability of the proposed algorithm (https://b23.tv/7ceLpLe).