CSMAAC: Multi-Agent Reinforcement Learning Based Flight Control in Partially Observable Multi-UAV Assisted Crowd Sensing Systems

Abstract

In mobile crowd sensing systems, existing flight control methods enable uncrewed aerial vehicles (UAVs) to provide high-quality data collection services for various applications. However, due to limited communication range, UAVs typically collect data under partial observability, hindering optimal performance without global environmental information. Additionally, many methods fail to enforce critical safety constraints. This paper proposes a communication-assisted safe multi-agent actor-critic-based UAV flight control method (CSMAAC). First, we propose an independent prediction communication partner model to address the partial observability problem. Based on the UAV’s local observation, causal inference is used to obtain prior communication information between UAVs through a feed-forward neural network to help UAVs determine potential communication partners. Second, we utilize a critic-network to predict and quantify inter-UAV influence and determine the necessity of communication. By exchanging necessary information inter-UAV, UAVs can perceive global information, thereby solving the UAV’s partial observability problem and reducing communication overhead. Moreover, we propose a similarity enhancement mechanism to improve the learning efficiency of the model by enhancing the connection between UAV observations and the policies of other UAVs. Finally, we introduce a safety layer to Actor-Network to ensure safe UAV flight. The simulation results show that the proposed method outperforms the baselines.