Spectrum Sharing in Cognitive UAV Networks Based on Multiagent Reinforcement Learning

Abstract

Uncrewed aerial vehicles (UAVs) have been widely used in various fields in recent years due to their affordability, mobility flexibility, and convenience. However, faced with the emergence of a large number of UAVs, the shortage of spectrum resources has become a key bottleneck that restricts the quality of service and communication efficiency of UAV networks. The cognitive radio (CR) technology can help to solve this spectrum shortage problem through spectrum-sharing technology. In order to make full use of the available spectrum resources, this article proposes a spectrum-sharing scheme based on multiagent deep reinforcement learning (DRL) in a scenario where the UAV network and terrestrial network coexist. The spectrum used by the UAVs in this scenario consists of two parts: 1) the dedicated spectrum of the UAV network and 2) the shared spectrum of the terrestrial network. The goal of our work in this article is to maximize the total throughput of the UAV network, with the maximum allowable transmission power of the UAV and the mutual interference between the UAV network and the terrestrial network as constraints. The optimization function is a mixed-integer nonconvex programming problem, DRL algorithms are an effective way to solve this problem. Therefore, we propose a multiagent DRL approach that jointly optimizes UAV signal-to-noise ratio control, power control, and access control (USPA) to effectively address this issue. Finally, by comparing with traditional algorithms, simulation results show that using the USPA algorithm can improve the effectiveness of data transmission in UAV networks.