DRL-based Joint Optimization for Energy Efficiency Maximization in DAV-NOMA Networks

Unmanned aerial vehicles (UAVs) can be deployed as aerial base stations or relays touring to serve ground users (GUs) chronologically. Combining with non-orthogonal multiple access (NOMA) technology, each UAV is able to serve multiple GUs at the same spectrum without causing server interference which greatly improves the spectrum efficiency. However, in the multi-UAV NOMA network, the service allocation between UAV and GU is binary variable, and there are non-convex constraints in the resource optimization problem, such as the transmission power control is integral-involved, which leads to a non-convex mixed integer nonlinear programming (MINLP) problem. The paper proposes a joint optimization algorithm based on block coordinate descent (BCD) to iteratively maximize the spectrum energy efficiency (SEE), i.e., solving the service assignment, trajectory optimization and transmission power control by K-means clustering, deterministic deep policy gradient (DDPG) and successive convex approximation (SCA), respectively. The numerical results demonstrate the validity of the proposed algorithm and the superiority to the other benchmarks in terms of the convergence speed and SEE value.