Learning-based Dynamic 3D Placement of Multiple Aerial Base Stations with Heterogeneous Traffic

Deployment of Unmanned Aerial Vehicles (UAVs) as aerial base stations (ABSs) has emerged as a promising solution to enable high-quality services in next-generation mobile networks. One key challenge is how to place the UAV-mounted Base Stations (UAV-BSs) dynamically over time in response to changes in network situations. Conventional methods usually utilize location information to place the UAV-BSs at the centroid of all users to maximize the average downlink rate. This approach might become ineffective when the traffic demand is highly heterogeneous among users, causing the spatial distribution of traffic demand not to correlate with the distribution of users. This paper considers a time-evolving multi-user multi-server system with heterogeneous traffic, where a group of UAV-BSs provides different services to ground mobile users. We formulate a long-term maximization problem of the user’s time-average download speed with the queue stability constraint under random movements of ground users. An actor-critic framework of deep reinforcement learning is proposed to position multiple UAV-BSs efficiently based on a comprehensive view of the network situation, including spatial statistics of the users, traffic demand, and the network’s downlink rate. The framework’s core module is a deep neural network (DNN) that obtains the UAV’s location and three encoded heatmaps of network statistics to predict the optimal movement for UAV-BSs. Extensive simulations demonstrate the guaranteed convergence and great potential of the proposed method regarding the backlog queue length and the average download speed of users.