Three-dimensional trajectory optimization of rotary-wing UAV with cellular network connectivity based on modified DDPG

Abstract

The integration of cellular network and unmanned aerial vehicles (UAVs) plays a critical role in the development of remote sensing and intelligent monitoring technologies. However, due to the limited onboard energy and the down-tilt characteristics of cellular base station (BS) antennas, UAVs navigating over urban areas still face practical challenges. By investigating the trade-off between UAV flight time and expected interruption time, this paper proposes a deep reinforcement learning (DRL) based joint optimization algorithm for UAV three-dimensional (3D) spatial cruising in dense urban areas. The algorithm enables the UAV to determine an optimal trajectory that navigates through designated waypoints within the cruising space while ensuring the completion of the journey under predefined energy constraints. Unlike traditional discretized trajectory optimization methods, our approach employs a deep deterministic policy gradient (DDPG) network to enable fully continuous and omnidirectional action selection, allowing the UAV to navigate more efficiently while avoiding low-coverage areas. Moreover, the algorithm is further modified through the incorporation of a prioritized experience replay (PER) mechanism and N-step learning method, aimed at enhancing overall performance. Numerical results verify that our proposed method significantly outperforms benchmark algorithms in connectivity-aware UAV path planning, demonstrating clear advantages in achieving robust and reliable aerial communication coverage in dynamic 3D environments.