3D UAV path planning based on an improved TD3 deep reinforcement learning for data collection in an urban environment

Abstract

With the rapid growth in the number of users and services in communication networks, unmanned aerial vehicles (UAVs) are expected to play a significant role in future wireless communication systems. One of the key applications of UAVs is data collection in Internet of Things (IoT) networks. This paper addresses a three-dimensional (3D) UAV path planning optimization problem aimed at minimizing the completion time of data collection in urban environments, taking into account real-world constraints such as frequent communication link blockages between UAVs and sensors caused by buildings. To tackle this challenge, we propose an improved Deep Reinforcement Learning (DRL) algorithm, referred to as the Dropout-Based Prioritized TD3 Algorithm (DPTD3). This method integrates the TD3 algorithm with the Prioritized Experience Replay Buffer (PER) strategy and introduces a new Actor network architecture incorporating the Dropout technique. Simulation results demonstrate that the proposed 3D UAV path planning approach reduces both data collection time and UAV energy consumption compared to a two-dimensional (2D) path planning method. Furthermore, the results indicate that during training, the DPTD3 algorithm outperforms other state-of-the-art DRL approaches in terms of both stability and performance.