UAV-driven task offloading and wireless power transfer: a fusion of lyapunov optimization and reinforcement learning in edge computing

Due to the flexible mobility and extensive service coverage of Unmanned aerial vehicle (UAV), UAV-assisted edge computing is increasingly emerging as a promising technology. The computational resources and battery capacity of ground Internet of Things Devices (IoTD) are often insufficient to support high-speed and stable local computing. Additionally, complex environmental conditions and electromagnetic interference severely impact the channel quality between servers and edge devices. In view of the above circumstances, this paper investigates a service architecture where aerial UAV provide task offloading and Wireless Power Transfer (WPT) to ground IoTD under time-varying wireless channel conditions. The entire system is formulated as a Markov Decision Process (MDP), and Lyapunov optimization theory is utilized to establish a reasonable objective function to maintain the stability of the system's task queue. A Lyapunov Optimized Twin Delayed Deep Deterministic Policy Gradient (LyTD3) algorithm is proposed. This algorithm aims to optimize task offloading and WPT decisions while maintaining the stability of the system’s task queue and achieving better system utility. We have verified the convergence of the proposed algorithm. Experimental results show that the algorithm model can make reasonable decisions on task offloading and WPT after training. Comparative experiments further validate the superiority of the LyTD3 algorithm, demonstrating that it outperforms other optimization methods in terms of performance and stability.