UAV-Enabled Energy-Efficient Aerial Computing: A Federated Deep Reinforcement Learning Approach

Abstract

Aerial computing paradigms, particularly those involving UAVs as access points and radio towers, show significant promise for local data analysis and real-time service provision in aerial access networks. However, the limited battery life of UAVs, compounded by the high-energy demands of communication tasks and prolonged computation delays, presents a significant challenge. Deep reinforcement learning (DRL) enables UAVs to autonomously optimize their operations, reducing both energy consumption and latency. Nevertheless, the prolonged learning process of DRL can lead to inefficiencies, especially in dynamic environments where the equitable participation of UAVs is crucial. To address these issues, we introduce a fairness-oriented federated learning (FL) scheme that employs importance sampling to select UAVs for training, ensuring equitable utilization of each UAV's data. Furthermore, we integrate this FL fairness scheme into the design of DRL algorithms, termed FedDRL. This algorithm jointly optimizes the computation capabilities and bandwidth allocation of UAVs to minimize system costs. Numerical results demonstrate the fairness of FedDRL in fFL networks. Specifically, compared to other state-of-the-art DRL algorithms (e.g., TD3 and DDPG), FedDRL reduces system costs by 43.82% and 49.45%, respectively.