Deep Reinforcement Learning for Energy Efficiency Maximization in SWIPT-Based Over-the-Air Federated Learning

Abstract

Federated learning (FL) is a promising solution for preserving user privacy in Internet of Things (IoT) networks thanks to its distributed computing feature. Furthermore, over-the-air FL (AirFL) can leverage the superposition property of wireless channels to achieve fast model aggregation through concurrent analog transmissions. To make AirFL sustainable for energy-constrained IoT devices, we apply simultaneous wireless information and power transfer (SWIPT) at the base station to broadcast the global model and charge local devices during the model training process. To characterize the optimality gap between the aggregated FL model and the ideal FL model brought by signal misalignment, channel fading, and random noise in the model distribution and aggregation processes, we prove the convergence of SWIPT-based AirFL to show the precise impact of up- and down-link communications on the learning performance. We formulate a long-term energy efficiency (EE) maximization problem and propose a deep reinforcement learning algorithm with a collaborative double-agent approach to optimize resource allocation strategies while guaranteeing learning performance. Numerical results demonstrate that the proposed algorithm can achieve a maximum of 41% improvement in EE under various network settings compared with benchmark schemes, and the learning performance of SWIPT-based AirFL can be improved significantly by alleviating transmission errors.