Communication-Learning Co-Design for Differentially Private Over-the-Air Federated Learning With Device Sampling

Abstract

Recent years have witnessed the development of federated learning (FL) that allows wireless devices (WDs) to collaboratively learn a global model under the coordination of a parameter server without sharing their local datasets. To meet the communication efficiency and privacy requirements, over-the-air computation and differential privacy (DP) have been incorporated in FL by leveraging the signal-superposition property of multiple-access channels and using artificial noises to perturb local model updates, thereby preserving DP. In this paper, we propose an exploration into device sampling with replacement as a potential mechanism for augmenting the DP levels of WDs in over-the-air FL. In particular, we delve into the joint optimization of device sampling strategy, the number of training rounds, and over-the-air transceiver design. Our goal is to maximize the learning performance while ensuring each WD meets the DP requirement. The problem is challenging due to the intractable FL convergence rate and privacy losses under random sampling, coupled with the strong interconnection among mixed continuous and integer decision variables. To tackle this problem, we first analyze the learning convergence rate and privacy losses of WDs. The analysis allows us to derive the optimal transceiver design per round in closed forms. Then, we propose an efficient alternating optimization algorithm by deriving the optimal device sampling strategy and the number of training rounds in semi-closed forms. Our numerical results, based on real-world learning tasks, showcase the effectiveness of our proposed approach compared with representative baselines.