Adaptive compressed learning boosts both efficiency and utility of differentially private federated learning

Abstract

In the federated learning (FL) research field, current research is confronted with several pivotal challenges, e.g., data privacy, model utility and communication efficiency. Furthermore, these challenges are further amplified by statistical data heterogeneous in the FL system. Thus, a novel Communication-efficient and Utility-assured Gaussian differential privacy-based Personalized Federated Adaptive Compressed Learning method, called CUG-PFACL, is proposed. Specifically, an end-to-end local adaptive compressed learning strategy is designed, including three crucial modules, namely the measurement matrix, the personalized compressed data transformation and the local model. Especially, jointly training the measurement matrix module and the personalized compressed data transformation module can mitigate the inherent statistical heterogeneity while preserving all important characteristics of the compressed private data of each local client, and alleviate the additional heterogeneity induced by Gaussian differential privacy in each global communication round. Numerous experimental simulation and comparisons demonstrate that CUG-PFACL has three notable advantages: data privacy guarantee, enhanced personalized model utility and high-efficient communication.