Communication-Efficient and Utility-Enhanced Local Differential Privacy-Based Personalized Federated Compressed Learning

Abstract

With the deeper and broader research on federated learning (FL), several inescapable challenges arise when putting FL into practice. However, existing research works predominately concentrate on addressing one or two challenges. This paper seeks to provide a comprehensive exploration of four fundamental issues, namely privacy, utility, communication efficiency and data heterogeneity. To simultaneously address these issues, we propose a communication-efficient and utility-enhanced local differential privacy (LDP)-based personalized federated compressed learning (FCL) method, called CUEL-PFCL. First and foremost, a general FCL framework is proposed to compress local visual data (e.g., images) while preserving data learnability, which can provide a certain degree of visual-level privacy protection and improve the communication efficiency. Subsequently, an analytically tractable Gaussian differential privacy is applied to enhance the trade-off between privacy and utility. Meanwhile, compressed sensing and SIGNSGD are respectively used to compress and quantify model gradients to further reduce the communication overhead. Besides, we keep the head representation locally to reduce communication costs, achieve the privacy amplification effect and solve the issue of data heterogeneity. Theoretical privacy analysis, experimental simulations and comprehensive comparisons all demonstrate that CUEL-PFCL has four advantages, i.e., strong privacy, enhanced utility, efficient communication and various personalized models.