pFL-SBPM: A communication-efficient personalized federated learning framework for resource-limited edge clients

Abstract

Federated learning has attracted widespread attention due to its privacy-preserving characteristic. However, in real-world scenarios, the heterogeneity of decentralized data and the limited communication resources of clients pose great challenges to the deployment of federated training. Although existing works have made great strides in dealing with heterogeneous data or compressing communication, they struggle to strike a balance between model accuracy and communication cost. To address the above issues, this paper proposes a novel federated learning framework called pFL-SBPM, which achieves communication-efficient personalized Federated Learning through Stochastic Binary Probability Masks. Specifically, we utilize probability mask optimization instead of conventional weight training, where clients obtain personalized sparse subnetworks adapted to local task requirements by cooperative optimization of probability masks in a randomly weighted network. We develop an uplink communication strategy based on stochastic binary masks and a downlink communication strategy based on binary encoding and decoding, which achieves enhanced privacy protection while dramatically reducing the communication cost. Furthermore, to effectively handle heterogeneous data while mitigating the negative impact of the introduction of stochasticity on the stability of federated training, we carefully design a soft-threshold based selective updating strategy for probability masks. The experimental results show the significant superiority and competitiveness of pFL-SBPM compared to existing baseline and state-of-the-art methods in terms of inference accuracy, communication cost, computational cost and model size.