Sparsified Random Partial Model Update for Personalized Federated Learning

Abstract

Federated Learning (FL) stands as a privacy-preserving machine learning paradigm that enables collaborative training of a global model across multiple clients. However, the practical implementation of FL models often confronts challenges arising from data heterogeneity and limited communication resources. To address the aforementioned issues simultaneously, we develop a Sparsified Random Partial Update framework for personalized Federated Learning (SRP-pFed), which builds upon the foundation of dynamic partial model updates. Specifically, we decouple the local model into personal and shared parts to achieve personalization. For each client, the ratio of its personal part associated with the local model, referred to as the update rate, is regularly renewed over the training procedure via a random walk process endowed with reinforced memory. In each global iteration, clients are clustered into different groups where the ones in the same group share a common update rate. Benefiting from such design, SRP-pFed realizes model personalization while substantially reducing communication costs in the uplink transmissions. We conduct extensive experiments on various training tasks with diverse heterogeneous data settings. The results demonstrate that the SRP-pFed consistently outperforms the state-of-the-art methods in test accuracy and communication efficiency.