Balancing the trade-off between global and personalized performance in federated learning

Abstract

Balancing the trade-off between global and personalized performance has been considered a critical problem in Federated Learning (FL). On the one hand, FL using a single global model often achieves good average performance across clients but lacks personalization, resulting in poor performance on individual client's local data. On the other hand, Personalized Federated Learning (PFL) methods may fit clients well but tend to sacrifice generalization to an uncontrollable extent. To address this issue, we propose a Generalization Preserving Personalized Federated Learning algorithm (GPPFL) to achieve a robust trade-off between global and personalized performance. In GPPFL, we first enhance the average performance of the global model by formulating a multi-objective optimization problem with a fair-driven objective for FL. We calculate a common descent direction to update the global model while simultaneously mitigating the update bias and conflicts caused by absent clients. We then design a direction-drift method to identify personalized directions, building personalized models that would not sacrifice global performance. We further demonstrate how GPPFL ensures the preservation of global performance and guarantees convergence. Extensive experiments across various scenarios verify that GPPFL outperforms state-of-the-art PFL algorithms in effectively balancing the trade-off between global and personalized performance in FL.