Toward customized model discrepancies in personalized federated learning on non-IID data

Abstract

Federated learning (FL) is a traditional framework comprising a central server and multiple local clients. In FL, a shared global model is trained for resource-constrained computing devices while preserving data privacy. However, in certain practical applications, the shared global model may exhibit poor inference performance in local clients owing to nonindependent and nonidentically distributed (non-IID) characteristics of data. To address this issue, researchers have proposed personalized FL (PFL), which involves learning a customized model for each client to mitigate the impact of weight divergences when the training datasets are non-IID. Unfortunately, existing studies fail to reveal the inherent connection between model discrepancies and non-IID data. Herein, we focus on demonstrating the relationship between weight divergences among customized models and non-IID data, and we provide a proposition to reveal the root cause of such divergences. Additionally, based on our theoretical analysis, we introduce two novel personalized FL methods, namely, PFL with neighbor clients (PFedNC) and PFL with neighbor layers (PFedNL), to address the issue of non-IID data scenarios. Theoretical convergence analysis and extensive experiments indicate that our proposed methods outperform state-of-the-art personalized algorithms in non-IID scenarios. Specifically, PFedNC achieves up to 4 % improvement in customized model accuracy, while PFedNL yields 8 %–10 % gains over multiple baselines.