FedDDB: Clustered Federated Learning based on Data Distribution Difference

Abstract

Clustered federated learning is a federated learning method based on multi-task learning. It groups similar clients into the same clusters and shares model parameters to solve the problem that the joint model is trapped in local optima on Non-IID data. Most of the existing clustered federated learning methods are based on the difference of model parameters for clients clustering. During the client model training process, the model parameters are biased and the clustering result is affected due to insufficient samples and missing eigenvalues in the dataset. In this paper, we develop a clustered federated learning method based on data distribution difference (FedDDB) in the dataset level. The method in this paper focuses on the distribution of label probability and eigenvalues, analyzes the difference of data distribution difference between clients and measures the distance between datasets which is used for client clustering. Every cluster will be trained independently and in parallel on the cluster center model. At the beginning of each round of training, the client clustering process needs to be repeated. We conduct relevant experiments and demonstrate the effectiveness of our method.