SCDFL: A Spectral Clustering-based framework for accelerating convergence in Decentralized Federated Learning

Decentralized Federated Learning (DFL) is a popular distributed machine learning framework that facilitates collaboration among multiple clients without dependence on a central server to develop a global model. This architecture faces issues with client convergence, resulting in network congestion and slower convergence during the DFL process. These challenges stem from various communications topologies and the non-independent and non-identically distributed nature of data on terminal devices in real-world scenarios, which affect both model convergence speed and overall terminal performance. Therefore, we propose SCDFL, a federated learning framework that leverages spectral clustering to efficiently and scalably handle client data heterogeneity. SCDFL introduces a novel spectral clustering strategy that focuses on grouping clients based on their characteristics. Key components include reducing the dimensionality of the client data by incremental PCA, which includes high-dimensional model updates or feature vectors, making the clustering process more efficient. Then, a similarity matrix based on the reduced data will be computed to measure client similarity. Utilizing this matrix, we apply spectral clustering to group clients with similar data characteristics. Finally, we apply the aggregation in intra-cluster and inter-cluster to the updated global model. Extensive experiments have been conducted across different topologies, and the results demonstrate that SCDFL achieves higher accuracy, faster convergence, reduced communication overhead, and improved generalization, particularly on complex datasets like MNIST, CIFAR10, and CIFAR100, while efficiently handling data heterogeneity and optimizing resource utilization across various network topologies.