Robust Federated Fuzzy C-Means Algorithm in Heterogeneous Scenarios

Abstract

The federated fuzzy C-means (federated FCM) extends the traditional Fuzzy C-means (FCM) to the federated learning (FL) scenario, aiming to address the data privacy preservation issue of soft clustering in distributed environments. However, a significant challenge persists with existing federated FCM algorithms, i.e., they struggle to converge effectively in complex heterogeneous scenarios, leading to unstable clustering outcomes. Here the complex heterogeneous scenarios stem from the combination of nonindependently and identically distributed (non-IID) data across different clients (statistical heterogeneity), coupled with the involvement of only some clients in each iteration (systematic heterogeneity). While prior research has attempted to address the impact of statistical heterogeneity in FL scenarios, it has overlooked the issue of system heterogeneity. In response, this article proposes a novel federated FCM algorithm (SC-FFCM) that remains robust even in such complex heterogeneous scenarios. First, the client-side clustering module of SC-FFCM adopts a gradient-based FCM algorithm, facilitating corrections to the direction of local optimization. Second, the algorithm introduces a control variates technique to rectify update bias during the iteration process, thereby mitigating the adverse effects of random client sampling and non-IID data distribution on the algorithm convergence. Finally, the proposed algorithm approximates the ideal federated FCM algorithm. Experimental studies verify the effectiveness of the proposed method.