Resource-Efficient Federated Clustering with Past Negatives Pool

Abstract

Federated learning (FL) provides a global model over data distributed to multiple clients. However, most recent work on FL focuses on supervised learning, and a fully unsupervised federated clustering scheme has remained an open problem. In this context, Contrastive learning (CL) trains distinguishable instance embeddings without labels. However, most CL techniques are restricted to centralized data. In this work, we consider the problem of clustering data that is distributed to multiple clients using FL and CL. We propose a federated clustering framework with a novel past negatives pool (PNP) for intelligently selecting positive and negative samples for CL. PNP benefits FL and CL simultaneously, specifically, alleviating class collision for CL and reducing client-drift in FL. PNP thus provides a higher accuracy for a given constraint on the communication rounds, which makes it suitable for networks with limited communication and computation resources. Numerical results show that the resulting FedPNP scheme achieves superior performance in solving federated clustering problems on benchmark datasets including CIFAR-10 and CIFAR-100, especially in non-iid settings.