Spectral Contrastive Clustering

Abstract

We combine online spectral clustering and contrastive representation learning into a novel deep clustering algorithm that can be used for unsupervised image classification. We estimate a spectral embedding using minibatches. Spectral cluster assignments are used by a pairwise contrastive loss to update the model’s latent space, allowing our spectral embedding to adapt over time. We obtain competitive unsupervised classification performance purely by applying K-Means to our spectral embedding. Unlike competing methods, our approach does not require strong augmentations, class-balancing penalties, offline example mining or softmax classifiers.