Clustering tendency assessment for datasets having inter-cluster density variations

Abstract

Clustering tendency assessment, i.e., determining if a dataset has any inherent clusters, and if so, how many clusters, k to seek is a crucial pre-clustering task. The visual assessment of tendency (VAT) and improved visual assessment of tendency (iVAT) algorithms provide a visual way to assess cluster tendency of a dataset by reordering the pairwise dissimilarity matrix so that potential clusters are displayed as dark blocks along the diagonal in the image of the reordered dissimilarity matrix. VAT and iVAT, being distance-based schemes, fail to perform well for datasets consisting of clusters characterized by different density levels. In this paper, we introduce two new members of the VAT family of algorithms: Locally Scaled VAT (LSVAT) and Locally Scaled iVAT (LS-iVAT), which produces better iVAT images for data having inter-cluster density variations. Numerical experiments comparing the proposed novel approach with baseline VAT/iVAT as well as spectral clustering and density-based clustering algorithms establish that LS-VAT and LS-iVAT are superior to the comparable algorithms in terms of clustering quality.