Modified K-means algorithm for automatic stimation of number of clusters using advanced visual assessment of cluster tendency

Abstract

One of the major problems in cluster analysis is the determination of the number of clusters in unlabeled data, which is a basic input for most clustering algorithms. In our paper, we investigate a new method for automatically estimating the number of clusters in unlabeled data sets, which is based on an existing algorithm for Spectral Visual Assessment of Cluster Tendency (SpecVAT) of a data set, using several common image and signal processing techniques. Its basic steps include 1) generating a VAT image of an input dissimilarity matrix, 2) Constructing Laplacian matrix 3) Normalize the rows and 4) Apply SpecVAT. Our new method is nearly “automatic,” depending on just one easy-to-set parameter. In this paper we propose direct visual validation method and divergence matrix for finding the automatic clustering. The experimental result shows that the proposed algorithm is much better than the other algorithms.