Achieving Natural Clustering by Validating Results of Iterative Evolutionary Clustering Approach

Abstract

Clustering is an essential process that leads to the classification of a given set of instances based on user-specified criteria; and different factors may lead to different clustering results. Thus, a large number of clustering algorithms exist to satisfy different purposes. However, scalability and the fact that algorithms in general need the number of clusters be specified a priori, which is mostly hard to estimate even for domain experts, are two challenges that motivate the development of new algorithms. This paper presents a novel approach to handle these two issues. We mainly developed a clustering method that works as an iterative approach to handle the scalability problem; and we utilize multi-objective genetic algorithm combined with validity indexes to decide on the number of clusters. The basic idea is to partition the dataset first; then cluster each partition separately. Finally, each obtained cluster is treated as a single instance (represented by its centroid) and a conquer process is performed to get the final clustering of the complete dataset. Test results on one large real dataset demonstrate the applicability and effectiveness of the proposed approach