Determining the optimal number of clusters using a new evolutionary algorithm

Abstract

Estimating the optimal number of clusters for a dataset is one of the most essential issues in cluster analysis. An improper preselection for the number of clusters might easily lead to bad clustering outcome. In this paper, we propose a new evolutionary algorithm to address this issue. Specifically, the proposed evolutionary algorithm defines a new entropy-based fitness function, and three new genetic operators for splitting, merging, and removing clusters. Empirical evaluations using the synthetic dataset and an existing benchmark show that the proposed evolutionary algorithm can exactly estimate the optimal number of clusters for a set of data