A Novel Equidistant-Scattering-Based Cluster Index

We propose a new non-parametric internal validity index based on mutual equidistant-scattering among within-cluster data for fine-tuning the number of clusters, i.e., the hyperparameter K. Most of the validity indexes found in the literature are considered to be dependent on the number of data objects in clusters and often tend to ignore small and low-density groups. Moreover, they select suboptimal clustering solutions when the clusters are in a certain degree of overlapping or low separation. We analysed our index performance with four of the most popular validity indexes. Experiments on both synthetic and real-world data show the effectiveness and reliability of our approach to evaluate the hyperparameter K.