A robust hierarchical clustering algorithm for automatic identification of clusters

Aggregation-based hierarchical clustering algorithms are widely used in data analysis due to their robust clustering performance. Although some existing hierarchical clustering methods can identify the number of clusters in a dataset, most are only effective for well-separated clusters and struggle to identify the number of clusters in complex datasets, particularly non-convex noisy datasets. To address these shortcomings, this paper proposes a robust hierarchical clustering algorithm for automatic identification of clusters(RHCAIC), which can identify the optimal number of clusters while providing reliable clustering results. To reduce the impact of noise in clustering, the method first calculates reverse density and designs a dynamic noise discriminator to denoise the dataset. Based on the fact that more similar points have a higher probability of being clustered into the same cluster among multiple results of hierarchical clustering, a robust solution was designed. After constructing a directed graph using the kNN algorithm, the graph merging process is performed by iteratively traversing the directed edges. During this process, the number of clusters is identified, and the clustering results of the denoised dataset are obtained. Finally, by incorporating density information into the noise clustering, the final clustering results are obtained. A series of experiments conducted on 12 synthetic datasets and 8 real datasets demonstrate that, compared to seven other benchmark algorithms, the RHCAIC algorithm not only accurately identifies the number of clusters in the dataset but also produces better clustering results.