Automatic clustering by automatically generated algorithms

Abstract

Clustering data based on similarity becomes particularly challenging when the number of clusters is not known in advance. This case, known as the automatic clustering problem (ACP), corresponds to an optimization problem that aims to identify the best possible clustering among the many existing options. Although several effective ACP methods have been proposed, identifying optimal clusterings remains a difficult task, and the space of algorithmic solutions has yet to be thoroughly explored. Existing approaches suggest that better results can be achieved by appropriately combining and assembling different techniques. While some combinations have been explored, many others remain unexamined and could be evaluated through a more exhaustive exploration, such as the automatic generation of algorithms (AGA). This article considers the combinations arising from the automatic construction of algorithms for the ACP. To this end, an optimization meta-problem is defined to construct algorithms with the best computational performance. The search for the optimal solution to the meta-problem allows a computational exploration of the space defined by all possible combinations of elementary algorithmic components. We specifically explore the potential of AGA to generate ACP-specialized algorithms tailored to each dataset. Through extensive computational experiments, we evaluate the effectiveness of these specialized algorithms with general-purpose algorithms generated by AGA and six state-of-the-art ACP algorithms across well-established datasets. The results demonstrate that both AGA-generated algorithms outperform the state-of-the-art ACP algorithms, with statistically significant differences. Furthermore, the specialized algorithms exhibit superior effectiveness, highlighting their advantage over their general-purpose counterparts.