Revolutionizing textual data insights: A comprehensive review of the dual relationship between transformers and clustering in textual data analysis

Abstract

In recent years, the integration of transformer models and clustering techniques has gained significant attention in the research community. Transformers excel at feature extraction, representation learning, and understanding data, which helps improve the accuracy and efficiency of clustering tasks. Conversely, clustering methods play a critical role in managing data distribution, enhancing interpretability, and improving the training of transformer models. This review looks at the dual relationship between these two domains: how transformers can advance clustering methodologies and how clustering techniques can optimize transformer performance. By examining this interaction, the paper highlights promising directions for future research.