Discriminative representation learning via attention-enhanced contrastive learning for short text clustering

Abstract

Contrastive learning has gained significant attention in short text clustering, yet it has an inherent drawback of mistakenly identifying samples from the same category as negatives and separating them in the feature space (i.e., the false negative separation problem). To generate discriminative representations for short text clustering, we propose a novel clustering method, called Discriminative Representation learning via Attention-Enhanced Contrastive Learning for Short Text Clustering (AECL). The AECL consists of two modules which are the contrastive learning module and the pseudo-label assisting module. Both modules utilize a sample-level attention mechanism to extract similarities between samples, based on which cross-sample features are aggregated to form a consistent representation for each sample. The contrastive learning module explores the similarity relationships and the consistent representations to form positive samples, effectively addressing the false negative separation issue, and the pseudo-label assisting module utilizes the consistent representations to produce reliable supervision information to assist the clustering task. Experimental results demonstrate that AECL outperforms state-of-the-art methods. The code is available at https://github.com/YZH0905/AECL-STC.