Which Feature is Better? TF*IDF Feature or Topic Feature in Text Clustering

Abstract

In this paper, we conduct a comparative study on two different text features in text corpus clustering: TF*IDF feature and Topic feature. The former is mainly used in similarity-based text corpus clustering methods, while the latter, which is produced by LDA model, is used to identify the topics of texts. We conduct clustering experiments on 20-newsgroups (20NG) datasets. Based on the dataset, two typical text clustering methods are respectively employed to compare the clustering performance of the above two text features. The experiments demonstrate if the optimal topic number is chosen, the topic feature outperforms in the clustering accuracy.