On Term Similarity Measures for Short Text Classification

Abstract

We study term expansion (or document expansion), which is used for classifying documents, especially for short documents such as twitter and blogs on the Web. Term expansion enables us to augment the sparse information in those short documents. Carpineto et al. have proposed a term expansion method based on FCA (Formal Concept Analysis), while Rogers et al. have proposed another term expansion method based on LDA (Latent Dirichlet Allocation). In this paper, we take the notion of weighted term similarity measures in FCA, and examine its effectiveness used for term expansion. We also study the effectiveness of some correlation measures in the field of association rule mining. We perform some experimental study on the effects of the proposed term similarity measures in term expansion using two short text corpora. The experimental results show that those weighted term similarity measures, when choosing an appropriate weight value, outperform the prior methods.