A novel clustering technique for short texts

Abstract

We describe a novel clustering technique for clustering short texts, such as URLs, without enriching it with the help of external knowledge sources. Our technique first performs feature clustering to identify the key features of the dataset and then reconstructs the dataset on the basis of the key features. Then, it computes the similarity of the short texts belonging to the reconstructed dataset using similarity measures such as Jaccard, Cosine and Dice measures. Finally, it performs short text clustering using Spectral Clustering. We compare our method with conventional Spectral Clustering method which runs directly on the original short text dataset. We performed experiments on a subset of ODP dataset as well as WebKB dataset. The empirical results demonstrate an improvement of 21% in terms of accuracy over the Spectral Clustering method for ODP dataset and 29.2% for the WebKB dataset.