Using latent dirichlet allocation for topic modelling in twitter

Abstract

Due to its predictive nature, Social Media has proved to be an important resource in support of the identification of trends. In Customer Relationship Management there is a need beyond trend identification which includes understanding the topics propagated through Social Networks. In this paper, we explore topic modeling by considering the techniques of Latent Dirichlet Allocation which is a generative probabilistic model for a collection of discrete data. We evaluate this technique from the perspective of classification as well as identification of noteworthy topics as it is applied to a filtered collection of Twitter messages. Experiments show that these methods are effective for the identification of sub-topics as well as to support classification within large-scale corpora.