Event detection from Twitter data

Abstract

Event detection from Twitter is important for people to extract valuable information about real world events. Automation of this task is challenging due to short and noisy nature of microblogging data. Topic modeling algorithms such as Latent Dirichlet Allocation (LDA) is the most popular algorithm to extract topics from news articles but not suitable for microblogging content due to the data sparsity problem. In this paper, we proposed a method to handle data sparsity problem that makes LDA topic model suitable for Twitter data by considering super tweet (aggregation of similar tweets) as a document instead of single tweet without modifying internal structure of model. Extensive experiments on real-time twitter data show that our approach outperforms the baseline approaches.