Events Insights Extraction from Twitter Using LDA and Day-Hashtag Pooling

Abstract

News extraction from Twitter data is a hot topic. But can we extract much more than just news? The purpose of this research is to find, either news is the only information which can be extracted from Twitter data or it contains much more insights about real life events. So, we introduce a technique for analysis of Twitter's raw content. After pre-processing of tweets data, we apply hashtag pooling and extract topics using available topic modeling algorithm Latent Dirichlet Allocation (LDA) without modifying its core machinery. In the second part, estimated number of tweets per day and correlated top hashtags for each topic are calculated using day-hashtag pooling. Finally, the continues time series graph is constructed for topic analysis. Our findings show interesting results of bursty news detection, topic popularity, people's way to perceiving an event, real-life event's transition over time and before & after affects of a specific event.