A Methodology to Detect and Track Breaking News on Twitter

Twitter is an interesting platform for the dissemination of news. The real-time nature and brevity of the tweets are conducive to sharing of information related to important events as they unfold. But, one of the greatest challenges is to find the tweets that we can characterize as news in the ocean of tweets. In this paper, we propose a novel method for detecting and tracking breaking news from Twitter in real-time. We filter the stream of incoming tweets to remove junk tweets using a text classification algorithm. We also compare the performance of different supervised text classification algorithms for this task. We then cluster similar tweets, so that, tweets in the same cluster relate to the same real-life event and can be termed as a breaking news. Finally, we rank the news using a dynamic scoring system which also allows us to track the news over a period of time.