A practical framework for real-time diffusion analysis in social media

In a microblogging service such as Twitter, timely knowledge about what kinds of information are diffusing in social media is quite important for companies. It is also effective to identify the influential users who are retweeted frequently by many users. We are now developing an information diffusion analysis system that enables real-time analysis of streaming social data. However, streaming data is usually divided into segments called windows. The window size is decided by the amount of data or a length of time. This means that we have to use fragmented diffusion data for our diffusion analysis. We propose a customized time-window model by effectively estimating diffusion extinction, which enables an early decision to remove stale data from in-memory data store. We evaluate our implementation in terms of both the efficiency of query processing and effectiveness of our time-window model.