Mining Twitter in the Cloud: A Case Study

Abstract

Mining and analyzing data from social networks can be difficult because of the large amounts of data involved. Such activities are usually very expensive, as they require a lot of computational resources. With the recent success of cloud computing, data analysis is going to be more accessible due to easier access to less expensive computational resources. In this work we propose to use cloud computing services as a possible solution for analysis of large amounts of data. As a source for a large data set, we propose to use Twitter, yielding a graph with 50 million nodes and 1.8 billion edges. In this paper, we use computation of PageRank on Twitter’s social graph to investigate whether or not cloud computing, and Amazon cloud services1 in particular, can make these tasks more feasible and, as a side effect, whether or not PageRank provides a good ranking of Twitter users.