Scalable Local Community Detection with Mapreduce for Large Networks

Abstract

Community detection from complex information networks draws much attention from both academia and industry since it has many real-world applications. However, scalability of community detection algorithms over very large networks has been a major challenge. Real-world graph structures are often complicated accompanied with extremely large sizes. In this paper, we propose a MapReduce version called 3MA that parallelizes a local community identification method which uses the $M$ metric. Then we adopt an iterative expansion approach to find all the communities in the graph. Empirical results show that for large networks in the order of millions of nodes, the parallel version of the algorithm outperforms the traditional sequential approach to detect communities using the M-measure. The result shows that for local community detection, when the data is too big for the original M metric-based sequential iterative expension approach to handle, our MapReduce version 3MA can finish in a reasonable time.