Semi-clustering That Scales: An Empirical Evaluation of GraphX

Abstract

GraphX is a distributed graph processing framework build on top of Spark Core. This work investigates the two questions, whether GraphX is an appropriate environment for the implementation of graph algorithms and how the computation of graph algorithms based on GraphX scales. This paper examines a graph algorithm for semi-clustering as used in social network analysis. We describe the implementation process of this algorithm beginning with a graph-oriented modeling tailored for GraphX up to an executable program. Based on our implementation, we have performed empirical evaluations regarding the scalability of our implementation and the GraphX platform. The experiments evidence that different kind of graph algorithms are supported by GraphX and that the execution of our algorithm can scale almost linearly when properly designed.