CLDA: Vertex-cut partitioning for streaming power-law graphs

Abstract

Graph partitioning is considered to be a standard solution to process large graphs efficiently when processing them on a single machine becomes inefficient due to its limited computation power and storage space. Although numerous algorithms are proposed in this area, but most of them create high computation cost and are not designed to process real-world large scale graphs as they process the whole graph prior to partitioning. Therefore, Early-stage research focuses streaming graph partitioning to reduce the computation cost by assigning the edges or vertices on-the-fly to the computing nodes. In this work, we propose “ Consider Low-degree Edges Assignment” (CLDA), a novel vertex-cut graph partitioning algorithm that exploits skewed degree distributions by explicitly taking into account vertex degree in the placement decision. This method can create partitions which are more balanced, so it would be able to reduce the computation overhead. We experimentally evaluate CLDA on real-world graphs and show that it outperforms all existing algorithms in partitioning quality.