pm-SCAN: an I/O Efficient Structural Clustering Algorithm for Large-scale Graphs

Abstract

Most existing algorithms for graph clustering, including SCAN, are not designed to cope with large volumes of data that cannot fit in main memory. When there is not enough memory, those algorithms will incur thrashing, i.e. result in huge I/O costs. We propose an I/O-efficient algorithm for structural clustering, pm-SCAN. The main idea of our scheme is to partition a large graph into several subgraphs that can fit into main memory. We first find clusters in each subgraph, and then merge them to produce final clustering of the input graph. Experimental results show that while other existing algorithms are not scalable to the graph size, our proposed method produces scalable performance for limited memory space.