Marbor: A novel large-scale graph data storage and processing framework

Abstract

In this paper, we propose Marbor, a novel graph data processing framework to analyze the large-scale data in social network services. It develops an efficient graph organization model to minimize the costs of graph data accesses and reduce the memory consumption. In addition, we present a novel control message method in Marbor to improve the synchronization iterations performance. During the graph data processing, in each iteration, it analyzes the relationships among tasks and forwards the tasks to the next iteration with control messages, so no synchronization operations are used. We compare Marbor with other graph processing methods on several large-scale real world SNS datasets with two widely used applications, and the results show that Marbor outperforms the current mechanisms.