Flexible application-aware approximation for modern distributed graph processing frameworks

Abstract

The interest in the ability of processing data that has an underlying graph structure has grown in the recent past. This has led to the development of many distributed graph processing systems. However, due to rapidly growing amount of data, e.g., web graphs and social graphs, even such distributed graph processing frameworks end up requiring several minutes or even several hours to execute popular graph algorithms. This leads to the question: do we always need to know the exact answer for a large graph? The aforementioned modern distributed graph processing frameworks execute graph algorithms by exchanging messages between vertices. This paper introduces a novel message-dropping approach for approximation in these frameworks. As dropping messages would result in degradation of quality of result, our objective is to drop messages that have least adverse impact on quality. More specifically, we propose an application-aware approach that dynamically drops messages at runtime. We evaluate the effects of our approach for the PageRank algorithm on several representative real-world web graphs and compare its performance to that of existing approximation techniques for modern frameworks..