How Far Have We Come in Fault Tolerance for Distributed Graph Processing: A Quantitative Assessment of Fault Tolerance Effectiveness

Abstract

With the increase in graph dataset size and algorithm complexity, distributed graph processing runs with severe reliability problems caused by high uncertainty. A range of fault tolerance specific to distributed graph processing has been proposed. Unfortunately, current work does not consider the complexity of actual failure but only verifies the effectiveness of fault tolerance by simply killing processes or crashing compute nodes. We investigate the impact of failures on the effectiveness of three widely-used fault-tolerance mechanisms in distributed graph processing, such as checkpoint-based fault tolerance, logging-based fault tolerance, and replication-based fault tolerance, by performing fault injection based on extensive research about actual faults. Based on the above analysis, we find that failure offsets cause fault tolerance's average recovery coverage factor to drop by 0.37% to 26.77 %, and small checkpoint intervals and the confined recovery bring weak robustness of failure recovery.