ECRaft: A Raft Based Consensus Protocol for Highly Available and Reliable Erasure-Coded Storage Systems

Abstract

Erasure-coded redundancy is a fault-tolerant method with low-cost storage overhead. It only stores data fragments and parity fragments rather than full data across the cluster. The write process of erasure-coded data can be asynchronous or synchronous. For synchronous write process, data are encoded when written to servers. The common method doing the process needs to confirm that each coded-fragment of the data is stored in a different server to maintain the best fault tolerance. This method underperforms in terms of availability, and also fails to achieve good performance because any failure of servers will shortly disturb the write process. Some consensus protocols such as RS- Paxos and CRaft, which are based on Paxos and Raft, can solve above problems by providing fault-tolerant ability for systems. However, RS-Paxos cannot achieve the same liveness as Paxos. CRaft still adopts full data redundancy to keep the same liveness as Raft when there are not enough healthy servers. Therefore, to solve the availability problem during synchronous erasure-coded data write process, we present a novel protocol ECRaft based on Raft. It always uses erasure-coded redundancy when the ratio of erasure-coded data fragments to parity fragments is bigger than 1. It also can reach the same liveness as Raft. With state machine purge, storage redundancy can be reduced to the extent that typical erasure-coded storage systems can achieve. We build a key-value store based on ECRaft to evaluate it. In our experiments, compared with CRaft using complete-entry replication, ECRaft can save 63 % of storage, increase write throughput by 28.2 %, and reduce write latency by 19 %.