HRaft: Adaptive Erasure Coded Data Maintenance for Consensus in Distributed Networks

Abstract

Distributed data services usually rely on consensus protocols like Paxos and Raft to provide fault-tolerance and data consistency across global and local-distributed data centers. Erasure coding replication has appealing storage and network cost saving compared with full copy replication, which helps consensus protocols achieve low latency, high fault tolerance, and high throughput for data access. Applying erasure coding in consensus protocols directly will degrade the liveness level when the number of failure servers reaches a certain level. To address the challenge, CRaft just stores full copy replication instead of erasure coding replication when the number of failed servers reaches a certain threshold. In such situation, CRaft will be downgraded sharply to the same storage and network costs as Raft. To overcome the shortcoming of CRaft, we propose a protocol, called HRaft, which can adapt the placement of data blocks in order to always have enough blocks to recover the stored value when servers fail. By replenishing some coded blocks in healthy servers instead of full copy replication, it can avoid switching to the full replication when a certain threshold on the number of failures is reached. We designed and implemented a key-value (KV) storage prototype to validate the proposed protocol and evaluate its performance. The experimental results show HRaft can significantly reduce storage and network costs and improve write performance while keeping the liveness level compared to CRaft.