Repair friendly wide-stripe erasure coding for in-memory key–value stores

Abstract

Erasure coding is a low-cost redundancy mechanism over replication for in-memory key–value stores by storing stripes of data and parity chunks. Wide stripes are recently proposed to suppress the fraction of parity chunks in a stripe for extreme storage savings. However, wide stripes aggravate the repair penalty, and existing repair-efficient approaches for erasure coding fail to address this effectively.

We investigate two representative approaches, locally repairable codes (LRCs) and regenerating codes (RGCs), which reduce repair overhead in distributed storage. Building on this foundation, we propose two systematic solutions to address the repair penalty in wide-stripe erasure coding: (i) Combine Locality (CL), a cross-rack data placement mechanism for Azure-LRC combining parity and topological locality, (ii) Repair-Contribution-Based Placement (RCBP), a cross-rack data placement mechanism for Hitchhiker codes (a subclass of RGCs), strategically co-locating chunks with high repair contributions in the same rack. We apply CL-based LRC and RCBP-based Hitchhiker codes to in-memory key–value (KV) stores, augmenting them with efficient encoding and update schemes. Experiments on Amazon EC2 show that CL reduces the single-chunk repair time by up to 90.5% compared to locality-based state-of-the-art. RCBP improves repair throughput by up to 19.3% and 44.6% compared to original Hitchhiker and RS codes, respectively.