Efficient disk read and recovery cost reduction approach in heterogeneous liberation-coded storage systems

Abstract

Distributed storage clusters provide large-scale data storage solutions; however they often experience node failures. Erasure codes are widely used to ensure data reliability while maintaining low storage overhead. However, during the recovery process, the high disk reads and network traffic associated with erasure codes can prolong recovery time and increase the risk of data loss. Current solutions that focus exclusively on reducing data reads to expedite recovery are often less effective in real-world network environments. This paper addresses the recovery problem in scenarios where storage nodes exhibit heterogeneity in network bandwidth. We assign recovery cost to each storage node based on its network bandwidth and propose the Recovery Cost Optimization for Heterogeneous Storage (RCOHS), a heterogeneous recovery method for Liberation-coded systems that minimizes data downloads while keeping low recovery cost. RCOHS incorporates the SearchCostOptSeq algorithm, which employs cyclic condition theory to refine the solution space. It determines the lowest-cost solution among all disk-read optimal options, in conjunction with the OptSeqRecov algorithm, which reconstructs failure symbols in the correct order using this solution. We conducted extensive experiments on Amazon EC2, and the results show that RCOHS reduces recovery time by an average of 31.2 % compared to the traditional method of RFPD and 8.4 % over the state-of-the-art technique, DROR.