A Method for Storage Node Allocation in Erasure Code Based Storage Systems

Abstract

Fault tolerance is a major issue for all storage service providers. Currently, the storage service providers make use of data replication as a method to ensure fault tolerance. In the big data era, relying on data replication for fault tolerance reduces the storage efficiency. Most of the modern applications make use of erasure code based storage systems as an alternative to the data replication. In erasure code based storage systems, the allocation of storage nodes for storing data is to be done with care so that the load on the nodes of the storage system is always balanced. In this paper, we propose a greedy solution for the storage node allocation problem in big data environment with load balancing. Other major contributions discussed in the paper are modeling this problem with graph theory and suggesting an integer linear program formulation for the problem.