Proactive Re-replication Strategy in HDFS based Cloud Data Center

Cloud storage systems use data replication for fault tolerance, data availability and load balancing. In the presence of node failures, data blocks on the failed nodes are re-replicated to other remaining nodes in the system randomly, thus leading to workload imbalance. Balancing all the server workloads namely, re-replication workload and current running user's application workload during the re-replication phase has not been adequately addressed. With a reactive approach, re-replication can be scheduled based on current resource utilization but by the time replication kicks off, actual resource usage may have changed as resources are continuously in use. In this paper, we propose a proactive re-replication strategy that uses predicted CPU utilization, predicted disk utilization, and popularity of the replicas to perform re-replication effectively while ensuring all the server workloads are balanced. We consider both reliability of a data block and performance status of nodes in making decision for re-replication. Simulation results from synthetic workload data demonstrate that all the servers' utilization is balanced and our approach improves performance in terms of re-replication throughput and re-replication time compared to baseline Hadoop Distributed File System (HDFS). Our proactive approach maintains the balance of resource utilization and avoids the occurrence of servers' overload condition during re-replication.