QoSC: A QoS-Aware Storage Cloud Based on HDFS

Abstract

Storage QoS is a key issue for a storage cloud infrastructure. This paper presents QoSC, a QoS-aware storage cloud for storing massive data over the dynamic network, based on the Hadoop distributed file system (HDFS). QoSC employs a data redundancy policy based on recovery volumes and a QoS-aware data placement strategy. We consider the QoS of a storage node as a combination of the transfer bandwidth, the availability of service, the workload (CPU utilization), and the free storage space. We have deployed QoSC on the campus network of Zhejiang University, and have conducted a group of experiments on file storage and retrieval. The experimental results show that QoSC improves the performance of file storage and retrieval and balances the workload among DataNodes, by being aware of QoS of DataNodes.