Massive Data Storage Algorithm Based on Node Performance Evaluation

Abstract

In order to make massive network data storage possible in heterogeneous environment, this paper introduces node performance evaluation method and data storage node selection algorithm in HDFS data storage mechanism. First of all, this paper describes how to evaluate each node in Hadoop cluster. And then, according to the result of node performance evaluation, the appropriate nodes are selected to store data blocks when data are stored. Through this algorithm, the number of data blocks placed in each node is allocated according to the performance value of the node.