A simple cluster-scaling policy for MapReduce clouds

Abstract

Due to the rise of cloud computing, many cloud services have been developed. Google proposed a programming model called MapReduce for processing large amounts of data. After YAHOO! proposed Hadoop, many companies and enterprises have started using this programming model to establish their own cluster for handling large amounts of data. Computing resources within a cluster are often not all be used. Therefore, many researches about cluster-scaling are presented. These studies were proposed to reduce the size of the cluster to achieve power saving or to add more computing nodes in order to obtain better performance. However, there is always a trade-off between performance and power saving. Therefore, taking both performance and energy saving into account, we propose a simple policy which can effectively identify how many computing nodes can be inactivated from a cluster without affecting the execution time. We evaluate our policy in many cases to prove that it is well-performed in different configurations and achieves performance and power saving both.