Empirical Analysis of MapReduce Job Scheduling with respect to Energy Consumption of Clusters

Abstract

The growing popularity of Cloud Computing has led to an increasing number of applications using MapReduce in cloud data centers. MapReduce workloads are mostly interactive workloads or batch processing workloads that consume an enormous amount of energy when hosted on multiple clusters. Currently, the scheduling of workloads is done with the sole goal of the faster execution time of jobs. However, in doing so, there is wastage of energy as the same jobs could be completed within the stipulated Service Level Agreement (SLA) using fewer resources when hosted on small clusters. Therefore, in order to minimize the energy consumption of MapReduce clusters, workloads could be deployed on a minimum number of clusters depending on the type and size with the goal of minimizing energy consumption and not faster response time. In this work, the three schedulers i.e FIFO, Fair and Capacity schedulers are compared with respect to energy efficiency on small-scale and large-scale workloads. Experiments performed on a small cluster using these workloads show significant energy savings with respect to Capacity scheduler compared to other schedulers.