Deploying and monitoring hadoop MapReduce analytics on single-chip cloud computer

Abstract

Modern data analytics applications exhibit scale-out characteristics, requiring large amount of computational power. Recent research has shown that modern manycore architectures forms a promising platform solution for this emerging type of workloads. In this paper, we present a framework for the deployment, monitoring and automated exploration of Hadoop MapReduce clusters implementing data analytics applications onto the Intel SCC manycore platform. We provide an in-depth analysis on the performance and energy characteristics of Hadoop MapReduce workloads on the Intel SCC, i.e. on a real-silicon manycore which highly differentiates from typical server and accelerator architectures. Through extensive experimentation, we show that there is a trade-off between the number of worker nodes and the per-node available I/O bandwidth and that intelligently scaling the frequency of data-nodes yields in energy savings with minimal impact on performance.