Performance Analysis of Hadoop for Query Processing

Abstract

Query processing using mostly various NoSQL languages becomes a significant application area for Hadoop. Despite significant work on performance improvement of these languages the performance dependence on basic configuration parameters seems not to be fully considered. In this paper we present a relatively comprehensive study into influence the basic configuration parameters have on performance of typical types of queries. We choose three queries from Lehigh University Benchmark that can represent the most typical challenges and we analyze their dependence on parameters such as: dataset size, number of nodes, number of reducers and loading overhead. The results indicate strong dependence on the amount of reducers and IO performance of the cluster, which proves the common opinion that MapReduce is IO bound. These results can help to compare performance behavior of different languages and serve as a basis for understanding the influence of configuration parameters on the final performance.