The Uniform Tuning Problem on SQL-On-Hadoop Query Processing

Abstract

SQL-On-Hadoop systems translate a given query into several MapReduce jobs. Each job executes a different set of query operators over different input data sets, which leads to distinct resource consumption patterns. Once each job has a different resource consumption pattern they should receive tailor made tuning setup. However, SQL-On-Hadoop systems propagate the same tuning to every job in the query plan because they are not able to apply a specific tuning setup per job. Propagating the same tuning through the query plan is a problem because it drives the query to sub-optimal performance and drives tuning advisors to re-profile similar jobs several times. In our research we characterize this problem and propose a solution. Preliminary results show that our approach can reduce the number of profiles required by tuning advisors in 67% for TPC-H.