Coupling GPU and MPTCP to improve Hadoop/MapReduce performance

Abstract

Apache Hadoop is the famous open source cloud computing software in recent years, the performance is much better than before due to lots of researchers' efforts, but its performance still has chance to be improved further because of unsatisfied distributed computing speed and slow response time from heterogeneous Internet's uncertain and dynamic environment. In this paper, coupling emerging GPU computing with multi-path TCP (MPTCP) protocol is proposed for current Hadoop/MapReduce architecture to further improve the distributed computing performance. We use GPU computing to speed up the Map's process, and use MPTCP to reduce Reduce's data transfer time. The Hadoop benchmark applications such as Terasort, WordCount and PiEstimate are applied to demonstrate the improved performance of our proposed scheme. According to the preliminary experimental results, the proposed scheme can improve the Hadoop/MapReduce performance by coupling GPU computing and MPTCP multipath protocol with robustness and bandwidth aggregation, to reduce further the distributed computing latency.