Distributed MapReduce framework using distributed hash table

Abstract

In past years, Cloud computing is gained more attention in industry and academic area. The advance technologies are needed to match the demand of the development of cloud computing. MapReduce is one of the enabling technology. MapReduce is a programming model supporting parallel computation especially for data-intensive cloud computing applications. However, MapReduce needs a master node to coordinate the execution of the parallel computation. This may cause communication bottleneck and single point of failure error. Therefore, in this paper we propose a distributed MapReduce framework based on Distributed Hash Tables to support large scale cloud computing applications.