Computation Capability Deduction Architecture for MapReduce on Cloud Computing

Abstract

MapReduce gradually becomes the de facto programming standard of applications on cloud computing. However, MapReduce needs a cloud administrator to manually configure parameters of the run-time system such as slot numbers for Map and Reduce tasks in order to get the best performance. Because the manual configuration has a risk of performance degradation, MapReduce should utilize the Computation Capability Deduction Architecture (CCDA) proposed in this paper to avoid the risk. MapReduce can use CCDA to help the run-time system to distribute appropriate numbers of tasks over computers in a cloud at run time without any manual configuration made by a cloud administrator. According to experiment observations in this paper, MapReduce can get great performance improvement with the help of CCDA in data-intensive applications such as Inverted Index and Word Count that are usually required to process big data on cloud computing.