A result correctness verification mechanism for cloud computing based on MapReduce

Abstract

MapReduce is widely applied as a parallel programming model to process massive amounts of data in cloud computing environment. However, in open systems, the workers of MapReduce framework are provided with various administration domains that may be unreliable or malicious. The existing schemes of MapReduce processing model based on multiply duplicate tasks can effectively detect the lazy and non-collusive workers. However, they can not cope with the vulnerability that malicious workers collude to return incorrect results and thereby undermine the final computation results of users' outsourced tasks. In this paper, we present an effective result correctness verification mechanism for MapReduce in public cloud computing environment. By using task duplication and weighted correctness attestation graph, our mechanism can effectively detect both non-collusive and collusive malicious workers in public cloud environment. In order to further improve the detection speed, we introduce a workers' selection method based on trust values and consistency relationship. Finally, we conduct the analysis and experimental evaluation, and the results indicate that our mechanism can guarantee higher detection rate with proper additional computation overhead.