A MapReduce-based algorithm for parallelizing collusion detection in Hadoop

Abstract

MapReduce as a programming model for parallel data processing has been used in many open systems such as cloud computing and service-oriented computing. Collusive behavior of worker entities in MapReduce model can violate integrity concern of open systems. In this paper, a MapReduce-based algorithm for parallel collusion detection of malicious workers has been proposed. This algorithm uses a voting matrix that is represented as a list of voting values of different workers. Three phases of majority selection, correlation counting and correlation computing are designed and implemented in this paper. Preliminary results show that speedup of 1.8 and efficiency of about 70% is achieved using data set containing 2000 worker's votes.