An Improved MapReduce Algorithm for Mining Closed Frequent Itemsets

Abstract

Mining closed frequent item sets is a key objective in the field of data mining due to its wide range of applications. Given a database of transactions, the task is to find closed subsets which appear frequently in different transactions. This subject has been studied thoroughly, and many efficient algorithms had been presented, however, most of them were designed for a non-distributed setting. The exponential growth of data in current times forces storing it in a distributed setting, meaning that most algorithms no longer apply. MapReduce is an acclaimed programming paradigm for processing large-scale, distributed data. In this paper we present an efficient algorithm for mining closed frequent item sets using the MapReduce paradigm. In addition to its novelty of running in a distributed setting, it also makes the duplication elimination step - a common step to all existing algorithms - redundant.