Incremental mining of association patterns on compressed data

Abstract

Introducing data compression concept to large databases has been proposed for many years. In this project, we propose a new algorithm for the compression of large databases. Our goal is to optimize the I/O effort for finding association rules. The algorithm partitions the databases into two parts and all transactions will be compressed with the help of a reference transaction found in the small partition. We also compared the proposed compression algorithms with a normal compression algorithm - the binary compression. Empirical evaluation shows that the proposed algorithm performs well both in reducing the storage space and the I/O process required to find the large item sets for association rules.