Ad Hoc Aggregation Query Processing Algorithms Based on Bit-Store in Data Intensive Cloud

Abstract

Ad-hoc Aggregation query is extremely important for data-intensive applications in the cloud which extracts valuable summary information on massive datasets to help decision-maker make right decisions. Current data storage schemes (row-store and column-store) cannot efficiently answer ad-hoc aggregation on massive data sets in the cloud. A new data storage structure (bit vector storage structure, bit-store for short) is proposed in the paper, which partitions tables vertically by bit position and stores all bit values in the same positions into a separate bit file. This paper focuses on proposing ad-hoc aggregation query algorithms based on bit-store. Firstly, the storage model of bit-store including its attribute encoding and bit file organization is introduced. Then, the implementation of different aggregation operations using different encoding schemes is presented. Finally, analytical and experimental results show the effectiveness and efficiency of the proposed approach.