Column-based RLE in row-oriented database

Abstract

In database systems, disk I/O performance is usually the bottleneck of the whole query processing. Among many techniques, compression is one of the most important ones to reduce disk accesses so to improve system performance. RLE (Run-Length Encoding) is one light-weight compression algorithm which incurs negligible CPU cost. A lot of work show that, although RLE is one of the most effective compression techniques in column-oriented systems, it is very hard to use due to bad value locality in row-oriented systems where values from multiple attributes are stored in the same page. We propose CRLE (Column-based RLE), one compression algorithm to apply RLE to row-oriented data storage. On row-oriented storage page, CRLE can exploit value locality in individual column and encode values from the same column in run-length format. Experiments show that CRLE can lead to very good compression ratio and performance in spite of row-oriented data storage.