On the use of bit filters in shared nothing partitioned systems

Abstract

Parallel query processing is in the core of many business analysis environments. Such applications impose a high demand on the computer hardware to achieve results in reasonable times, specially when queries are launched against huge amounts of warehouse data. We look into the problem of parallel query processing on large data sets focusing on a rational use of the network and memory resources. In this context, we propose a new protocol to make use of bit filters in parallel shared nothing systems for non-collocated joins. We call our protocol remote bit filters with requests (RBF/sub R/). We have implemented a prototype of RBF/sub R/ for the first time in a major commercial database, IBM/spl reg/ DB2 Universal Database/spl trade/(DB2 UDB). RBF/sub R/ has two important advantages over the previous usage of bit filters in the same context. First, it reduces the amount of memory used compared to previous solutions. This allows for the processing of more or larger queries. Second, the protocol itself has an insignificant impact on communication. This means that it is as efficient as the previous strategies, avoiding the saturation of the network in parallel intensive network usage environments.