Performance of the pipelined hash-join algorithm in a heterogeneous distributed environment

Abstract

A pipelined distributed parallel hash-join algorithm is executed in a distributed heterogeneous supercomputing environment which consists of the Connection Machine CM2, and the Cray C90. This algorithm implements the computationally intensive join operation of relational databases. The hash and join phases of the algorithm are executed on the architectures determined to be best suited for them. The hash phase of the algorithm is implemented on the Cray C90. The hashed data sets of the first join relation are transmitted from the Cray to the CM2. A pipeline is established between the two machines as the Cray continues to hash each page of the second join relation and transmits it to the CM2 where the join is performed. Limited improvements in performance of the pipelined algorithm for different combinations of data sizes, data distributions, and join sizes is analyzed and the limitations of the distributed environment are discussed.