Interconnecting shared-everything systems for efficient parallel query processing

Abstract

The most debated architectures for parallel database processing are Shared Nothing (SN) and Shared Everything (SE) structures. Although SN is considered to be most scalable, it is very sensitive to the data skew problem. On the other hand, SE allows the collaborating processors to share the work load more efficiently. It, however, suffers from the limitation of the memory and disk I/O band-width. The authors present a hybrid architecture in which SE clusters are interconnected through a communication network to form a SN structure at the inter-cluster level. In this approach, processing elements are clustered into SE systems to minimize the skew effect. Each cluster, however, is kept small within the limitation of the memory and I/O technology to avoid the data access bottleneck. A generalized performance model was developed to perform sensitivity analysis for the hybrid structure, and to compare it against SE and SN organizations.<>