Efficient gather operation in heterogeneous cluster systems

Abstract

A heterogeneous cluster system consisting of different types of workstations and communication links plays an important role in parallel computing. In many applications on the system, collective communication operations are commonly used as communication primitives. Thus, design of the efficient collective communication operations is the key to achieve high-performance parallel computing. But the heterogeneity of the system complicates the design. In this paper, we consider design of an efficient gather operation, one of the most important collective operations. We show that an optimal gather schedule is found in O(n/sup 2k-1/) time for the heterogeneous cluster system with n processors of k distinct types, and that a nearly-optimal schedule is found in O(n) time if k = 2.