Efficient implementation of reduce-scatter in MPI

Abstract

We discuss the efficient implementation of the MPI collective operation called reduce-scatter. We describe the implementation issues and the performance characterization of two algorithms for the reduce-scatter that have been proven to be highly efficient in theory under the assumption of fully connected parallel system. A performance comparison with existing mainstream implementations of the operation is presented which confirms the practical advantage of the new algorithms. Experiments show that the two algorithms have different characteristics which make them complementary in providing a performance gain over standard algorithms.