Multiplexing endpoints of HCA to achieve scalability for MPI applications: design, implementation and performance evaluation with uDAPL

With an ever increasing demand for computing power, number of nodes to be deployed in a cluster based supercomputer is increasing. Limited hardware resources such as Endpoints on HCA of a high speed interconnect limit the scalability of a parallel application based on MPI that sets up reliable connections between every process pair using endpoints, prior to communication. In this paper, we propose a novel approach of multiplexing endpoints to extend scalability. We discuss critical design issues in connection management and data transfer routines with the multiplexing technique. Using this approach, we are able to scale up MPI applications with nearly equal or better performance with the same HCA.