Design and Performance Evaluation of UCX for Tofu-D Interconnect with OpenSHMEM-UCX on Fugaku

The partitioned global address space (PGAS) model with one-sided communication has recently received attention as an easy and intuitive method for describing remote data access in nodes. PGAS can be implemented using remote direct memory access, which provides lightweight one-sided communication and low overhead synchronization semantics. In this paper, to enable portable, lightweight, and efficient one-sided communication on the Fugaku supercomputer, we designed and implemented Universal Communication X (UCX) for Tofu Interconnect D. An evaluation using OpenSHMEM-UCX and OSHMPI indicates that OpenSHMEM with UCX on Tofu Interconnect D enables smaller latency and better efficiency compared with that for OpenSHMEM with MPI and that it is beneficial for several applications based on PGAS models.