GPS: Towards Simplified Communication on SGL Model

Abstract

Parallel programming and data-parallel algorithms have been the main techniques supporting high-performance computing for many decades. A major conceptual step was taken by L. Valiant who introduced the Bulk-Synchronous Parallel (BSP) model. Parallel algorithms on BSP can be designed and measured by taking into account not only the classical balance between time and parallel space but also communication and synchronization. Inspired by BSP, the SGL bridging model was proposed in order to improve the simplicity of parallel program development, the portability of parallel program code, and the precision of parallel algorithm performance prediction on both classical parallel machines and novel hierarchical machines. The programming model of SGL replaces the BSML (BSP-OCaml) programming primitives with scatter, gather and pardo. However, SGL does not express "horizontal" communication patterns. In this paper we introduce the GPS theorem which can be implemented later in a compiler to optimize the SGL's "horizontal" all-to-all communication. We then propose a simplified version of BSML's put based on GPS and implement Tiskin-McColl parallel sample-sort with it. The comparison of BSML's put and SGL's GPS shows that GPS has a better code readability and lower execution time.