SABO: Dynamic MPI+OpenMP Resource Balancer

Abstract

Scientific parallel applications often use MPI for inter-node communications and OpenMP for intra-node orches-tration. Parallel applications such as particle transport, seismic wave propagation simulator, or Finite-Element applications often exhibit workload imbalance due to their nature of ongoing data movement. These applications usually develop software balancing strategies triggered when some imbalance thresholds are detected to reduce this imbalance. These developments are complex to implement and impact the entire distributed applications' performance by synchronizing and exchanging the load over the network. This paper proposes a method to dynamically detect load imbalance and balance the computation by redistributing OpenMP threads between MPI processes local to the node. With minimal impact on the applications' codes, we demonstrate how this technique can improve the overall applications' performance up to 28% on MiniFE, 17 % on Quicksilver, and 3 % on Ondes3D. We also present its impact when executing multiple nodes and our proposed approach's limitations.