Low-Cost Load Balancing for Parallel Particle-in-Cell Simulations with Thick Overlapping Layers

Abstract

This paper describes a parallel implementation of our practical particle-in-cell (PIC) simulator with the OhHelp dynamic load-balancing algorithm. Although the code parallelization is based on simple block domain decomposition, OhHelp accomplishes load balancing and thus the scalability in terms of the number of particles by making each computation node help another heavily loaded node. In addition to the OhHelp application, a number of additional layers overlapping with adjacent domains are newly introduced outside the boundaries of each subdomain for the purpose of minimizing overhead costs of OhHelp. The optimization can drastically reduce overhead costs for particle transfer among nodes, whereas it leads to increase in domain size which each node is responsible for. Despite this trade-off feature, the overlapping layer attachment and a further lower-level optimization exert 1.8-fold improvement of the PIC simulator performance. Consequently, the optimized simulator exhibits a good scalability and a stable efficiency in parallel executions using up to 4096 cores, showing small parallel efficiency degradation of 3% from 16- to 4096-core parallel executions.