A localized dynamic load balancing strategy for highly parallel systems

Two dynamic load-balancing strategies, a local diffusion (RID) and a global exchange (DEM) strategy, designed to support massively parallel systems are presented and compared. The effects of system size and task granularity are studied. Both strategies are implemented on a 32-processor iPSC/2 and a 256-processor IBM Victor. Even for low degrees of parallelism the performance of the DEM and RID strategies is very similar. The efficiency of the DEM strategy, however, depends heavily on the system interconnection topology. Furthermore, the system sizes tested were small in the context of massively parallel systems. The overhead costs of synchronization (scale as O(N)) for the DEM approach may cause a serious deterioration of performance. The RID strategy is easily embedded into simpler topologies, and can scale gracefully for larger systems. Finally, the RID scheme is able to maintain task locality, supporting a wider variety of applications that exhibit local communication dependencies between tasks. Therefore, the RID strategy may offer a superior performance when locality is important.<>