Enabling hierarchy-aware MPI collectives in dynamically changing topologies

Hierarchy-awareness for message-passing has been around since the early 2000s with the emergence of SMP systems. Since then, many works dealt with the optimization of collective communication operations (so-called collectives) for such hierarchical topologies. However, until now, all these optimizations basically assume that the hierarchical topology remains static in a parallel program. In contrast, this paper strives for a discussion of how dynamically changing topologies can be considered during runtime, especially with focus on collective communication patterns. In doing so, the discussion starter for this is the possibility of process migration, e. g., in virtualized environments where the MPI processes are encapsulated within virtual machines. Consequently, processes originally located on distinct nodes can then (dynamically) become neighbors on the same SMP node. The central subject for the discussion on how such changes can be taken into account for optimized collectives is a new experimental MPI function that we propose and detail within this paper.