Improving the performance of Dragonfly networks through restrictive Proxy routing strategies

Abstract

Dragonfly has become the network of choice for large-scale high-performance computing systems and, indeed, it dominates the top positions of supercomputer rankings. The reason for this is that it offers a sweet spot in terms of cost, simplicity, performance, fault-tolerance and power consumption. In this work, we propose a collection of routing strategies which restrict proxies to be adjacent to either the local or the remote router. This way, it features shorter paths than the standard Valiant routing. We carry out an extensive simulation-based evaluation to assess their performance. Our experiments found latency reductions of up to 76% and throughput improvements of up to 26% when compared with standard Valiant routing when using synthetic traffic from independent sources at different scales. Furthermore, when using realistic application-inspired workloads, we found the strategies required between 5% and 20% less time to perform communications. In general, we observe that selecting proxies that are adjacent to the sender is more beneficial than those adjacent to the destination because the latter tends to generate backpressure in the last level of the interconnect. Interestingly, we found that the most restrictive proxy routing strategies obtain the best results in all scenarios and show that counterintuitively, the lower the path diversity, the more balanced the use of network resources. Our study includes investigating the interplay between routing and Dragonfly parameters and provide optimal parameters for proxy-based routing algorithms. Finally, we discuss some practical considerations related to the deployment of our strategies.