Analyzing Cost-Performance Tradeoffs of HPC Network Designs under Different Constraints using Simulations

Identifying a suitable network topology and deciding its optimal configuration parameters are critical aspects of the overall HPC system design, procurement and installation process. Typically, multiple network topology choices are compared under the balanced injection-to-global bandwidth criterion to identify the best candidate. However, deviating from this balanced criterion may not impact application performance adversely and is often done in practice due to other considerations such as monetary cost. In this paper, we identify different practical constraints that determine the number of nodes, routers, and links, and in turn, influence dollar costs and impact network design. We design network topologies under one or more such constraints which represent different design points (iso-{*} analysis). We then perform a comprehensive, comparative evaluation of three scalable network topologies -- dragonfly, express mesh, and fat-tree -- enabled by parallel discrete-event simulations (PDES) of relevant HPC workloads. We identify network topologies that perform best under different iso-{*} configurations and compare their performance per dollar based on market data.