Decentralised, dynamic network path selection in high performance computing

Abstract

In this paper, we investigate the problem of providing highly available, decentralized, dynamic path selection in high performance computing networking. We look at a use case for dynamic path selection that better utilizes bandwidth available in the network. The network architecture we propose is a partial mesh grid whereby each host is directly connected to four forwarding devices. We propose an approach that decouples artificial network information and state from the devices making forwarding decisions. This method constructs a path one hop at a time while at each hop, selecting the next link based on the status of the link. The goal of this approach is to create a network topology in which new transmissions are routed around congestion. It also allows us to build a network with low-cost, lower bandwidth links instead of a few high cost, high bandwidth links. The goal of this work is to take the ideas of a controller based SDN solution and examine the merits of distributing that functionary to the actual nodes in the network.