ScaIR: Scalable Intelligent Routing based on Distributed Graph Reinforcement Learning

Abstract

Traditional routing typically relies on simpler performance metrics that can be derived directly through mathematical methods for decision-making, which often results in limited optimization outcomes. As future networks expand, along with the diversity of applications and traffic volume, the network environment grows increasingly complex. In contrast, Intelligent Routing (IR) that leverages machine learning methods can model more complex performance metrics, rendering it better suited to the intricate scenarios of future networks. The increasing complexity of networks also indicates that the workload associated with collecting routing information and executing decision calculations is growing exponentially. Compared to centralized IR, Distributed IR (DIR) distributes the computational load and interaction demands across multiple nodes, thereby offering enhanced scalability. However, DIR makes decisions based on local information, which limits global optimization. In this paper, we propose a novel Scalable Intelligent Routing based on Distributed Graph Reinforcement Learning, called ScaIR. ScaIR is a full y distributed multi-agent routing method. Each router is an independent agent based on local graph Reinforcement Learning (RL). Graph Neural Networks (GNN) are employed to extract global network characteristics which serve as input for RL, thereby enhancing global optimization. Especially, GNN here is also fully distributed. Each router has an independent sub-GNN determined by the adjacency relationships with its one-hop neighbors. Instead of entire network information and model parameters, each sub-GNN only iteratively interacts with its neighbors and computes a highly compressed Feature Vector (FV) representing the current network state, which greatly saves the computing and communication cost. We carried out extensive simulation experiments under multiple real network topologies of different scales. Simulation results show that ScaIR reduces forwarding time by more than 25% and achieves faster convergence. It can better adapt to congested, dynamic or unknown environments. Compared to other methods, it significantly reduces communication cost and computational time, and has better scalability. In addition, by changing the FV length of sub-GNNs, it is verified that GNN does play a key role in ScaIR.