MATE: A multi-agent reinforcement learning approach for Traffic Engineering in Hybrid Software Defined Networks

Abstract

Hybrid Software Defined Networks (Hybrid SDNs), which combines the robustness of distributed network and the flexibility of centralized network, is now a prevailing network architecture. Previous hybrid SDN Traffic Engineering (TE) solutions search an optimal link weight setting or compute the splitting ratios of traffic leveraging heuristic algorithms. However, these methods cannot react timely to the fluctuating traffic demands in dynamic environments and suffer a hefty performance degradation when traffic demands change or network failures happen, especially when network scale is large. To cope with this, we propose a Multi-Agent reinforcement learning based TE method MATE that timely determines the route selection for network flows in dynamic hybrid SDNs. Through dividing the large-scale routing optimization problem into small-scale problem, MATE can better learn the mapping between the traffic demands and routing policy, and efficiently make online routing inference with dynamic traffic demands. To collaborate multiple agents and speed up the convergence in the training process, we innovatively design the actor network and introduce previous actions of all agents in the training of each agent. Extensive experiments conducted on different network topologies demonstrate our proposed method MATE has superior TE performance with dynamic traffic demands and is robust to network failures.