MOOO-RDQN: A deep reinforcement learning based method for multi-objective optimization of controller placement and traffic monitoring in SDN

Abstract

Software Defined Networks (SDN) necessitates efficient controller placement strategies to address the NP-hard Controller Placement Problem (CPP), which involves minimizing propagation latency, balancing controller loads, and ensuring adaptability to dynamic network conditions. Traditional heuristic and deterministic algorithms face challenges in balancing optimality and computational efficiency, particularly in large-scale heterogeneous networks. This paper proposes Multi-Objective Optimization Oriented-Rainbow Deep Q Network (MOOO-RDQN), deep reinforcement learning framework that synergizes five advanced techniques, including double Q-learning, prioritized experience replay, dueling networks, multi-step learning, and noisy networks, to jointly optimize controller placement and switch-controller mapping. Experimental evaluations on real-world topologies demonstrate that MOOO-RDQN outperforms standard and state-of-the-art algorithms, achieving reductions of up to 42.49% in average controller-switch latency, 59.39% in worst-case latency, 30.56% in load imbalance, and 28.73% in training time. The solution gap from brute-force global optima remains below 15% across diverse network scales. Complementing the algorithmic innovation, we design an FPGA (Field-Programmable Gate Array) based traffic monitoring module utilizing CAN (Controller Area Network) interfaces and LED (Light-Emitting Diode) indicators to detect controller overloads in real-time. This hardware-software co-design not only validates the practicality of MOOO-RDQN but also lays the foundation for future works on closed-loop control plane optimization.