Multi-objective optimization for submarine optical fiber cable route planning based on collaborative reinforcement learning with LPGP framework

Abstract

Submarine optical fiber cables are essential to international communication, transmitting approximately 99% of global traffic. The cost and survivability of these cables are key factors that must be carefully considered during the design stage. However, the cost of submarine cables and the risks closely associated with their survivability have not yet been decoupled and simultaneously optimized. In this paper, we propose a Local Pareto to Global Pareto (LPGP) paradigm for multi-objective optimization. Based on this paradigm, we design an offline collaborative reinforcement learning LPGP (Off-CRL-LPGP) framework that effectively decouples and simultaneously optimizes the cost and risk of submarine optical fiber cable routing. The results demonstrate that Off-CRL-LPGP reduces accumulated costs by 28.83% compared to ant colony optimization (ACO) under the same risk conditions, while requiring significantly less computational time. Compared to multi-agent cross reinforcement learning (MA-XRL), under the same accumulated risk and accumulated cost conditions, the Off-CRL-LPGP could respectively reduce accumulated cost by 3% and accumulated risk by 1.1%, at the expense of some additional computational time. Compared to online summation for global Pareto (On-Sum-GP), Off-CRL-LPGP could respectively reduce accumulated cost by 7.8% and risk by 23.48%. We also investigate the impact of algorithm combinations on the performance of Off-CRL-LPGP. Alternating Q-learning and SARSA (alternate Q-S/S-Q) could reduce accumulated costs by 3.83% and risk by 13.78%, while improving the convergence level for cost by 2.18 times and for risk by 3.30 times. Furthermore, data smoothing method proposed in this work reduces accumulated cost and risk by 4.58% and 6.17%, respectively, and improves stability in 97.66% of iterations, with a maximum stability improvement of 6.64 times.