Improving Multi-Band Elastic Optical Networks Performance using Behavior Induction on Deep Reinforcement Learning

Abstract

Deep Reinforcement Learning (DRL) has proven a considerable potential for enabling non-trivial solutions to resource allocation problems in optical networks. However, applying plain DRL does not ensure better performance than currently known best heuristics solutions. DRL demands a parameter tuning process to improve its performance. One tuning possibility is the reward function design. The reward function allows feedback to the agents on whether the actions sent to the environment were successful or not. A transparent reward function returns whether the action succeeds or not, but an elaborate reward function may allow inducing the desired behaviour to improve DRL performance. Our work designs reward functions in multi-band elastic optical networks (MB-EON) to improve the overall network blocking probability. A test environment was set up to analyze the performance of four reward functions for inducing a lower blocking probability. The proposed reward functions use band usage, link compactness, spectrum availability and link fragmentation as feedback information to the agents. Analysis was carried out using the DQN agent in the NSFNet network topology. Results show that reward function design improves the blocking probability. The best-performing one uses the band availability criteria, decreasing the blocking probability, as an average, by 22% compared to the baseline reward function, with a peak of 63,67% of improvement for a 1000 Erlang traffic load scenario.