Reinforcement learning for dynamic resource allocation in optical networks: hype or hope?

Abstract

The application of reinforcement learning (RL) to dynamic resource allocation in optical networks has been the focus of intense research activity in recent years, with almost 100 peer-reviewed papers. We present a review of progress in this field and identify weaknesses in benchmarking practices and reproducibility. To demonstrate best practice, we exactly recreate the problem settings from five landmark papers and apply improved benchmarks. To determine the best benchmarks, we evaluate several heuristic algorithms and optimize the candidate path count and sort criteria for path selection. We apply the improved benchmarks and demonstrate that simple heuristics outperform the published RL solutions, often with an order of magnitude lower blocking probability. Finally, to estimate the limits of improvement on the benchmarks, we present empirical lower bounds on blocking probability using a novel, to our knowledge, defragmentation-based method. Our method estimates that traffic load can be increased by 19%–36% for the same blocking in our examples, which may motivate further research on optimized resource allocation. We make our simulation framework and results openly available to promote reproducible research and standardized evaluation: https://doi.org/10.5281/zenodo.12594495.