A GAN enhanced meta-deep reinforcement learning approach for DCN routing optimization

Abstract

In large-scale data center networks (DCN), dynamic changes in traffic and topology lead to traffic patterns following a long-tail distribution (meaning a small number of traffic samples appear frequently, while most appear infrequently). This results in clear Non-IID (non-independent and identically distributed) characteristics, posing a serious challenge for traffic routing optimization. Traditional deep reinforcement learning (DRL) methods often assume relatively stable data distributions, while approaches that combine DRL with meta-learning assume only small shifts in traffic or the environment. However, such methods struggle to cope with the scarcity of samples and cross-scenario feature-space shifts caused by long-tail distributions. To address this issue, this paper proposes a global intelligent routing optimization scheme based on meta-reinforcement learning (Meta-DRL) enhanced by generative adversarial networks (GAN). First, a two-level nested Meta-DRL model is built. The lower level focuses on specific task policy optimization, while the upper level learns generalized global network parameters, improving the model’s initialization quality and generalization in unfamiliar network settings. Next, we introduce an innovative mechanism that combines GAN-based feature encoding with a meta-learning discriminator, refining the GAN’s feature discrimination boundary and greatly enhancing the quality of synthesized samples, especially where data is scarce. Furthermore, a parameter feature space mapping mechanism is proposed to unify features from new and old tasks into a shared representation space, avoiding repeated Meta-DRL training when the network environment changes. This substantially boosts the model’s generalization and decision-making efficiency. Simulation results show that, in terms of convergence speed, accumulated rewards, and network routing performance, the GAN-enhanced Meta-DRL method significantly outperforms traditional DRL approaches and other meta-learning methods.