Optimizing the Ratio-Based Offloading in Federated Cloud-Edge Systems: A MADRL Approach

In the evolving landscape of cloud-edge federated systems, Multi-Access Edge Computing (MEC) plays a crucial role by being closer to user equipment (UEs). However, it has limited capacity compared to the cloud, leading to challenges during periods of high network traffic, commonly referred to as hotspot traffic, when MEC resources can become overwhelmed. To mitigate this issue, horizontal and vertical traffic offloading between edges and core sites, and from core sites to the cloud, respectively, is employed. The offloading decisions, crucial for ensuring network efficiency, must be made within seconds. Traditional optimization techniques are unsuitable due to their computational intensity and time-consuming nature, necessitating a shift toward machine learning methods. This research introduces a ratio-based offloading approach, leveraging a multi-agent deep reinforcement learning (MADRL) approach based on the twin-delayed deep deterministic policy gradient (TD3) algorithm. In a comparative evaluation against the simulated annealing (SA) algorithm and single-agent deep reinforcement learning (DRL) approaches, our proposed solution exhibits superior performance, particularly in terms of decision time. The DRL-based approach achieves convergence within seconds, whereas SA takes minutes. Additionally, the average latency experienced by traffic in the multi-agent TD3 configuration is approximately 3–4 times less than in the single-agent configuration.