Hybrid deep reinforcement learning-based workload migrating and resource allocation policies for weighted cost minimization in edge collaboration networks

Abstract

In the context of efficient collaboration on 5G heterogeneous networks, computation and communication, mobile edge computing (MEC) and cloud–edge collaboration encounter several issues. These include multi-agent cooperation in deep reinforcement learning (DRL) for multi-task processing, the efficiency of action decisions by mobile edge computing server (MECS), and the ineffectiveness of traditional DRL algorithms in resource allocation. To address these challenges, a framework consisting of multiple wireless mobile terminals (WMTs) enabled by unmanned aerial vehicle (UAV) and multiple MEC servers is constructed, considering diversity and priorities of the workload generated by WMTs. Furthermore, to optimize edge collaborative workload offloading, migrating, and resource allocation decisions for minimizing the weighted cost of workload processing, we propose a hybrid DRL approach combining the dueling double deep Q-network (D3QN) and deep deterministic policy gradient (DDPG) algorithms, named OMRA-DRL. Regarding the proposed OMRA-DRL, a K-Means-based clustering algorithm groups similar workloads to simplify optimization. Additionally, a mixture-of-expert (MoE) system enables efficient action selection. Along with D3QN for better MECS selection, the maximum advantage selection strategy of the advantage function (MMSS-D3QN) is formulated to migrate workload clusters to multiple MECSs, achieving multi-edge cooperation. Comprehensive simulation experiments prove the convergence of OMRA-DRL under various parameters. Moreover, it outperforms the five benchmark algorithms in terms of average cumulative reward and unfinished ratio of workloads, with an increase of over 15% in average cumulative reward and a decrease of about 5% in unfinished rate of workloads, demonstrating its effectiveness in achieving weighted cost minimization in edge collaborative networks.