Multi-agent reinforcement learning for task offloading with hybrid decision space in multi-access edge computing

Abstract

Multi-access Edge Computing (MEC) has become a significant technology for supporting the computation-intensive and time-sensitive applications on the Internet of Things (IoT) devices. However, it is challenging to jointly optimize task offloading and resource allocation in the dynamic wireless environment with constrained edge resource. In this paper, we investigate a multi-user and multi-MEC servers system with varying task request and stochastic channel condition. Our purpose is to minimize the total energy consumption and time delay by optimizing the offloading decision, offloading ratio and computing resource allocation simultaneously. As the users are geographically distributed within an area, we formulate the problem of task offloading and resource allocation in MEC system as a partially observable Markov decision process (POMDP) and propose a novel multi-agent deep reinforcement learning (MADRL) -based algorithm to solve it. In particular, two aspects have been modified for performance enhancement: (1) To make fine-grained control, we design a novel neural network structure to effectively handle the hybrid action space arisen by the heterogeneous variables. (2) An adaptive reward mechanism is proposed to reasonably evaluate the infeasible actions and to mitigate the instability caused by manual configuration. Simulation results show the proposed method can achieve $7.12\text{\%}-20.97\text{\%}$ performance enhancements compared with the existing approaches.