Collaborative Task Offloading and Resource Allocation in Small-Cell MEC: A Multi-Agent PPO-Based Scheme

Abstract

Small-cell mobile edge computing (SE-MEC) networks amalgamate the virtues of MEC and small-cell networks, enhancing data processing capabilities of user devices (UDs). Nevertheless, time-varying wireless channels, dynamic UD requirements, and severe interference among UDs make it difficult to fully exploit the limited network resources and stably provide computing services for UDs. Therefore, efficient task offloading and resource allocation (TORA) is essential. Moreover, since multiple small cells are deployed, decentralized TORA schemes are preferred in practice. Thus, this paper aims to design distributed adaptive TORA schemes for SE-MEC networks. In pursuit of an eco-friendly design, an optimization problem is formulated to minimize the total energy consumption (TEC) of UDs subject to delay constraints. To effectively deal with network's dynamic characteristics, the reinforce learning framework is applied, where the TEC minimization problem is first modeled as a partially observable Markov decision process (POMDP), and then an efficient multi-agent proximal policy optimization (MAPPO)-based scheme is presented to solve it. In the presented scheme, each small-cell base station (SBS) serves as an agent and is capable of making TORA decisions only with its own local information. To promote collaboration among multiple agents, a global reward function is designed. A state normalization mechanism is also introduced into the presented scheme for enhancing learning performance. Simulation results show that although the proposed MAPPO-based scheme works in a distributed manner, it achieves very similar performance to the centralized one. In addition, it is demonstrated that the state normalization mechanism has a significant effect on reducing TEC.