Multi-Agent DRL-Based Two-Timescale Resource Allocation for Network Slicing in V2X Communications

Abstract

Network slicing has been envisioned to play a crucial role in supporting various vehicular applications with diverse performance requirements in dynamic Vehicle-to-Everything (V2X) communications systems. However, time-varying Service Level Agreements (SLAs) of slices and fast-changing network topologies in V2X scenarios may introduce new challenges for enabling efficient inter-slice resource provisioning to guarantee the Quality of Service (QoS) while avoiding both resource over-provisioning and under-provisioning. Moreover, the conventional centralized resource allocation schemes requiring global slice information may degrade the data privacy provided by dedicated resource provisioning. To address these challenges, in this paper, we propose a two-timescale resource management mechanism for providing diverse V2X slices with customized resources. In the long timescale, we propose a Proximal Policy Optimization-based multi-agent deep reinforcement learning algorithm for dynamically allocating bandwidth resources to different slices for guaranteeing their SLAs. Under the coordination of agents, each agent only observes its partial state space rather than the global information to adjust the resource requests, which can enhance the privacy protection. Moreover, an expert demonstration mechanism is proposed to guide the action policy for reducing the invalid action exploration and accelerating the convergence of agents. In the short-term time slot, with our proposed Cross Entropy and Successive Convex Approximation algorithm, each slice allocates its available physical resource blocks and optimizes its transmit power to meet the QoS. Simulation results show our proposed two-timescale resource allocation scheme for network slicing can achieve maximum 8.4% performance gains in terms of spectral efficiency while guaranteeing the QoS requirements of users compared to the baseline approaches.