Traffic Priority-Aware Multi-User Distributed Dynamic Spectrum Access: A Multi-Agent Deep RL Approach

Abstract

Real-time information exchange on traffic and channel selection results among users in dynamic spectrum access (DSA) system consumes scarce spectrum resources. However, it is difficult to avoid collision and improve system-wide global utility simultaneously without assistance of these information in a distributed way. To solve this problem, we propose a multi-agent deep reinforcement learning (RL) based traffic priority-aware multi-user distributed DSA scheme for a multiple orthogonal channels scenario. Different from the conventional approaches for throughput sum maximization, we maximize a total network utility parameterized by the situation of each user’s traffic buffer queue. This scheme includes off-line centralized training and distributed execution. The deep Q-learning neural network (DQN) of each user is trained by an offline simulator with global information to learn near-optimal channel selection policies from the transition history. The input of DQN requires only user’s local observation to ensure that the scheme based on the trained DQNs can be executed in a distributed way. Simulation results show that the proposed scheme compared with benchmark algorithms can achieve about 90% or more of performance of Genie-aided algorithm based on global information, and is much better than random-type algorithms.