Partially Observable Multi-Agent Deep Reinforcement Learning for Cognitive Resource Management

Abstract

In this paper, the problem of dynamic resource management in a cognitive radio network (CRN) with multiple primary users (PUs), multiple secondary users (SUs), and multiple channels is investigated. An optimization problem is formulated as a multi-agent partially observable Markov decision process (POMDP) problem in a dynamic and not fully observable environment. We consider using deep reinforcement learning (DRL) to address this problem. Based on the channel occupancy of PUs, a multi-agent deep Q-network (DQN)-based dynamic joint spectrum access and mode selection (SAMS) scheme is proposed for the SUs in the partially observable environment. The current observation of each SU is mapped to a suitable action. Each secondary user (SU) takes its own decision without exchanging information with other SUs. It seeks to maximize the total sum rate. Simulation results verify the effectiveness of our proposed schemes.