Cooperative Multi-Agent Reinforcement Learning for Spectrum Management in IoT Cognitive Networks

The paper investigates the applications of cooperative Multi-Agent Reinforcement Learning (MARL) schemes to Cognitive Radio Networking (CRN), which in turn can facilitate spectrum utilization for wireless (ad hoc) networks within the Internet of Things (IoT). These schemes provide the ability of wireless transceivers to learn the optimal control and configuration in unknown environmental and application conditions, exploiting potential for cooperation among spectrum secondary users. An overview of the existing MARL approaches to the CRN is provided, with an analysis of their advantages and weaknesses compared to the rest of CRN approaches. We argue that in typical CRN practical scenarios including IoT systems, it is of essential importance that the cooperative algorithms are completely decentralized and distributed, having also a capability that the agents/nodes together can successfully calculate the optimal strategy even if the individual agents cannot. Hence, we propose a new scheme for cooperative spectrum sensing and selection within CRN, based on an adaptation of a recently proposed cooperative MARL scheme, provide detailed analysis of its properties and potential performance, indicating its superiority compared to the existing schemes.