A Reinforcement Learning Approach for D2D Spectrum Sharing in Wireless Industrial URLLC Networks

Abstract

Distributed Radio Resource Management (RRM) solutions are gaining an increasing interest recently, especially when a large number of devices are present as in the case of a wireless industrial network. Self-organisation relying on distributed RRM schemes is envisioned to be one of the key pillars of 5G and beyond Ultra Reliable Low Latency Communication (URLLC) networks. Reinforcement learning is emerging as a powerful distributed technique to facilitate self-organisation. In this paper, spectrum sharing in a Device-to-Device (D2D)-enabled wireless network is investigated, targeting URLLC applications. A distributed scheme denoted as Reinforcement Learning Based Matching (RLBM) which combines reinforcement learning and matching theory, is presented with the aim of achieving an autonomous device-based resource allocation. A distributed local Q-table is used to avoid global information gathering and a stateless Q-learning approach is adopted, therefore reducing requirements for a large state-action mapping. Simulation case studies are used to verify the performance of the presented approach in comparison with other RRM techniques. The presented RLBM approach results in a good tradeoff of throughput, complexity and signalling overheads while maintaining the target Quality of Service/Experience (QoS/QoE) requirements of the different users in the network.