Heterogeneous Wireless Resources On-demand Access Distributed Decision Based on Spectrum Sensing

In this paper, we investigate the problem of multi-user on-demand access to heterogeneous wireless resources based on spectrum sensing (SS) considering the impact of traffic buffering scenarios, the variation of channel occupancy, and channel quality levels. Because both variations can be modeled as Markov chains, a two-layer Markov chain modeling approach and a two-layer Multi-Agent deep reinforcement learning (MARL) training model are used to achieve a reasonable matching of users’ load degrees with the heterogeneous wireless resource blocks. Specifically, the local information of spectrum sensing is used as the input state of the outer layer Agent, while the user’s current traffic queue information and the action decisions collected by the local accumulation window, and the corresponding acknowledge (ACK) feedback are used as the input state of the inner layer. A network-wide utility function associated with the user traffic changes is used to centrally train the two layers of user Agents offline. According to the simulation results, the proposed algorithm significantly simplifies the online computation and online interaction process because the complex training process has been completed offline.