Probing-aided spectrum sensing-based hybrid access strategy for energy harvesting CRNs

To address the issues of energy supply and spectrum scarcity in Internet of Things (IoT), energy harvesting (EH) and cognitive radio (CR) technologies have been proposed and widely applied. In EH-CR networks (EH-CRNs), miss detection causes significant energy and time wastage of IoT devices, especially secondary users (SUs), and causes serious interference to primary users (PUs). To alleviate this concern, we propose a probing-aided spectrum sensing (PaSS) model for EH-CRNs, where $M$ pairs of PUs and one pair of SUs coexist. The secondary transmitter (ST) harvests energy from the radio frequency (RF) signals of PUs for opportunistic spectrum access. In the PaSS model, probing operation is employed to further confirm the real state of the spectrum that has been sensed as free in order to avoid the waste of time and energy resulting from miss detection. Based on the PaSS model, we propose a novel hybrid access strategy, where the ST’s actions (i.e., sensing, probing, EH, underlay/overlay transmission mode) depend on the belief vector of $M$ channels, energy state and data buffer state of the ST. By developing an adjusted double deep Q-network (ADDQN) reinforcement learning algorithm, we aim to find the optimal strategy that minimizes the long-term average number of packet losses (ANPL) and the ANPL minimization problem is an integer programming problem. Simulation results validate the ANPL performance of the ST in the ADDQN-PaSS model, and reveal impacts of network parameters on the performance of the ST, and find that at least 7.9% reduction of ANPL is achieved by using the ADDQN-PaSS model.