Joint optimization of cognitive RF energy harvesting and channel access using Markovian Multi-Armed Bandit problem

With the growing number of connected devices (e.g. smartphones, sensors, actuators and cameras) in 5G, the massive Internet of Thing (IoT) is expected to address a wide range of characteristics and demands, with more radio-frequency (RF) bands to support multiple frequency transmission by 2020. Separate antennas and RF chips may be required for widely separated frequency bands. The development of cognitive radio systems focused on channel access and spectrum sensing, which can be programmed and configured dynamically, is therefore fundamental for the successful deployment of such systems. This paper introduces an innovative system for spectrum sensing, channel access and power for energy constrained wireless sensor networks. A commercially available evaluation board is used to detect and convert RF signals to a DC voltage for powering a sensor board that communicates over other channels to an access point. Our spectrum sensing conceptualizes the Markovian formulation of the exploration/exploitation balancing technique of available frequency bands, also known as the Multi-Armed Bandit (MAB) problem. To optimize RF energy harvesting and data transmission in channels modeled by a Rayleigh distribution, we formulate and solve a joint reward equation for the MAB Gittins indices, then compare with another MAB algorithm. The simulation results show the optimality of the Gittins index allocation strategy in terms of channel selection percentage and minimization of the regret.