Evaluation of Radio Channel Utility using Epsilon-Greedy Action Selection

Abstract

This paper presents an algorithm that supports the dynamic spectrum access process in cognitive radio networks by generating a sorted list of best radio channels or by identifying those frequency ranges that are not in use temporarily. The concept is based on the reinforcement learning technique named Q-learning. To evaluate the utility of individual radio channels, spectrum monitoring is performed. In the presented solution, the epsilon-greedy action selection method is used to indicate which channel should be monitored next. The article includes a description of the proposed algorithm, scenarios, metrics, and simulation results showing the correct operation of the approach relied upon to evaluate the utility of radio channels and the epsilon-greedy action selection method. Based on the performed tests, it is possible to determine algorithm parameters that should be used in this proposed deployment. The paper also presents a comparison of the results with two other action selection methods. Keywords—cognitive radio, dynamic spectrum access, spectrum monitoring, machine learning, Q-learning.