Multichannel non-persistent CSMA MAC schemes with reinforcement learning for cognitive radio networks

This paper presents two multichannel non-persistent CSMA (M-np-CSMA) MAC schemes using Simple Reinforcement Learning and State-Action-Reward-State-Action (SARSA) learning respectively for distributed cognitive radio networks. The two learning schemes both use reinforcement learning to help the users learn the environment and historical transmissions. Compared with M-np-CSMA MAC protocol with random channel choice, the learning schemes can help the cognitive users choose the best channels which offer more spectrum access opportunities to sense and access. The results show that both learning schemes can effectively improve the throughput and decrease the packet delay at heavy traffic loads and with a large number of cognitive users. The Simple Reinforcement Learning scheme and SARSA scheme can achieves a 15% and 25% improvement in the maximum throughput respectively, compared with the M-np-CSMA without learning.