Decentralized cognitive MAC protocol design based on POMDP and Q-Learning

Abstract

A decentralized cognitive MAC protocol is proposed in this paper, whose core depends on the partially observed Markov process (POMDP) and Q-Learning. Limited by the hardware and environment, a secondary user may not be able to have ability to sense the entire spectrum space. Therefore, the POMDP is exploited to model the secondary network. In this paper, Q-Learning is applied to solve the POMDP because it can make full advantage of the past observation and decision experiences to optimize current action, and needs not transfer the POMDP into a belief MDP. The numeral simulation results show that Q-Learning-based decentralized cognitive MAC protocol improves the overall performance of the networks.