Deep Reinforcement Learning-Based Dynamic MultiChannel Access for Heterogeneous Wireless Networks with DenseNet

Abstract

In this paper, we consider the problem of dynamic multi-channel access in the heterogeneous wireless networks, where multiple independent channels are shared by multiple nodes which have different types. The objective is to find a strategy to maximize the expected long-term probability of successful transmissions. The problem of dynamic multi-channel access can be formulated as a partially observable Markov decision process (POMDP). In order to deal with this problem, we apply the deep reinforcement learning (DRL) approach to provide a model-free access method, where the nodes don’t have a prior knowledge of the wireless networks or the ability to exchange messages with other nodes. Specially, we take advantage of the double deep Q-network (DDQN) with DenseNet to learn the wireless network environment and to select the optimal channel at the beginning of each time slot. We investigate the proposed DDQN approach in different environments for both the fixed-pattern scenarios and the time-varying scenarios. The experimental results show that the proposed DDQN with DenseNet can efficiently learn the pattern of channel switch and choose the near optimal action to avoid the collision for every slot. Besides, the proposed DDQN approach can also achieve satisfactory performance to adapt the time-varying scenarios.