Deep Reinforcement Learning based Wireless Network Optimization: A Comparative Study

There is a growing interest in applying deep reinforcement learning (DRL) methods to optimizing the operation of wireless networks. In this paper, we compare three state of the art DRL methods, Deep Deterministic Policy Gradient (DDPG), Neural Episodic Control (NEC), and Variance Based Control (VBC), for the application of wireless network optimization. We describe how the general network optimization problem is formulated as RL and give details of the three methods in the context of wireless networking. Extensive experiments using a real-world network operation dataset are carried out, and the performance in terms of improving rate and convergence speed for these popular DRL methods is compared. We note that while DDPG and VBC demonstrate good potential in automating wireless network optimization, NEC has a much improved convergence rate but suffers from the limited action space and does not perform competitively in its current form.