Ensemble Link Learning for Large State Space Multiple Access Communications

Abstract

Wireless communication networks are well-modeled by Markov Decision Processes (MDPs), but induce a large state space which challenges policy optimization. Reinforcement learning such as Q-learning enables the solution of policy opti-mization problems in unknown environments. Herein a graph-learning algorithm is proposed to improve the accuracy and complexity performance of Q-learning algorithm for a multiple access communications problem. By exploiting the structural properties of the wireless network MDP, several structurally related Markov chains are created and these multiple chains are sampled to learn multiple policies which are fused. Furthermore, a state-action aggregation method is proposed to reduce the time and memory complexity of the algorithm. Numerical results show that the proposed algorithm achieves a reduction of 80% with respect to the policy error and a reduction of 70% for the runtime versus other state-of-the-art $Q$ learning algorithms.