Q-Learning Based Power Allocation in Self Organizing Heterogeneous Networks

Abstract

Resource allocation and interference management have been fundamental topics of interest in wireless cellular networks. Low powered small cells can be deployed by indoor users to diminish cellular coverage problem, offload traffic from the macrocell based cellular network, and enhance over-all user throughput. In order to facilitate extensive usage of small cells coexisting with the macrocells for next generation cellular networks, new interference management schemes are required. We consider a two-tier heterogeneous cellular network with conventional macrocells overlaid with femtocells. A multi agent Markov decision process based distributed framework is proposed to model resource allocation in the cellular network. We explore a reinforcement learning (RL), in particular, a Q-learning based self organizing mechanism for power allocation that enables adaptation of transmit power as new femtocells are being added to the network. By mitigating the co-tier and cross-tier interference simultaneously, the proposed technique maximizes the sum capacity of femtocell network, even though it maintains the quality of service (QoS), represented in terms of the transmission rate requirement, of all the femtocell user equipment (FUEs) and macrocell user equipment (MUE).