Q-Learning Based Load Balancing in Heterogeneous Networks with Human and Machine Type Communication Co-existence

Abstract

A heterogeneous network, also known as a HetNet, is a network made up of numerous distinct wireless network nodes placed throughout the cellular service coverage area. These nodes have differing features and capabilities. In some areas, the user density may be high or a single macro eNB cannot supply good coverage. This problem can be solved by deploying low-power nodes, such as pico-cells, within the range of a macro-cell. A model built of macro eNBs and pico eNBs, used to fulfill machine-to-machine (M2M) and human-to-human (H2H) devices, no matter how different is the quality of service (QoS), is the main topic of this paper. Furthermore, the paper presents a method to associate cells and balance loads on the network for both M2M and H2H devices, by utilizing a new scheme based on Q-learning. The scheme employs two independent algorithms, where both of them are based on Q-learning. The performance of the proposed scheme is assessed by comparison in two ways: traditional comparison and a Q-learning based scheme deployed in the UE devices, to compare the H2H and M2M blocking probability with the M2M uplink transmission power. As concluded from the results of testing the proposed scheme, the blocking probability of M2M and H2H devices is reduced by about 20–30%. Furthermore, the uplink transmission power of M2M devices is reduced by 50% even under high load conditions, which makes this scheme power-efficient.