Q-learning based Radio Resource Adaptation for Improved Energy Performance of 5G Base Stations

Radio resource adaptation (RRA) is an effective strategy to reduce the energy consumption (EC) of a base station (BS) under variable input traffic demand. By combining RRA with advanced sleep modes (ASMs), one could achieve relatively higher energy savings (ES) during the low traffic hours of the day while managing to meet the quality of service (QoS) requirements of the user equipments (UEs). However, identifying appropriate resources for a certain period is challenging as different resources (i.e., the bandwidth and the antenna array size) have a varying impact on the instantaneous power consumption (PC) and activity of the BS. Various works have looked into the potential of RRA and ASMs in reducing the EC of a BS when implemented independently. In this work, we combine RRA with ASMs and propose a dynamic Q-learning algorithm that adapts a BS’s resources according to the traffic demand. The algorithm also takes into account the sleep modes (SMs) that the BS can switch to during the idle periods. Through simulations, we show the convergence of our algorithm and the impact of combining RRA with ASMs on the overall ES as we observe up to 16% additional savings in a super-dense urban (SDU) deployment scenario by combining these techniques as compared to the baseline scenario using only ASMs.