Adaptive Beam Handover Algorithm Based on Reinforcement Learning for mmWave System

In millimeter wave (mmWave) frequency band, the link quality is greatly affected by the environment. Coupled with the dense deployment of mmWave access points (APs) and the using of beamforming technology, beam handovers frequently occur in mobile communication systems. This paper optimizes the mmWave beam handover process using the Q-Learning method. Our proposed algorithm learns and senses the complex communication environment by the beam reporting information from the users. We consider the effects of handover cost and historical beam quality during the handover process. The adaptive handover threshold is obtained by querying the Q table according to the current state. Simulation results show that our proposed algorithm reduces the number of beam handover times and improves the system performance compared with the original scheme in the 3GPP protocol.