Deep Q-Network Enabled Low Complexity Beam Alignment for mmWave Massive MIMO System

Abstract

In this letter, we model the beam alignment problem in millimeter-wave massive MIMO system as a Markov decision process. Given the highly dynamic environment, we leverage deep reinforcement learning to solve the non-convex beam alignment problem. Specifically, by leveraging the locations of the target vehicle and its neighboring vehicles, we propose a deep Q-network (DQN)-based beam alignment algorithm with lower computation complexity. In addition, to address the issue of model obsolescence caused by the changed traffic density, an online mode is elaborated to dynamically adjust to changing environments. Simulation results show that the proposed DQN scheme has better alignment performance and lower complexity than the existing baseline schemes.