Deep Reinforcement Learning-Based Adaptive Nulling in Phased Array Under Dynamic Environments

Abstract

A phased array consists of multiple antenna elements that can control the direction of the radiated signal by adjusting each antenna element’s phase and amplitude, which are encapsulated in the phased array weights. To obtain better communication quality, nulling, which can weaken the interference signal, is helpful by adjusting the phased array weights. In dynamic environments, rapid changes in the directions of both interference and desired signals demand equally rapid, continual updates of phased-array weights. Traditional heuristic optimizers—such as Particle Swarm Optimization (PSO) and Genetic Algorithms (GA)—struggle to keep up because their iterative searches depend on pre-computed configurations that are unrealistic to obtain on the fly. To date, no heuristic, supervised-learning, or reinforcement-learning method simultaneously achieves all three requirements: fast adaptation, dataset-free operation, and continuous complex-weight control under highly dynamic environments. In this paper, an innovative deep reinforcement learning-based adaptive nulling, called DRLNuller, is proposed. DRLnuller adopts the Proximal Policy Optimization (PPO) algorithm, a typical reinforcement learning algorithm, to dynamically optimize phased array weights through continuous interaction with the environment without relying on pre-computed or labeled data. In experiments, DRLNuller after the training process outperforms PSO and GA in computation speed by $2.83\times 10^{5}$ times faster and maintains effective communication quality, an average Signal-to-Interference Ratio (SIR) of 25.06 dB, under different conditions.