Deep reinforcement learning based interference optimization for coordinated beamforming in ultra-dense Wi-Fi networks

Abstract

Next-generation Wi-Fi networks are expected to have an ultra-dense deployment of access points (APs), thus, interference from overlapping basic service sets (OBSSs) poses challenges for interference management. Wi-Fi 8 aims at mitigating such interference using multi-access point coordination (MAPC). One of the MAPC variants is coordinated beamforming (Co-BF), where neighboring APs direct their signals towards specific users. Besides beam steering, APs can also perform null steering, which is more complex but can bring greater performance gains. In this paper, we present a centralized approach named intelligent null steering by reinforcement learning (IntelliNull), designed to reduce interference from neighboring transmitters by coordinated nulling while maximizing the signal quality at each station. We show that training the beam and null steering mechanism with a deep deterministic policy gradient (DDPG), it is possible to steer beams toward associated stations while intelligently nulling the most destructive interference from OBSS rather than nulling random interference directions. This method enhances communication between the AP and neighboring stations by reducing channel access contention, enabling transmissions at full power, and reducing worst-case latency. The proposed IntelliNull agent continuously adapts to changes in the network environment, including node mobility using channel state information (CSI) collected in real-time. We also compare our IntelliNull, which is based on beamforming plus nulling, with the baseline which is based on beamforming only. Our results demonstrate that IntelliNull outperforms the baseline by effectively mitigating interference, leading to higher throughput and better signal-to-interference-plus-noise ratio (SINR), especially in dense deployment scenarios where beamforming alone fails to sufficiently suppress OBSS interference.