Hybrid Deep Reinforcement Learning and Convex Optimization for RIS-Assisted Cell-Free NOMA Networks

Abstract

Cell-free networks are a promising technology for 6G mobile systems in which all active access points (APs) coherently convey user signals to increase diversity and improve user performance. The integration of reconfigurable intelligent surfaces (RISs) and non-orthogonal multiple access (NOMA) further enhances spectral and energy efficiency by providing auxiliary reflection paths and power-domain multiplexing. However, the joint optimization of AP activation and power allocation constitutes a mixed-integer non-convex problem with an exponentially growing action space, which is intractable for conventional optimization techniques. To address this challenge, we propose a hybrid framework that combines deep reinforcement learning with convex optimization for cell-free NOMA networks with RIS assistance. In this framework, a double deep Q-network (DQN) with prioritized experience replay selectively activates APs and RIS units, whereas second-order cone programming (SOCP) optimally allocates transmission power for each configuration. This decomposition reduces the action space from exponential to linear complexity, and thus make the learning process tractable. Simulation results demonstrate that the proposed mechanism achieves significant power reduction while maintaining higher QoS satisfaction compared to baseline ones.