Efficient Optimization in RIS-Assisted UAV System Using Deep Reinforcement Learning for mmWave-NOMA 6G Communications

In the evolving landscape of wireless communications for 5G, 6G, and beyond, the deployment of unmanned aerial vehicles (UAVs) has emerged as a groundbreaking strategy to expand coverage areas due to their flexibility and ease of deployment. Simultaneously, reflecting intelligent surfaces (RISs) have introduced a transformative paradigm aimed at improving key performance metrics, such as average sum-rate and energy efficiency (EE). The seamless integration of advanced technologies, including UAVs, RIS, and nonorthogonal multiple access (NOMA), presents a promising avenue for significantly boosting the performance and efficiency of next-generation communication systems. This study investigates EE maximization for two scenarios in a NOMA-enabled mmWave network: 1) multi-UAV-mounted base stations (BSs) and 2) multi-UAV-mounted distributed RIS. In both cases, each UAV serves a NOMA cluster with imperfect successive interference cancellation (SIC), capturing the impact of hardware impairments in real-world NOMA systems. For each scenario, an optimization problem is formulated to maximize EE by jointly optimizing the beamforming matrix, phase shift matrix, NOMA power allocation, and UAV 3-D placement. The nonconvex problems are tackled using both model-based and model-free deep reinforcement learning (DRL) algorithms under constraints, such as minimum Quality of Service (QoS), beamforming and phase shift limits, and UAV trajectory constraints. The simulation results demonstrate that the proposed DRL algorithms significantly enhance spectral efficiency (SE) and EE, showcasing their suitability for 6G communication systems. Furthermore, a comparative analysis with orthogonal multiple access (OMA) and spatial-division multiple access (SDMA) confirms that NOMA outperforms both techniques, achieving substantial gains in efficiency and performance.