Fast adaptation of multi-cell NOMA resource allocation via federated meta-reinforcement learning

Abstract

Radio resource allocation in multi-cellular systems, particularly with non-orthogonal multiple access (NOMA), must be carefully optimized based on real-time user and network conditions, such as channel responses, user population, and inter-cell interference patterns, which naturally fluctuate over time. Fixed machine learning models for radio resource allocation often fail to adapt to these dynamic conditions, leading to suboptimal resource allocation. Moreover, such models struggle to handle inputs and outputs of varying dimensions, limiting their scalability and generalization in time-varying resource allocation problems. To address these challenges, we propose a novel multi-cell, multi-subband NOMA radio resource allocation solution that integrates meta-learning and federated learning (FL) with multi-agent reinforcement learning (MARL). Our solution maximizes energy efficiency (EE) by enabling one-shot adaptation to environmental variations and dynamically managing information dimensionality through the instantiation and removal of agents from a pretrained model. Under this framework, power allocation (PA) and subband allocation (SA) are jointly optimized in a two-stage process: the first stage employs a central reinforcement learning (RL) agent to solve the PA subproblem, while the second stage leverages multi-agent meta-RL combined with FL to address the SA subproblem. Evaluation results demonstrate that our solution effectively adapts to dynamic environments, including variations in channel conditions due to path loss and Doppler effects, as well as fluctuations in the user set. Notably, our approach consistently outperforms the benchmark algorithms, highlighting its robustness and superior adaptability.