Deep Reinforcement Learning for Dynamic OCW in UORA

Orthogonal Frequency Division Multiple Access-based Uplink Random Access (OFDMA-UORA) is a significant media access control mechanism in IEEE 802.11ax. An optimized OFDMA random access back-off (OBO) scheme is proposed to improve the performance of both light and heavy load networks. In the process of uplink random access,multiple users compete for multiple channels at the same time and follow an unknown joint Markov model. Users avoid collisions when competing for channels and maximize the throughput of the entire uplink process. The process can be formulated as a partially observable Markov decision process with unknown system dynamics. To this end, we apply the concepts of reinforcement learning and implement a deep q-network (DQN).Based on the original OBO mechanism, the OFDMA contention window size is dynamically decided via a deep reinforcement learning framework. For the proposed Deep Reinforcement Learning (DRL) solution, we design a discrete action agent that accommodates the contention window size by taking the channel and user state into account, e.g. the number of active users, available resources unit, and retries. Simulation results confirmed the advantages of the proposed scheme in throughput, delay, and access rate. This scheme can therefore be adopted in practical 802.11ax use cases to improve the network performance.