Timely-throughput Optimal Scheduling for Wireless Flows with Deep Reinforcement Learning

Abstract

This paper addresses the problem of scheduling real-time wireless flows under dynamic network conditions and general traffic patterns. The objective is to maximize the fraction of packets of each flow to be delivered within their deadlines, referred to as timely-throughput. The scheduling problem under restrictive frame-based traffic models or greedy maximal scheduling schemes like LDF has been extensively studied so far, but scheduling algorithms to provide deadline guarantees on packet delivery for general traffic under dynamic network conditions are very limited. We propose two scheduling algorithms using deep reinforcement learning approach to optimize timely-throughput for general traffic in dynamic wireless networks: RL-Centralized scheduling algorithm and RL-Decentralized scheduling algo-rithm. Specifically, we formulate the centralized scheduling problem as a Markov Decision Process (MDP) and a multi-environments double deep Q-network (ME-DDQN) structure is proposed to adapt to the dynamic network conditions. The decentralized scheduling problem is formulated as a Partially Observable Markov Decision Process (POMDP) and an expert-apprentice centralized training and decentralized execution (EA-CTDE) structure is designed to accelerate the training speed and achieve the optimal timely-throughput. The extensive results show that the proposed scheduling algorithms converge fast and adapt well to network dynamics with superior performance compared to baseline policies. Finally, experimental tests confirm simulation results and also show that the proposed algorithms are feasible in practice on resource limited platforms.