Decomposition of large-scale MDPs for wireless scheduling with load- and channel-awareness

Scheduling delay-tolerant tasks based on both load-and channel-awareness can significantly reduce the peak demand in cellular networks. However, solving the optimal scheduling problem leads to a large-scale Markov Decision Process (MDP) with extremely high complexity. In this work, we propose a scalable and distributed approach to this problem, called Coordinated Scheduling (CoSchd). CoSchd decomposes the large-scale MDP problem into many individual MDP problems, each of which can be solved independently by each user under a limited amount of coordination signal from the BS. We show that CoSchd is close to optimal when the number of users becomes large. Further, we propose an online version of CoSchd that iteratively updates the scheduling policy based on online measurements. Simulation results demonstrate that exploiting load- and channel-awareness with CoSchd can effectively alleviate cellular network congestion.