Timely Status Updating over Markov Channels in Downlink Wireless Networks with Stochastic Arrivals

In this paper, we consider a downlink wireless network, where a base station (BS) with stochastic arrivals transmits fresh status updates to different users over Markov channels. To quantity the freshness of information, a recently developed metric, called Age of Information (AoI), is used. We formulate a constrained optimization problem to find the optimal scheduling policy that minimizes the average weighted sum of the AoI at the user and the energy consumption at the BS. Due to the hardness of this problem, we recast it as a Restless Multi-Armed Bandit (RMAB) problem and decouple the problem into multiple sub-problems. After deriving the structural property of the optimal policy for the sub-problem, we establish the indexability of the sub-problem and design an algorithm to compute the Whittle’s index. Finally, we propose a scheduling policy based on the Whittle’s index. Simulation results demonstrate that our proposed scheduling policy performs comparably with the optimal policy.