An Index Policy for Dynamic Fading-Channel Allocation to Heterogeneous Mobile Users with Partial Observations

Abstract

This paper addresses a system model where multiple heterogeneous mobile users share a common wireless channel to a base station, extending work of Koole et al. (2001) which considered the corresponding homogeneous-user case. The channel's bandwidth is to be dynamically allocated to different users, based on partial information on their connectivity status. During each time slot, a user may or may not be connected to the base station, where the Gilbert-Elliott model is assumed that a user's connectivity evolves as a two-state (on-off) Markov chain. Transmitting from/to a user reveals its true connectivity status, while those of other users are only partially known by tracking their probabilities of being connected. The goal is to design a tractable dynamic channel allocation policy that comes close to maximizing the infinite-horizon discounted or long-run average value of the through put minus transmission costs. The paper exploits a restless bandit problem formulation drawing on and extending to the Partially Observed Markov Decision Process (POMDP) setting the powerful indexation theory introduced by Whittle (1988) and developed by the author, to obtain a new dynamic priority-index policy that is readily implementable. Computational results are presented showing that the proposed policy can substantially outperform the conventional greedy policy in instances with two heterogeneous users.