QBD Modelling of a finite state controller for queueing systems with unobservable Markovian environments

Abstract

We address the problem of stabilizing control for complex queueing systems with known parameters but unobservable Markovian random environment. In such systems, the controller needs to assign servers to queues without having full information about the servers' states. A control challenge is to devise a policy that matches servers to queues in a way that takes state estimates into account. Maximally attainable stability regions are non-trivial. To handle these situations, we model the system under given decision rules. The model is using Quasi-Birth-and-Death (QBD) structure to find a matrix analytic expression for the stability bound. We use this formulation to illustrate how the stability region grows as the number of controller belief states increases.