Convex optimization formulation of density upper bound constraints in Markov chain synthesis

Abstract

This paper introduces a new approach for the synthesis of Markov chains with density upper bound constraints. The proposed approach is based on a new mathematical result that formulates the density upper bound constraints, known also as safety constraints, as linear, and hence convex, inequality constraints. It is proved that the new convex constraints are equivalent, necessary and sufficient, to the density upper bound constraints, which is the main contribution. Next, this result enabled the formulation of the Markov chain synthesis problem as an Linear Matrix Inequality (LMI) optimization problem with additional constraints on the steady state probability distribution, ergodicity, and state transitions. The LMI formulation presents an equivalent design formulation in the case of reversible Markov chains, that is, it is not conservative. When reversibility assumption is relaxed, the LMI condition is only sufficient due to the ergodicity constraint, i.e., it is conservative. Since LMI problems can be solved to global optimality in polynomial time by using interior point method (IPM) algorithms of convex optimization, the proposed LMI-based design approach is numerically tractable.