Finite-state approximations to constrained Markov decision processes with Borel spaces

Abstract

We consider the finite-state approximation of a discrete-time constrained Markov decision process with compact state space, under the discounted cost criterion. Using the linear programming formulation of the constrained problem, we prove the convergence of the optimal value function of the finite-state model to the optimal value function of the original model. Under further continuity condition on the transition probability of the original model, we also establish a method to compute approximately optimal policies.