Stochastic optimal control problems of discrete‐time Markov jump systems

In this paper, we consider the indefinite stochastic optimal control problems of discrete‐time Markov jump linear systems. Firstly, we establish the new stochastic maximum principle, and by solving the forward‐backward stochastic difference equations with Markov jump (FBSDEs‐MJ), we derive the necessary and sufficient solvability condition of the indefinite control problem with non‐discounted cost, which is in an explicit analytical expression. Then, the optimal control is designed by a series of coupled generalized Riccati difference equations with Markov jump (GRDEs‐MJ) and linear recursive equations with Markov jump (LREs‐MJ). Moreover, based on the non‐discounted cost case, we deduce the optimal control problem with discounted cost. Finally, a numerical example for defined‐benefit (DB) pension fund with regime switching is exploited to illustrate the validity of the obtained results.