Optimal control for networked control system with Markovian packet loss and delay

This paper focuses on the optimal output-feedback control and stabilization of a networked control system, where both time delay and packet loss are involved. Different from most previous work, the packet loss is Markovian, and the time delay occurs between the actuator and controller. The difficulty is rooted in the failure of the separation principle. The main tools are solving the delayed forward-backward stochastic difference equations (D-FBSDEs) and convergent analysis. Resorting to solving D-FBSDEs and completing the square, we acquire the analytical solution, including the necessary and sufficient solvability condition and the explicit controller, of the finite-horizon optimal control problem. Based on the finite-horizon result and convergent analysis, we then obtain the analytical solution of the infinite-horizon optimal control and stabilization problems. In addition, by analyzing the convergence of the optimal estimator, we establish a constraint relationship between the packet loss probability $q$ and the system matrix $A$. The results are evaluated by numerical examples.