A characterization method of terminal ingredients for nonlinear MPC using value-based reinforcement learning

Abstract

The stability of nonlinear model predictive control (MPC) relies significantly on stabilizing factors such as the terminal region and cost. A larger terminal region not only expands the region of attraction for the closed-loop system but also contributes to reducing online computation costs. However, existing methods in the literature often impose limitations on the degrees of freedom available for characterizing terminal ingredients. This limitation arises from the reliance on either a predetermined linear local controller or a preset control Lyapunov function. This paper introduces an innovative approach to terminal ingredient characterization leveraging value-based reinforcement learning (RL). This method provides ample degrees of freedom for expanding the terminal region. To achieve this, a deep neural network is employed to learn the parametric state value function, serving as the terminal cost for MPC. The local controller adopts a one-step MPC instead of a predetermined linear or nonlinear feedback controller. Subsequently, a terminal set sequence is constructed iteratively through the one-step set expansion. The proposed approach’s effectiveness is validated through simulations.