Explainable artificial intelligence for deep learning-based model predictive controllers

Model predictive control (MPC) has been established in a wide range of control applications as the standard approach. But applying MPC requires solving a potentially complex optimization problem online to generate a new control input signal. To avoid the expensive online computations, deep learning-based MPC has been developed, in which neural networks imitate the behavior of the MPC. When such a data-driven approximate controller is derived, there is no straightforward way to trace the reasons for its proposed actions back to its inputs, hence making the controller a black-box model. In this paper, we propose the use of SHAP, an explainable artifical intelligence technique, to generate insights from learning-based MPC for the purpose of model debugging and simplification. Our results show that SHAP can explain general control behaviors and can also support model simplification in an informed way, representing a better alternative to dimensionality reduction techniques such as principal component analysis.