USLC: Universal self-learning control via physical performance policy-optimization neural network

This article proposes an online universal self-learning control (USLC) algorithm based on a physical performance policy-optimization neural network, which aims to solve the problem of universal self-learning optimal control laws for nonlinear systems with various uncertain dynamics. As a key system characterization, this algorithm predicts the discrepancy between the optimal and current control laws by evaluating overall performance in each iterative learning cycle, leveraging an offline-trained universal policy network. This approach is universal, as it does not rely on an exact system model and can adaptively control performance preferences across various tasks by customizing the physical performance cost weights. Using the established control law-performance surface and contraction Lyapunov function, the necessary assumptions and proofs for the stable convergence of the system within a three-dimensional manifold space are provided. To demonstrate the universality of USLC, simulation experiments are conducted on two different systems: a low-order circuit system and a high-order variable-span aircraft attitude control system. The stable control achieved under varying initial values and boundary conditions in each system illustrates the effectiveness of the proposed method. Finally, the limitations of this study are discussed.