Conservative Q-learning with physics guidance for generator tripping control

Abstract

Maintaining transient stability following large disturbances is a critical challenge in modern power systems with high renewable energy penetration. Generator tripping control serves as an essential emergency strategy to mitigate rotor angle instability. However, conventional reinforcement learning approaches lack physical interpretability and robustness, limiting their operational acceptance. To address these concerns, this paper proposes a novel Physics-Informed Conservative Q-learning framework for emergency generator tripping. Specifically, the proposed approach integrates physics-informed neural networks (PINN) with conservative Q-learning (CQL) in a modular framework. A two-layer long short-term memory-based PINN, independently trained to predict generator rotor angle trajectories by embedding the governing swing equations to ensure physical consistency. Meanwhile, a convolutional neural network-based CQL agent is employed to learn robust generator tripping control policies, where the PINN outputs are incorporated as auxiliary dynamic features to enhance learning stability and safety. An action masking mechanism guided by physics-informed trajectory clustering further improves policy robustness by restricting decisions to critical generators. The proposed method is validated on a modified IEEE-39 bus system under multiple fault scenarios with different levels of renewable energy integration. Furthermore, to demonstrate scalability and generalization, additional validation is performed on the larger and more complex IEEE-118 bus system. Results from both testbeds show that the proposed approach significantly improves training efficiency, enhances transient control performance, ensures stable deployment behavior, and reduces generator tripping costs.