Reinforcement learning based early classification framework for power transformer differential protection

Abstract

The balance between response speed and diagnosis accuracy forms a critical concern in transformer protection. However, prevailing AI-based transformer protection methods tend to adopt fixed data length to extract electrical quantity information, thus impeding prompt responsiveness to situations where discriminative fault features emerge in the early stages. This study formulates transformer protection as a Markov decision process and proposes an Early Classification Proximal Policy Optimization (ECPPO) framework to utilize reinforcement learning (RL) for data-length adaptive transformer protection with timely action and notable high accuracy. However, the limited generalization of RL algorithms poses a significant issue in the transformer protection scenario. While enhancing the feature extraction capability of a model is essential for improving its generalization ability, ECPPO constructs a two-stage training paradigm to augment the policy model accordingly. In the first stage, a multi-task deep learning framework trains a feature-extraction module with normalization layers employing fault label information and a signal reconstruction task to enrich the feature representation. In the second stage, the pre-trained feature-extraction module is transferred to the agent model with frozen weights, and PPO training is performed. Additionally, to improve the utilization efficiency of samples, a period-circle-shift data augmentation method is proposed, which enhances the generalization capability by cyclically reconstructing data in periodic sequences. To validate the proposed framework, a series of experiments were conducted using simulation data generated by PSCAD/EMTDC software as the training data and practical data generated by experimental transformer system as the testing data. The experimental results demonstrate a significantly enhanced testing accuracy of 99.19 %, coupled with an average response time of 12.10 ms, indicating that the ECPPO algorithm not only achieves superior accuracy but also effectively reduces the average response time. Furthermore, the results highlight its robust generalization capability when transitioning from simulation to experimental systems.