Adaptive PPO With Multi-Armed Bandit Clipping and Meta-Control for Robust Power Grid Operation Under Adversarial Attacks

Abstract

The seamless and resilient operation of power grids is crucial for ensuring a reliable electricity supply. However, maintaining high operational stability is increasingly challenging due to evolving grid complexities and potential adversarial threats. This paper proposes a novel composite enhanced proximal policy optimization (CePPO) algorithm to improve power grid operation under adversarial conditions. Specifically, our approach introduces three key innovations: 1) multi-armed bandit (MAB) mechanism for dynamic epsilon-clipping that adaptively adjusts exploration-exploitation trade-offs; 2) meta-controller framework that automatically tunes hyperparameters including the activation learning rate (ALR) penalties and exploration factors; and 3) integrated gradient-based optimization approach that combines policy gradients with environmental feedback. The effectiveness of the proposed model on the IEEE 14-bus system demonstrates that the CePPO achieves approximately 50% higher average rewards and 51% longer stability periods compared to standard PPO while reducing computational overhead by 35%. CePPO demonstrates superior performance under adversarial attacks compared to baseline approaches. The simulation results validate that CePPO’s adaptive parameter tuning and enhanced exploration strategies make it particularly well-suited for the dynamic nature of power grid control. To foster further research and reproducibility, the code is available upon request at https://github.com/Dr-Kate-Davis-s-Research-Team/DRL-CP.S