Revolutionising Battery Energy Storage Systems Energy Management: Dynamic-Aware Solutions With Integrated Quantum Particle Swarm Optimisation and Deep Reinforcement Learning

Abstract

The increasing integration of inverter-based resources (IBRs) into microgrids (MGs) poses considerable challenges for dynamic stability and energy management, primarily due to their variable and uncertain inertia and damping characteristics. Unlike conventional synchronous generators (SGs), IBRs exhibit complex nonlinear behaviour, complicating both mathematical modelling and real-time control. Battery energy storage systems (BESS) are essential for ensuring grid stability, operational efficiency and flexibility. Nevertheless, dynamic-aware energy management of BESS remains insufficiently explored, with current approaches often lacking adaptability to uncertainty and real-time requirements. This paper proposes a two-step hybrid framework combining quantum particle swarm optimisation (Q-PSO) with deep reinforcement learning (DRL). In the first step, Q-PSO efficiently generates an initial solution, significantly reducing computational demands. Subsequently, DRL dynamically refines this solution, effectively managing real-time uncertainties linked to inertia and damping variations. The proposed method addresses the non-Markovian nature of MG dynamics by constraining the DRL action space using the Q-PSO-derived solution, thereby alleviating the curse of dimensionality and enhancing training stability. Furthermore, dynamic constraints on frequency deviations and the rate of change of frequency (RoCoF) are incorporated to maintain robust grid stability during transients. Extensive simulations demonstrate that the proposed dynamic-aware energy management system (EMS) achieves economic efficiency improvements of 52.2% compared to Q-PSO alone and 22.7% compared to DQN alone. Additionally, BESS charging efficiency improves by 39.6% and 22.7%, whereas discharging efficiency increases by 38.3% and 28.25%, respectively, against the same benchmarks.