Integrated DDPG-PSO energy management systems for enhanced battery cycling and efficient grid utilization

Abstract

Effective energy management is crucial in hybrid energy systems for optimal resource utilization and cost savings. This study integrates Deep Deterministic Policy Gradient (DDPG) with Particle Swarm Optimization (PSO) to enhance exploration and exploitation in the optimization process, aiming to improve energy resource utilization and reduce costs in hybrid energy systems. The integrated DDPG-PSO approach leverages DDPG's reinforcement learning and PSO's global search capabilities to enhance optimization solution quality. The PSO optimizes the DDPG actor-network parameters, providing a strong initial policy. DDPG then fine-tunes these parameters by interacting with the energy system, making decisions on battery scheduling and grid usage to maximize cost rewards. The results show that the integrated DDPG-PSO EMS outperforms the traditional DDPG in terms of battery scheduling and grid utilization efficiency. Cost evaluations under critical peak tariffs indicate that both EMS algorithms achieved a 34 % cost saving compared to a grid-only system. Under differential grid tariffs, the proposed DDPG-PSO approach achieved a 28 % cost reduction, outperforming the standalone DDPG, which achieved a 25 % saving. Notably, the DDPG-PSO effectively reduced overall grid dependency, yielding a total operational cost of $665.19, compared to $780.70 for the DDPG. resenting a 14.8 % reduction. The battery charge/discharge profiles further highlight the advantages of the DDPG-PSO strategy. It demonstrated more stable and efficient energy flow behavior, characterized by shallow cycling and partial discharges sustained over several hours. In contrast, the DDPG exhibited more aggressive deep cycling, fluctuating frequently between minimum and maximum charge levels. This improved energy flow management by DDPG-PSO not only reduces wear on the battery system but also promotes long-term sustainability and reliability in hybrid energy management.