An Energy Management Strategy for Virtual Power Plants Integrating EVs Behaviour Analysis Based on TD3 and DQN

Abstract

An efficient energy management plays a crucial role in enhancing the operational efficiency of distributed energy systems (DERs) and virtual power plants (VPPs). However, existing optimization methods face persistent challenges in coordinating dynamic EV behaviours. High-dimensional state spaces and stochastic demand fluctuations lead to suboptimal performance. This study introduces a dispatch optimization strategy for VPPs that incorporates electric vehicles (EVs) based on the twin delayed deep deterministic policy gradient (TD3) and proposes a behaviour analysis driven by deep Q-network (DQN) to precisely evaluate the charge demands of EVs for the TD3 optimization. The simulation results show that: (1) The proposed TD3-EV achieves a 16.7% cost reduction compared to battery scenario, with an 89.45% reduction in MT output and a 38.5% enhancement in grid interaction. Notably, it exhibits 89.07% and over 100% lower costs than deep deterministic policy gradient (DDPG) and Gurobi; (2) TD3-EV reduces peak unbalance power by 26.1% and 14.3% compared to TD3-Battery and DDPG-EV, highlighting its robustness in energy management; (3) TD3-EV demonstrates 84.71% and 94.39% higher mean reward than TD3-battery and DDPG-EV with more stable convergence.