Multi-Task Deep Reinforcement Learning with Scenario Clustering for Real-Time Scheduling of Wind-Solar-Hydro Complementary Generation Systems

Abstract

Real-time scheduling of wind-solar-hydro complementary power generation systems (WSHCPGS) is crucial for enhancing energy utilization efficiency and power supply quality. However, WSHCPGS encounter challenges stemming from the complexity of multi-energy coupling systems and the inherent uncertainty of renewable energy sources. Traditional scheduling methods struggle to quickly and accurately adapt to the dynamic environment. Therefore, this paper proposes a multi-task deep reinforcement learning (DRL) method with scenario clustering for real-time scheduling of WSHCPGS. Conventional single-task DRL methods suffer from low learning efficiency and insufficient generalization ability. Their scheduling strategies may not be robust enough when facing uncertain environments. To address these challenges, this paper divides typical scenarios using the t-distributed stochastic neighbor embedding (t-SNE) and density-based spatial clustering of applications with noise (DBSCAN) methods and identifies scenario categories based on the stacking ensemble learning (SEL) algorithm. Then, a multi-task soft actor–critic (MTSAC) algorithm is proposed for real-time scheduling. The proposed method enables targeted training for specific scenarios to ensure the optimality of scheduling strategies. Simulation results indicate that the multi-task method can handle uncertainty better and converge faster than conventional single-task DRL algorithms. Unlike single-task DRL methods, MTSAC with scenario clustering enhances adaptability and robustness. Furthermore, compared to traditional methods such as model predictive control (MPC) and particle swarm optimization (PSO), the proposed method achieves significant increases of 6.92% and 30.21% in reservoir energy storage, all while maintaining decision-making times below 0.1 s.