A multi-agent reinforcement learning model incorporating historical operational data for pump station operational scheduling

Abstract

The development of operational scheduling for pump units is a critical focus in pump station management. This study introduces a pump station operation scheduling model based on multi-agent reinforcement learning that integrates historical operational data, addressing inefficiencies, poor generalization, and operational complexity encountered with traditional evolutionary algorithms. Utilizing a graph neural network, the model incorporates historical data and prior knowledge about pump station performance curves to establish a performance computation model for multi-unit operation combinations. The scheduling of multi-unit operations at a pump station is conceptualized as a parallel decision-making problem, incorporating rules aimed at cost reduction, efficiency improvement, and meeting water delivery volume requirements while minimizing operational complexity. A MARL model is developed, taking into account the variability in initial operating conditions to enhance generalization capabilities. The study compares the performance of various reinforcement learning models with evolutionary algorithms. Results indicate that the trained MARL model adapts effectively to dynamic water delivery conditions and exhibits strong generalization capabilities. Compared to actual operational scheduling, it achieves significant savings of over 419,000 units in operational costs and over 390,000 kWh in energy consumption. Furthermore, compared to evolutionary algorithms, the decision-making solutions generated by the reinforcement learning model align more closely with operational logic and are more efficient, achieving a planning speed increase of over 70 times.