Optimization of start-up strategies of gas injection compressor in underground gas storage using deep reinforcement learning

Abstract

Injection compressors, as the core equipment in the gas injection process of underground gas storage (UGS) facilities, play a vital role in ensuring the safe and efficient operation of UGS systems. However, traditional optimization methods often struggle to adapt dynamically under complex operating conditions and may lead to excessive energy consumption. To address these challenges, this study proposes a deep reinforcement learning (DRL)-based approach to optimize compressor start-up strategies. First, a high-fidelity hybrid simulation model is developed by integrating thermodynamic equations of reciprocating compressors with a residual correction network based on a multilayer perceptron, forming a Mechanism-Data fusion Model framework. This model achieves prediction errors of <5 % for power and <3 % for discharge flow rate. Based on the accurate simulation model, an optimization framework is constructed using the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. Within this framework, continuous control variables—such as the number of operating compressors, inlet throttling levels, and relative clearance volume adjustments—are mapped to the action space of the reinforcement learning agent. A multi-objective reward function is designed to incorporate penalties for gas injection deviations, the number of active compressors, inlet pressure constraints, and clearance volume limits. By introducing delayed updates to the target network and applying an adaptive noise clipping mechanism, the proposed strategy ensures optimal parameter control across the entire gas injection cycle while satisfying operational and safety requirements. Experimental results demonstrate that the proposed method reduces compressor energy consumption by 5.18 %, offering a precise, adaptive, and intelligent decision-making solution for dynamic optimization of UGS compressor operations.