Reinforcement Learning-Based Injection Schedules for CO2 Geological Storage Under Operation Constraints

Abstract

This study develops an advanced deep reinforcement learning framework utilizing the Advantage Actor–Critic (A2C) algorithm to optimize periodic CO₂ injection scheduling with a focus on both containment and injectivity. The A2C algorithm identifies optimal injection strategies that maximize the CO₂ injection volume while adhering to fault-pressure constraints, thereby reducing the risk of fault activation and leakage. Through interactions with a dynamic 3D geological model, the algorithm selects actions from a continuous space and evaluates them using a reward system that balances injection efficiency with operational safety. The proposed reinforcement learning approach outperforms constant-rate strategies, achieving 22.3% greater CO₂ injection volumes over a 16-year period while maintaining fault stability at a given activation pressure, even without incorporating geomechanical modeling. The framework effectively accounts for subsurface uncertainties, demonstrating robustness and adaptability across various fault locations. The proposed method is expected to serve as a valuable tool for optimizing CO₂ geological storage that can be applied in complex subsurface operations under uncertain conditions.