Deep reinforcement learning for adaptive control of thermoacoustic instabilities in a lean-premixed methane/hydrogen/air combustor

Thermoacoustic instabilities are a challenge in the design and operation of combustion systems. Addressing this challenge is becoming even more critical with the development of fuel-flexible combustors capable of operating with hydrogen and other sustainable fuel sources. While active control is a well-known method for damping combustion instabilities, identifying appropriate control parameters becomes increasingly complex in the presence of changing fuel composition and operating conditions. In this work, we present a model-free deep reinforcement learning (RL) technique to adaptively tune an active control system. We demonstrate that the RL-based active control system is able to adaptively suppress thermoacoustic instabilities over an extended range of operating conditions with minimal training. The demonstration is performed on a laboratory-scale bluff-body-stabilized premixed methane/hydrogen/air flame, at equivalence ratios ranging from 0.5 to stoichiometric, and with up to 80%_vol hydrogen in the fuel. After training the RL system on a single operating condition, combustion instabilities can be mitigated over the entire operating range of the burner. Extending the training to three additional operating conditions allows the RL control system to fine-tune its policy and further reduce thermoacoustic instabilities, achieving a sixfold reduction in the acoustic source term over most of the operating range. We observe a reduction of up to 40 dB in acoustic pressure over 50% of the operating range. The proposed approach offers a promising path towards more efficient, adaptive control systems for thermoacoustic instabilities, demonstrating the potential of RL to address the operational challenges of fuel-flexible combustion systems.

Novelty and Significance Statement

We show the first experimental demonstration of a reinforcement learning-based control method for thermoacoustic instabilities. The experiments are performed on a laboratory-scale premixed methane/hydrogen/air bluff-body burner, which exhibits strong combustion instabilities over a wide range of operating conditions. Building upon a conventional control system, which utilizes a pressure sensor, an acoustic driver, and a gain- and phase-shift controller, the reinforcement learning-based controller is able to dampen instabilities over the entire operating range. This is achieved while training the controller on a single operating condition. Extending training to a total of four distinct operating conditions further fine-tunes the control policy and yields an additional reduction in the acoustic pressure amplitude. This research illustrates the potential of reinforcement learning for robust control in combustion systems - capable of addressing the challenges of complex combustion physics, adapting to unseen conditions, and merging information from heterogeneous sensors.