AutoRL framework for bioprocess control: Optimizing reward function, architecture, and hyperparameters

Abstract

This study proposes a structured AutoRL framework for the development of deep reinforcement learning (DRL) controllers in chemical process systems. Focusing on the control of a 3$\times$ 3 MIMO yeast fermentation bioreactor, the methodology jointly optimizes three key internal components of the DRL agent: the reward function, the neural network architecture, and the hyperparameters of the algorithm. A parameterizable logistic reward formulation is introduced to encode control objectives, such as steady-state accuracy, minimalization of actuation effort, and control smoothness, into a flexible and tunable structure. A dual loop optimization strategy combines grid search and Bayesian optimization to systematically explore and refine the agent’s design space. The resulting controller achieved average steady-state errors of 0.009 °C for reactor temperature and 0.19 g/L for ethanol concentration, while maintaining smooth and stable behavior under diverse operational scenarios. By formalizing reward design and integrating it with hyperparameter and architecture optimization, this work delivers a AutoRL methodology for DRL-based control, reducing reliance on expert heuristics and enhancing reproducibility in complex bioprocess applications.