Deep deterministic policy gradient reinforcement learning based temperature control of a fermentation bioreactor for ethanol production

Abstract

Manufacturing of ethanol is an essential industrial bioprocess for energy production. The primary objective of this research is to control the temperature of the fermentation reactor to produce ethanol considering process (bioreactor) complexity, model uncertainty and sluggish response. Sophisticated model-based controllers face hurdles stemming from the necessity for precise models, demanding computationally intensive algorithms for planning and optimization, and susceptibility to uncertainties and process - model mismatch. However, a sub class of machine learning called reinforcement learning (RL) can assist by enabling agents to learn policies directly from their plant or environment. In this study, the efficacy of using a deep deterministic policy gradient (DDPG), an RL method for control of a bioreactor is ascertained. For applications requiring in-process control, DDPG is the appropriate choice due to the advantages of continuous state-action spaces. The continuous bioreactor act as an environment and it is modelled using MATLAB - 2023a. The DDPG agent with a reward function is tested on non-linear bio reactor model. For an even comparison, the results are compared with an another existing popular RL algorithm viz., deep Q-learning network (DQN) algorithm. The DDPG agent gives better results as compared to DQN in terms of integral squared error (ISE) and these act as performance indices. The DDPG trained agent successfully rejects the disturbances such as change in input flow (F_in), inlet temperature (T_in), and its performance is evaluated in terms of ISE.