From titer to quality: Exploring reinforcement learning for bioprocess control in silico

Abstract

The production of monoclonal antibodies in mammalian cells is a highly complex and nonlinear process. The industry standard for controlling this process fails to capture its complex dynamics, leading to batch-to-batch variability. This inherent complexity makes bioprocesses challenging to model purely mechanistically, while the lack of rich experimental datasets and the need for interpretability in control policies further prevent the use of fully data-driven solutions. We propose a hybrid methodology for optimising the nutrient feeding strategy that leverages Reinforcement Learning (RL) with mechanistic models of cellular metabolism and glycosylation. The RL agent is trained using an off-policy method for data efficiency and is capable of learning from partial observations of the state, which allows for improved generalization. The controller is adaptable to processes with or without additional product quality considerations, such as glycosylation. We demonstrate that accounting for product glycosylation yields different control strategies whereas neglecting it to focus on titer alone can compromise product quality. The continuous learning abilities of the proposed method ensure adaptability in response to process changes, while the inclusion of a mechanistic model in the environment aids in the interpretability of the learned control actions.