Deep Reinforcement Learning for Intervention of Partially Observable Regulatory Networks.

Abstract

This paper presents a deep reinforcement learning framework for designing optimal intervention policies in Gene Regulatory Networks (GRNs) under partial observability. Existing methods often assume full observability of the system states, which is unrealistic in practice due to incomplete or noisy gene expression data. To address these limitations, we extend Boolean network models to include partial observability. The uncertainty in gene expression data and stochasticity in gene activities impacted by interventions are captured through the posterior distribution of states, called the belief state. We formulate the optimal intervention policy over the belief space, maximizing long-term rewards by reducing harmful gene activations while accounting for system and data uncertainties. Deep reinforcement learning, particularly deep Q-network (DQN), is developed to enable approximation of the optimal intervention policy at scale. Our analytical results demonstrate that the method converges to the optimal dynamic programming solution if the uncertainty in the gene state disappears. Numerical experiments on a melanoma gene regulatory network demonstrate the efficacy of the proposed approach, showing improved performance compared to existing methods in maintaining desirable system states and reducing the activation of cancer-related genes.