NPEX: Never give up protein exploration with deep reinforcement learning

Abstract

Elucidating unknown structures of proteins, such as metastable states, is critical in designing therapeutic agents. Protein structure exploration has been performed using advanced computational methods, especially molecular dynamics and Markov chain Monte Carlo simulations, which require untenably long calculation times and prior structural knowledge. Here, we developed an innovative method for protein structure determination called never give up protein exploration (NPEX) with deep reinforcement learning. The NPEX method leverages the soft actor-critic algorithm and the intrinsic reward system, effectively adding a bias potential without the need for prior knowledge. To demonstrate the method's effectiveness, we applied it to four models: a double well, a triple well, the alanine dipeptide, and the tryptophan cage. Compared with Markov chain Monte Carlo simulations, NPEX had markedly greater sampling efficiency. The significantly enhanced computational efficiency and lack of prior domain knowledge requirements of the NPEX method will revolutionize protein structure exploration.