Meta-learning of pseudo force field generation and estimation for enhancing 3D molecular property prediction

Abstract

Learning Energy-based Model via 3D molecular denoising has been shown to be effective in pretraining the 3D molecular representation. However, existing works carry out denoising in task-agnostic manner, causing inevitable domain gap with the downstream tasks. To overcome this issue, we introduce a task-aware pretraining framework, dubbed Mol-MFFGE, for adapting the energy-based pretraining to downstream tasks in meta-learning approach. In this framework, we design learnable pretraining tasks as generating and estimating pseudo force fields. This is achieved by proposing a learnable noise transformation module to generate the noisy motions of atoms and the model is pretrained to estimate them. These tasks are taken as the auxiliary self-supervised training tasks and learned with the downstream task jointly, formulated as a bi-level meta-learning optimization problem. Based on such an approach, the force field generation and estimation tasks are meta-learned to enhance the downstream tasks for molecular property prediction. Extensive experiments are conducted on three molecular property prediction datasets, and results demonstrate performance improvement over the state-of-the-art 3D molecular pretrained models.