Decoding the brain via multi-view brain topology contrastive learning

Abstract

Recently, Graph Neural Networks (GNNs) have been widely used in neural decoding due to strong topological feature mining and interpretability. GNNs are heavily based on manually defined brain topology; if there are false connections or noise, it will greatly affect the decoding performance. To address the aforementioned challenges, a series of GNN-based graph topology learning (GTL) methods have received widespread attention due to their ability to automatically optimize brain topology. However, existing GTL methods are usually implemented in a supervised manner and rely on a large amount of annotated data, making it difficult to directly transfer them to different decoding scenarios. Therefore, in this paper, a Brain Topology Inference framework based on Multi-View Contrastive Self-supervised Learning (BTI-MVCSL) is proposed for neural decoding. Specifically, BTI-MVCSL first designs a series of graph learners, which can infer brain topological connections as “learner”, generate topology learning objectives as “instructor” from the original fMRI data, and maximize consistency between “instructor” and “learner” to extract the rich information in hidden connections. Furthermore, in order to achieve fully automated topology learning guidance, BTI-MVCSL develops a new self-learning mechanism that can use the “learner”-view brain topology to update the “instructor”-view brain topology during model optimization and further achieves comparative constraints through the “instructor” topology. The proposed BTI-MVCSL has been extensively evaluated in two publicly available fMRI datasets, demonstrating superior performance and revealing potential changes in brain topology under different decoding tasks.