An Intra- and Inter-Modality Fusion Model with Invariant- and Specific-Constraints Using MR Images for Prediction of Glioma Isocitrate Dehydrogenase Mutation Status

In the 2021 World Health Organization classification of gliomas, it is proposed that Isocitrate Dehydrogenase (IDH) plays a key role. The prognosis of glioma is largely affected by IDH mutation status. Therefore, IDH mutation status needs to be predicted in advance before surgery. In the past decade, with the development of machine learning, more and more machine learning methods, especially deep learning methods, have been applied to the development of computer-aided diagnosis systems. At present, in this field, many deep learning and radiomics based methods have been proposed for IDH prediction using multimodal Magnetic Resonance Imaging (MRI). In this study, we proposed an intra- and inter-modality fusion model with invariant- and specific- constraints to improve the performance of IDH status prediction. First, MRI-based radiomics features were fused with deep learning features in each modality (intra-modality fusion) and then the features extracted from each modality of brain MRI were fused by using an inter-modality fusion model with invariant and specific constraints. We experimented our proposed method on the dataset provided by the Affiliated Hospital of Zhengzhou University in Zhengzhou, China and demonstrated the effectiveness of the proposed method. In our study, we propose two inter-modality fusion models, and our experimental results show that our best proposed method outperformed state-of-the-art methods with an accuracy of 0.79, precision of 0.80, recall of 0.75, and F1 score of 0.78. Thus, we predicted the IDH mutation status for glioma treatment with a 2% increase in accuracy and 4% increase in precision to predict the IDH mutation status for glioma treatment.