MFIFN: A multimodal feature interaction fusion network-based model for Alzheimer’s disease classification

Abstract

The classification of Alzheimer’s disease (AD) and the identification of abnormal connections in brain networks have important research implications. Existing classification methods are mainly based on fMRI and sMRI features of brain regions, ignoring the multimodal fusion information of the brain, which has some limitations. Therefore, in this paper, we utilize the time course (TC) of fMRI decomposition with the complementary information between the two, as well as the interactive fusion with sMRI, to learn the multifaceted representational information of the brain, and propose a multimodal feature interaction fusion network (MFIFN) framework that fuses the brain connectivity and activity information. The framework aims to improve the accuracy of brain disease classification through temporal consistency and the combined use of fMRI and sMRI data. A CNN-AM module was designed to process the TC data to extract the time dependence, with a three-layer GRU providing the interpretability of the model. The TC data were processed by PCA downscaling, and the complementarity with the fMRI data was obtained by cross-using the HAN module to obtain the complementary information of both. The GCN uses the information for feature propagation and learning, and the final decision is obtained by the fully connected layer. The effectiveness of MFIFN was verified on the ADNI dataset, achieving a high classification accuracy of AD and NC (99.3%). The results show that the method proposed in this paper is effective in identifying different brain networks in AD patients, which provides biological interpretability for AD diagnosis.