Multi-modal orthogonal fusion network via cross-layer guidance for Alzheimer’s disease diagnosis

Abstract

Multi-modal neuroimaging techniques are widely employed for the accurate diagnosis of Alzheimer’s Disease (AD). Existing fusion methods typically focus on capturing semantic correlations between modalities through feature-level interactions. However, they fail to suppress redundant cross-modal information, resulting in sub-optimal multi-modal representation. Moreover, these methods ignore subject-specific differences in modality contributions. To address these challenges, we propose a novel Multi-modal Orthogonal Fusion Network via cross-layer guidance (MOFNet) to effectively fuse multi-modal information for AD diagnosis. We first design a Cross-layer Guidance Interaction module (CGI), leveraging high-level features to guide the learning of low-level features, thereby enhancing the fine-grained representations on disease-relevant regions. Then, we introduce a Multi-modal Orthogonal Compensation module (MOC) to realize bidirectional interaction between modalities. MOC encourages each modality to compensate for its limitations by learning orthogonal components from other modalities. Finally, a Feature Enhancement Fusion module (FEF) is developed to adaptively fuse multi-modal features based on the contributions of different modalities. Extensive experiments on the ADNI dataset demonstrate that MOFNet achieves superior performance in AD classification tasks.