Deep association analysis framework with multi-modal attention fusion for brain imaging genetics

Abstract

Brain imaging genetics is a crucial technique that integrates analysis of genetic variation and imaging quantitative traits to provide new insights into genetic mechanisms and phenotypic characteristics of the brain. With the advancement of medical imaging technology, correlation analysis between multi-modal imaging and genetic data has gradually gained widespread attention. However, existing methods usually employ simple concatenation to combine multi-modal imaging features, overlooking the interaction and complementary information between modalities. Moreover, traditional correlation analysis is used for the joint study of phenotypic and genotypic, resulting in an incomplete exploration of the complex intrinsic associations between them. Therefore, in this paper, a deep association analysis framework with multi-modal attention fusion (DAAMAF) is proposed for the early diagnosis of Alzheimer’s disease (AD). First, multi-modal feature representations are extracted from the imaging genetics data to achieve nonlinear mapping and obtain enriched information. Then, we design a cross-modal attention network to learn the interaction between multi-modal imaging features for better utilizing their complementary roles in disease diagnosis. Genetic information is mapped onto the imaging representation through a generative network to capture the complicated intrinsic associations between neuroimaging and genetics. Finally, the diagnostic module is utilized for performance analysis and disease-related biomarkers detection. Experiments on the AD Neuroimaging Initiative dataset demonstrate that DAAMAF displays superior performance and discovers biomarkers associated with AD, promising to make a significant contribution to understanding the pathogenesis of the disease. The codes are publicly available at https://github.com/Yeah123456ye/DAAMAF.