CMACF: Transformer-based cross-modal attention cross-fusion model for systemic lupus erythematosus diagnosis combining Raman spectroscopy, FTIR spectroscopy, and metabolomics

Abstract

As complex multi-omics data in the medical field tend to be multi-modal. Integrating these multimodal information into novel disease diagnosis models has become challenging. However, previous methods mainly focus on single omics, which cannot effectively capture the contributions between different combinations of multi-omics information. To solve this problem, based on Raman spectroscopy, FTIR spectroscopy, and metabolomics data, this paper proposes a new Cross-modal Cross-fusion network based on the Transformer self-attention mechanism (CMACF). The research focuses on effectively combining the feature patterns of different omics for disease prediction. Specifically, by constructing the Raman-IR, Raman-metabolomic, and IR spectral-metabolomic feature pairs and reasonably focusing on the information of different combination pairs through multiple stages of feature sub-network, attention cross-fusion, bimodal interaction, and sequence interaction feature level fusion, it is interesting to find that the information contribution between different pairs is different. We conducted extensive experiments on the systemic lupus erythematosus multi-omics dataset, and the accuracy and AUC values are as high as 99.44 % and 99.98 %, respectively, with the best classification effect. The results show that CMACF can efficiently fuse multi-omics medical data, provide an efficient baseline for processing medical multimodal data, and analyze the contribution of multi-omics data fusion.