Multimodal deep learning method based on multiple spectra for lung cancer early diagnosis

Lung cancer remains one of the most lethal malignancies worldwide, and the early and accurate diagnostic is critical. Traditional diagnostic techniques such as imaging and histopathology often suffer from limitations including high cost, radiation exposure, and reliance on expert experience. In this study, a multimodal deep learning method based on multiple spectra is proposed for lung cancer detection. The method integrates four common spectra including Fourier transform infrared spectra, UV–vis absorbance spectra, fluorescence spectra, and Raman spectra into a unified detection framework. Specifically, every spectrum sample is represented by its one-dimensional (1D) sequence and two-dimensional (2D) Gramian Angular Summation Field (GASF) image. A dual-branch architecture was designed to capture 1D patterns and 2D features. These complementary features are fused through a MambaVision-based fusion module to achieve efficient cross-modal interaction and global context modeling. The proposed method achieved outstanding performance with accuracy of 97.65 %, precision of 98.14 %, recall of 97.52 %, F1-score of 97.82 %, and AUC of 99.76 %. Moreover, the comparison and ablation experiments confirm the superiority of the training strategy and the proposed method. Interpretability analysis based on T-SNE and Class Activation Mapping proves that the model focuses on meaningful spectral regions in the detection process. This work demonstrates a promising paradigm for spectral-based intelligent diagnosis.