Adaptive multi-modal temporal fusion network with dynamic synergistic integration for breast cancer survival prediction

Abstract

Breast cancer, as the malignant tumour with the highest incidence rate in women, faces severe challenges in survival prognosis prediction due to its molecular heterogeneity. Currently, multi-modal deep learning-based prediction methods suffer from sample category imbalance, insufficient cross-modal characterization, and defective static fusion strategies. To address these issues, we propose the adaptive multi-modal temporal fusion network (AMTFN). Firstly, an adaptive weighted sample generation mechanism is designed to alleviate the category imbalance by dynamically adjusting the synthesis strategy, which significantly improves the prediction accuracy. Secondly, a CNN-BiLSTM-BiGRU feature extraction network was constructed to extract gene expression data, CNA, and clinical features, respectively, to enhance the cross-modal collaborative characterization. Then, a hierarchical dynamic modal fusion method is proposed to enhance the embedding representation using gating units, and residual fusion is achieved by Transformer encoding with dynamic weight calibration. Finally, in the classification stage, a dynamic synergetic integration mechanism is proposed to enhance the generalization capability through multi-classifier interaction optimization. The experiments show that AMTFN outperforms the comparison method on the METABRIC dataset in several metrics, in which the AUC reaches 97.26%. In addition, validation on the TCGA-BRCA dataset further demonstrates the robustness and generalization ability of AMTFN. The source code can be downloaded from Github: (https://github.com/Xue-U/AMTFN).