Multi-Modal Hybrid Encoding Approach Based on Information Bottleneck for Brain Tumor Grading

Abstract

Grade classification of gliomas is critical in clinical diagnosis and treatment decisions. Although histological images are commonly used for grading and as an important factor in prognostic prediction, their results are prone to inter-observer variability. Recent advancements in molecular genetics have significantly improved tumor classification, but challenges persist in effective feature selection and multi-modal data fusion. This letter proposes a multi-modal hybrid encoding method based on information bottleneck (MHEIB), combining histological images and genetic data to enhance glioma grading. MHEIB effectively fuses multi-modal features through the information bottleneck module and the self-attention mechanism, which compresses and filters the key features and dynamically adjusts the weights of multi-modal features to improve the classification accuracy. Experimental results on The Cancer Genome Atlas (TCGA) glioma dataset demonstrate that MHEIB outperforms several fusion methods in terms of F1-score, AUC, and AP. In particular, MHEIB significantly improved the classification AUC to 89.3% and 83.7% for similar categories of Grades II and III respectively. Overall, the MHEIB method provides an efficient multi-modal data fusion solution for glioma grading.