Multimodal brain tumor segmentation and classification based on optimized DeepLabV3 + and fused fire module with self-attention.

Abstract

In this work, we propose a novel deep learning architecture for brain tumor segmentation and classification, SMDeepNet, which is based on an optimized DeepLabV3 +  + and a Fused Fire Module with Self-Attention. The segmentation framework comprises down- and up-sampling based on a DeepLabV3 + neural network and is optimized by dynamically initializing hyperparameters for training the backbone ResNet-50 architecture. During the down-sampling or encoder stage, the Atrous Spatial Pyramid Pooling (ASPP) module extracted features using convolutional layers with various filter sizes and dilations. These features are then passed to the up-sampling or decoder section for final segmentation. The classification framework comprises two sub-frameworks: a fire-residual bottleneck (Fire-RB) and a Hybrid Efficient Attention (Hybrid-EA). The Fire-RB framework comprises several parallel blocks: one side implements squeeze-and-expand behavior in the fire mechanism, and the other implements residual bottlenecks. The two parallel blocks are concatenated, and features are extracted from Fire-RB. The Hybrid-EA model is a custom variant of the pre-trained EfficientNetB0 model that incorporates a self-attention mechanism. The self-attention mechanism enhanced the functionality of the EfficientNetB0 model. Features from Fire-RB and Hybrid-EA are concatenated channel-wise, and the final modality classification is performed. The BraTS 2023 dataset is used in this work to evaluate the proposed methodologies. Segmentation results indicate that accuracy, Dice Score, and Intersection over Union (IOU) are 0.9871, 0.9420, and 0.8951, respectively. Modality classification results indicate an accuracy of 0.9920, which is improved over the recent state-of-the-art techniques.