DC-MSSFF Net: Dule-channel multi-scale spatial-spectral feature fusion network for cholangiocarcinoma pathology high-resolution hyperspectral image segmentation

Abstract

Background and Objective:

High-precision segmentation of pathological images is a challenging task in the field of medical image processing. Hyperspectral microscopic imaging offers a distinct advantage in histopathological image segmentation due to its abundance of spectral and spatial data.

Methods:

Here, a Dule-Channel Multi-Scale Spatial-Spectral Feature Fusion Network (DC-MSSFF Net) is proposed for semantic segmentation of cholangiocarcinoma hyperspectral images (HSI). The DC-MSSFF Net is composed of two parallel channels, graph-within-graph (GwG) and multi-scale CNN. The GwG can greatly reduce the computational burden while establishing the spatial context relationship of the HSI image. The multi-scale CNN channel is able to fine-tune the segmented edges of the HSI images at the pixel-level based on hyperspectral information in the depth dimension. Afterwards, the segmentation results are achieved by fusing the features from the two channels. Furthermore, an ensemble-based framework is applied to further improve the performance of the model.

Results:

The image segmentation evaluation indexes such as dice similarity coefficient (Dice) of the Cholangiocarcinoma HSI data can reach 70.47, which is much higher than the SOTA method and RGB-based image segmentation methods.

Conclusion:

The superior performance of the DC-MSSFF network pioneers the inductive learning task of deep frameworks for semantic segmentation of high-resolution hyperspectral image (HR-HSI).