MUFNet: Toward Semantic Segmentation of Multi-spectral Remote Sensing Images

Abstract

In this paper, a new convolutional neural network called multi-U fusion networks (MUFNet) is proposed for accurate semantic segmentation of multi-spectral remote sensing. Essentially, MUFNet is inspired by UNet, MFNet and CAM and fully combines their advantages. First, MUFNet introduces the skip connections into a multi-encoder-to-mono-decoder architecture, thereby facilitating the fusion of multi-scale and multi-channel spectral information. Second, the shortcut module in the decoder is revised by concatenating multiple spectral features from different encoders and then feeding the concatenated data into a CAM unit. Thus, the multi-spectral context semantics are fused and also the redundant feature maps are attention-compressed. Extensive simulations were conducted by testing UNet, UNet-4ch, MFNet and MUFNet on the 8400 RGB-NIR multi-spectral images with five categories from the GID image dataset. The visual results clearly showed that the proposed MUFNet can achieve more smoothing and complete segmentation performance than the other networks. Moreover, the measure values of mIoU, FWIoU and PA indicate that the proposed MUFNet can outperform the other networks in average semantic segmentation accuracy.