Semantic Segmentation of Multimodal Optical and SAR Images With Multiscale Attention Network

The joint semantic segmentation of multimodal remote sensing (RS) images can make up for the problem of insufficient features of single-modal images and effectively improve classification accuracy. Some deep learning methods have achieved good performance, but they face problems such as complex network structure, large number of parameters, and deployment difficulty. In this letter, more attention is paid to front-end and branch-level feature transformation to obtain multiscale semantic information. The multiscale dilated extraction module (MDEM) is constructed to mine the specific features of different modalities. The multimodal complementary attention module (MCAM) is designed for further acquiring prominent complementary content. The concatenated features are transmitted and reused by the dense convolution to complete the encoding. Ultimately, a general and concise end-to-end model is proposed. Comparative experiments are carried out on three heterogeneous datasets, and the model put forward performs well in qualitative analysis, quantitative comparison, and visual effect. Meanwhile, the dexterity and practicability of the model are more prominent, which can provide support for lightweight design and hardware deployment.