HR-MM Segformer: Enhancing land use and land cover semantic segmentation through transformer-based multisource remote sensing feature fusion

Abstract

Automatic land use and land cover (LULC) semantic segmentation is constrained by spectral confusion and the semantic gap inherent in independent-branch multisource and multisensor (MM) data fusion architectures. To address these limitations, this study proposes HR-MM SegFormer, a framework incorporating a unified transformer encoder. Unlike conventional dual-stream approaches, the architecture projects heterogeneous high-resolution (HR) optical imagery and MM auxiliary data (multispectral/DSM) into a shared semantic manifold via a feature correction (FC) layer. Additionally, a multimodal cross-attention fusion (MCAF) module dynamically retrieves complementary spectral and geometric contexts guided by HR spatial structures. Experimental evaluations of the HR-MS LULC and HR-DSM Potsdam datasets yield mean intersection over union (mIoU) scores of 88.50 % and 87.84 %, respectively. These results correspond to improvements of 8.89 % and 12.67 % over single-modal baselines, accompanied by a 6.3 % increase in computational overhead. This study substantiates that the unified full-attention paradigm bridges cross-modal disparities, providing an effective solution for fine-grained Earth observations.