HRVM-UNet: Dual-path vision mamba U-Net with frequency-aware skip fusion for high-resolution remote sensing semantic segmentation

Abstract

High-resolution remote sensing semantic segmentation requires simultaneously modeling long-range spatial dependencies and preserving fine-grained boundaries. Convolutional networks are efficient but often struggle to capture global context, whereas Transformers incur quadratic self-attention cost on large images. To address these issues, we propose HRVM-UNet, an asymmetric encoder–decoder segmentation framework built upon Vision Mamba (VMamba). HRVM-UNet introduces (i) a dual-path HR-VSS block that couples a selective-scan state-space global path with a multi-scale dilated-convolution local path, enabling complementary global–local representation learning; and (ii) a Frequency-Aware Skip Fusion (FASF) module that formulates skip integration as spatial–frequency coupling: CARAFE-style content-adaptive upsampling restores structural consistency, DCT-based multi-spectral channel attention emphasizes boundary and texture cues, and a lightweight gate adaptively balances encoder–decoder information. In addition, a top-down feature pyramid is employed to enhance multi-scale representations, and the final decoding stage is strengthened with stacked HR-VSS blocks and coordinate attention for refined spatial localization. Experiments on three public datasets (ISPRS Vaihingen, ISPRS Potsdam, and LoveDA) demonstrate that HRVM-UNet consistently improves segmentation performance and produces sharper object boundaries compared with strong CNN-, Transformer-, and Mamba-based baselines, and our per-class and ablation analyses attribute the gains to the proposed global–local modeling and frequency-aware fusion strategy.