Uncertainty-Aware Dynamic Fusion Network with Criss-Cross Attention for multimodal remote sensing land cover classification

Abstract

Multimodal Remote Sensing Land Cover Classification (LCC) is a promising technology that integrates multi-source remote sensing data to improve classification accuracy and robustness. However, existing multimodal fusion methods are primarily static, failing to account for the variability of information in each modality and sample. In this paper, we propose an Uncertainty-aware Dynamic Fusion Network (UDFNet) for multimodal land cover classification to address this issue. UDFNet consists of three modules: a feature extraction and propagation module, a global feature association module based on Criss-Cross Attention (CCA), and a dynamic fusion module that assigns weights to each modality based on their respective uncertainties, measured using energy scores. Extensive experiments conducted on the Berlin, Augsburg, MUUFL, and Trento public datasets show that our proposed method achieves superior performance compared to state-of-the-art approaches. Specifically, UDFNet achieved Overall Accuracy (OA) of 86.38%, 94.56%, 95.08%, and 99.76% on the Berlin, Augsburg, MUUFL, and Trento datasets, respectively, surpassing existing methods by up to 5% in OA. These results highlight the critical role of each module in enhancing classification accuracy and robustness.