Dual-Branch Retrieval Network for Satellite Cloud Image Classification Based on Multilevel Semantic Information

Abstract

The weather system has a profound impact on human activities. Conducting research on satellite cloud image classification can provide critical parameters for weather forecasting, climate analysis, and severe weather detection. However, conventional satellite cloud image classification methods typically neglect higher level semantic constraints and rarely incorporate decision-level adaptive calibration, resulting in confusion among visually similar categories and restricting interpretable, content-based inference. Here, we propose a dual-branch retrieval network with multilevel semantic information (DBR-MSI) to address these gaps. DBR-MSI jointly optimizes high-level semantics (e.g., broad meteorological and surface categories) and low-level semantics (e.g., specific cloud or surface attributes), and we explicitly highlight critical semantic content via a gradient-based attention sharing module. Moreover, a retrieval-based inference approach driven by high-level semantic guidance supports interpretable content reasoning and adaptive decision calibration, which in turn allows the proposed method to deliver enhanced robustness and efficient integration of additional data. Experimental results on two satellite cloud image datasets confirm that DBR-MSI exhibits stronger interpretability and achieves overall accuracy (OA) gains of 1.06% and 0.39% over the best competing methods.