CMKD-Net: a cross-modal knowledge distillation method for remote sensing image classification

Abstract

Cross-modal knowledge distillation (KD) offers the potential to synergize the strengths of Vision Transformers (ViTs) and Convolutional Neural Networks (CNNs) in remote sensing image (RSI) classification. However, existing KD techniques in this field are frequently ineffective and time-consuming. We contend that this inefficiency stems from the data sparsity inherent in RSI samples—a challenge long overlooked in previous studies. To address this issue, we propose a novel algorithm designed to alleviate data sparsity and enhance the quality of the training data. Building upon this, we introduce CMKD-Net, a KD framework that facilitates knowledge transfer from a ViT teacher to a CNN student model. Experimental evaluations on three RSI datasets demonstrate that CMKD-Net outperforms 17 state-of-the-art models published since 2022 on classification accuracy and model compactness. Furthermore, our method cuts down training time by at least 83% compared to current KD methods, making cross-modal KD for RSI classification much more effective.