Module-Pruning-Based Neural Architectural Search for Remote Sensing Image Captioning

Abstract

Remote sensing image captioning (RSIC) has garnered significant attention for enhancing the interpretability of aerial imagery through textual descriptions. Conventional approaches employ convolutional neural networks (CNNs) for visual feature extraction paired with recurrent neural networks (RNNs) or transformers for caption generation. However, these architectures suffer from high complexity and computational costs. While neural architecture search (NAS) via network pruning has been extensively studied, module-based pruning for RSIC systems remains largely unexplored. We propose a novel dedicated decoder pruning methodology for sequential caption generators—a module-based pruning method for end-to-end encoder–decoder architectural adaptation. It features two key innovations: 1) structured pruning of a pre-trained ResNet encoder and transformer encoder–decoder components and 2) a cross-entropy-based caption matching strategy replacing conventional prediction training in the decoder’s final layer. The proposed method enables simultaneously enhancing inference efficiency and reducing storage requirements without compromising performance. As evaluated on the RSICD dataset using CIDEr, ROUGE, METEOR, bilingual evaluation understudy (BLEU), and Sm metrics, our method achieves 42.8% model size reduction while improving accuracy, establishing new benchmarks in efficient RSIC.