Multi-Label Auroral Image Classification Based on CNN and Transformer

Abstract

Auroral image classification has long been a focus of research in auroral physics. However, current methods for automatic auroral classification typically assume that only one type of aurora is present in an auroral image. This oversight neglects the complex transition states and coexistence of multiple types during the auroral evolution process, thus limiting the exploration of the intricate semantics of auroral images. To fully exploit the physical information embedded in auroral images, this paper proposes a multi-label auroral classification method, termed MLAC, which integrates convolutional neural network (CNN) and Transformer architectures. Firstly, we introduce a multi-scale feature fusion framework that enables the model to capture both fine-grained features and high-level information in auroral images, resulting in a more comprehensive representation of auroral features. Secondly, we propose a lightweight multi-head self-attention mechanism that captures long-range dependencies between pixels during the multiscale feature fusion process, which is crucial for distinguishing subtle differences between auroral types. Furthermore, we design a residual focused multilayer perceptron module that integrates large kernel depth-wise convolution with an improved multilayer perceptron. This integration enhances the model’s ability to represent complex spatial structure, thus improving local feature extraction and global contextual understanding. The proposed method achieves a mean average precision (mAP) of 88.20% on the auroral observation data collected at the Yellow River Station from 2003 to 2008. This performance significantly surpasses that of the most advanced multi-label classification models while maintaining competitive computational efficiency. Moreover, our method also outperforms the state-of-the-art multi-label methods in both computational efficiency and classification accuracy on two publicly available multi-label image datasets: WIDER-Attribute and VOC2007. These results demonstrate that our method skillfully leverages the robust feature extraction capability of CNNs for local features and the superior global information processing capability of Transformer.