SceneFormer: Neural Architecture Search of Transformers for Remote Sensing Scene Classification

Abstract

Deep learning-based scene classification methods have long been a key research area in remote sensing imagery due to their wide-ranging applications. Recently, Transformer models have achieved significant progress in computer vision, making vision transformers (ViTs) a promising direction for scene classification. However, the spatial complexity of remote sensing imagery poses unique challenges for applying Transformers directly. Manually designing Transformers tailored for remote sensing scene classification is time-consuming under model parameter constraints and requires extensive domain expertise. To address this challenge, neural architecture search (NAS) methods provide an effective solution to construct optimal Transformer architectures for remote sensing scene classification automatically. In this work, we propose SceneFormer, an automated Transformer architecture search framework tailored for scene classification tasks. In SceneFormer, we construct a dedicated search space to search for the optimal Transformer. Moreover, we design a supernet training strategy to train numerous candidate architectures within the search space simultaneously. Furthermore, SceneFormer employs the evolutionary search to find the optimal Transformer architecture under specific resource constraints. Experiments on three high-spatial-resolution (HSR) datasets demonstrate the effectiveness of SceneFormer.