SFCE-VT: Spatial feature fusion and contrast-enhanced visual transformer for fine-grained agricultural pests visual classification

Abstract

Climate change has led to the intensification of agricultural pests, which are diverse and difficult to identify accurately, and fine-grained classification of agricultural pests is an important method to effectively prevent and control the increasing number of pests, and to ensure the stability and sustainable development of agricultural production. Agricultural pest species can be accurately recognized using deep learning, but current problems such as the small scale agricultural pest data, single scene, and relatively coarse classification results bring challenges to fine-grained image classification of agricultural pests. Therefore, a visual transformer based on spatial feature fusion and contrast enhancement (SFCE-VT) is proposed for fine-grained image classification(FGIC) methods for agricultural pests. First, to accurately localize to the target location, two images, the foreground target, and the occluded background, are cropped using the self-attention mechanism to form three image inputs to complement the detail representation. To further distinguish the foreground target from the background noise, the inputs of three different images are utilized to compare the loss values to enhance the model’s ability to distinguish the foreground target from the background. In addition, to address the challenge of pest recognition from different viewpoints, a self-attention mechanism and graph convolutional network (GCN) are utilized to extract spatial contextual information of the pest region and learn the spatial gesture features of the pests. The experimental results achieved significant performance improvement on both CUB-200-2011 and A-pests, a reconstructed agricultural fine-grained pest dataset, by 1.95% and 3.23% compared to the base vit, respectively. The effectiveness of the cropping contrast enhancement and spatial information learning modules in paying attention to fine-grained features and enriching pest feature information is demonstrated. The source code is publicly available at https://github.com/193lulu/SFCE-VT.