CSWin-MBConv: A dual-network fusing CNN and Transformer for weed recognition

Abstract

Accurately acquiring the weed category information is a crucial and indispensable step for effective field management. However, most of weed recognition techniques mainly rely on either pure Convolutional Neural Network (CNN) or Transformer architectures. CNNs excel at extracting local features but struggle to capture global representations. On the other hand, Transformers can capture long-distance feature dependencies but often lose local feature details. These limitations result in suboptimal performances of existing weed recognition models. To address these challenges, this paper proposes a novel hybrid network model, coined as CSWin-MBConv. CSWin-MBConv combines CNN and Transformer architectures in parallel, with CNN branch used to extract local features and Transformer branch employed to capture global representations. In order to enhance the fusion of feature maps from these two branches, we customize the CBAM feature fusion module (CFFM), which facilitates the generation of more comprehensive feature representations. Extensive experiments demonstrate that CSWin-MBConv, whilst being more parameter- and computation-conserving, achieves superior recognition accuracy (98.50 %) and F1-score (98.56 %), outperforming the state-of-the-art CNN and Transformer architectures (e.g., EfficientNet and Swin Transformer). Taking the accuracy as well as the efficiency into account, our proposed model provides a practical support for weed management of paddy fields.