Morphology-based weed type recognition using Siamese network

Abstract

Automatic weed detection and classification can significantly reduce weed management costs and improve crop yields and quality. Weed detection in crops from imagery is inherently a challenging problem. Because both weeds and crops are of similar colour (green on green), their growth and texture are somewhat similar; weeds also vary based on crops, geographical locations, seasons and even weather patterns. This study proposes a novel approach utilising object detection and meta-learning techniques for generalised weed detection, transcending the limitations of varying field contexts. Instead of classifying weeds by species, this study classified them based on their morphological families aligned with farming practices. An object detector, e.g., a YOLO (You Only Look Once) model is employed for plant detection, while a Siamese network, leveraging state-of-the-art deep learning models as its backbone, is used for weed classification. This study repurposed and used three publicly available datasets, namely, Weed25, Cotton weed and Corn weed data. Each dataset contained multiple species of weeds, whereas this study grouped those into three classes based on the weed morphology. YOLOv7 achieved the best result as a plant detector, and the VGG16 model as the feature extractor for the Siamese network. Moreover, the models were trained on one dataset (Weed25) and applied to other datasets (Cotton weed and Corn weed) without further training. The study also observed that the classification accuracy of the Siamese network was improved using the cosine similarity function for calculating contrastive loss. The YOLOv7 models obtained the mAP of 91.03 % on the Weed25 dataset, which was used for training the model. The mAPs for the unseen datasets were 84.65 % and 81.16 %. As mentioned earlier, the classification accuracies with the best combination were 97.59 %, 93.67 % and 93.35 % for the Weed25, Cotton weed and Corn weed datasets, respectively. This study also compared the classification performance of our proposed technique with the state-of-the-art Convolutional Neural Network models. The proposed approach advances weed classification accuracy and presents a viable solution for dataset independent, i.e., site-independent weed detection, fostering sustainable agricultural practices.