PIS-Net: Efficient weakly supervised instance segmentation network based on annotated points for rice field weed identification

Weed damage in rice fields is one of the main causes of reduced rice yields and quality. Accurate and efficient weed identification is the prerequisite for realizing intelligent and precise weeding in paddies. Recently, Vision Transformers (ViTs) have emerged with superior performance on computer vision tasks compared to the convolutional neural network (CNN)-based models. However, the lack of fully labeled weed datasets hinders the potential application of deep learning models in weed identification. To address the above issues, this study customizes a novel point-supervised instance segmentation network (PIS-Net) for weakly supervised instance segmentation of weeds in rice fields. More correctly, we first propose a novel instance segmentation point labeling scheme that utilizes randomly generated annotation points within each instance, aiming to decrease both labeling time and difficulty. Additionally, to make optimal use of point labels, this study puts forth a mask generation strategy based on the adaptive selection of pyramid levels. In this sense, the network model can flexibly choose the pyramid level expected to generate the most suitable instance mask based on the network's reliability. Finally, we establish the pseudo label refinement network (PLR-Net) to refine rough instance masks. The proposed PIS-Net utilizes 13 randomly generated annotation points for each instance, yet achieving an AP of 38.5 and an AP50 of 68.3, which is superior to the baseline mask-R-CNN with an AP of 8.2 and AP50 of 6.9, achieving 90 % fully supervised performance. This method effectively utilizes point labels, annotated with high efficiency, as a robust source of weak supervision to address challenges in weed data annotation and the low accuracy of existing weakly supervised models. Experiments show that the point annotation scheme of the PIS-Net is faster than full-object mask annotation, and the AP is also higher than the current semi-supervised weed segmentation model, enjoying high potentials in practical paddy fields.