MRSU2Net: A novel method for semantic segmentation of group lettuce from individual Objectives to group Objectives

Abstract

Semantic segmentation methods have played an important role in a wide range of applications, as they contribute to more accurate phenotypic information extraction in the field of plant phenotype. However, the high annotation cost of semantic segmentation datasets remains a major challenge, and most of them are constructed and validated on training and testing datasets with similar scales. Most studies overlook its effectiveness on multi-scale datasets, especially on low resolution datasets. Although some semantic segmentation methods extract and learn multi-scale features from datasets through methods such as multi-scale feature fusion modules and attention mechanisms, the model’s scale down compatibility, i.e. the segmentation reliability of the model on low resolution datasets, has not yet been verified. To address this challenge, this study proposes for the first time a new approach to plant object oriented semantic segmentation, which involves modeling individual target datasets and validating group target datasets. This modeling approach can significantly reduce the annotation cost of datasets to some extent. On this basis, we propose a multi-scale feature fusion module (MSFAF-M) for multi-level feature relationship exploration and a multi receptive field feature fusion module (MRFFF-S) for single-layer feature relationship exploration. By applying MSFAF-M and MRFFF-S to U2Net, an upgraded semantic segmentation method MRSU2Net is proposed, which can fully extract global and local feature information of target objects at multiple scales, and improve the segmentation reliability of semantic segmentation models based on individual target datasets on multi-scale group target datasets. Due to the fact that the construction approach of the semantic segmentation model proposed in this study is different from traditional semantic segmentation methods, we validated the scale down compatibility of MRSU2Net on the target dataset of lettuce populations collected at the seedling stage. When MRSU2Net is applied to group target images with the same resolution (2992 × 2992), the MIoU is 0.9719 and the inference-time is 0.3550. When MRSU2Net is applied to group target images of the same input size (224 × 224), the MIoU can reach 0.7346 and the inference time is 0.0219. The results demonstrate that the segmentation performance of the MRSU2Net constructed in this study is significantly superior to other classic semantic segmentation methods in low resolution images.