Deep learning-based instance segmentation for improved pepper phenotyping

Vegetable breeding companies invest a considerable amount of their resources in phenotyping. The advancement of computer vision technology has made it possible to digitalize these processes, leading to improved efficiency and quality. However, phenotyping activities often take place in outdoor fields or greenhouses, where the environmental/illumination conditions are constantly changing. This lack of standardization presents a problem for automatically isolating the relevant elements in the images, which is an important first step for phenotyping. Classical image analysis methods have shown not to be robust enough for that in these changing conditions. However, in the last years, deep learning models have demonstrated to be able to identify and learn meaningful features that are more robust and representative of the underlying patterns, enabling them to handle diverse and changeable conditions effectively.

In this work, we propose a pepper instance segmentation solution based on deep learning after harvest under field conditions. We implement the method and validate it for three pepper varieties: Blocky Bell, Jalapeño and Lamuyo. We compare the performance of this new method for each variety with a previous solution based on classical image processing techniques, with the objective of measuring and demonstrating the superiority of deep learning-based instance segmentation over traditional methods as a first step for phenotyping.

The instance segmentation deep learning based models outperform the results obtained by classical image processing algorithms for the three pepper varieties: in Blocky Bell mAP is increased from 0.63 to 0.97, in Jalapeño from 0.39 to 0.52 and in Lamuyo from 0.67 to 0.97.