Highly Accurate Tomato Maturity Recognition: Combining Deep Instance Segmentation, Data Synthesis and Color Analysis

Abstract

Automatic maturity recognition and counting of tomatoes during different growth stages from images is of great significance for optimal management in tomato farming, long-term yield prediction and robotic harvesting. In this study, we present a novel method that combines deep instance segmentation, data synthesis and color analysis to accurately recognize and count tomatoes during different growth stages. In our approach, we trained the Mask R-CNN instance segmentation neural network with synthetically generated dataset to accurately segment all tomato instances in an image, then color-based thresholding was applied to identify their growth stage and count the tomato number accordingly. The synthetic data generation algorithm preserved the physical structure of the data objects, thus produced photorealistic synthesized cultivation scenes. The trained model demonstrated substantial performance with maximum 92.1% average precision and 91.4% recall against the real-world test datasets for tomato segmentation. The tomato maturity recognition accuracy of the color-analysis method was evaluated by comparing estimated count with ground-truth manual counts. Our experimental results demonstrated high accuracy of tomato counting during three different growth stages: green, half ripened and fully ripened.