A unified multi-task model for leaf disease region detection and segmentation

Crop diseases challenge agricultural production. Pinpointing disease spots, assessing infection areas, and gauging infection severity are crucial for effective disease control. However, lesion variations, blurred boundaries, and small, dense lesions make precise detection and segmentation difficult. This paper presents an end-to-end unified multi-task model based on Detection Transformer (DETR) for leaf disease region detection and segmentation. It integrates Convolutional Neural Networks (CNNs) and Transformer, uses the Contextual Transformer Network (CoTNet) for feature extraction, and incorporates innovative mechanisms like box-attention and reference window update. Additionally, we have devised a novel instance segmentation head. This head effectively addresses the misclassification issue between minute disease spots and leaf surfaces. Experiments show the model achieves Average Precision ($\text{AP}$). The Average Precision of Bounding Box (${\text{AP}}^{\text{box}}$) of 73.9 %, the Average Precision of Mask (${\text{AP}}^{\text{mask}}$) of 68.2 %, the Average Precision of Small-sized Bounding Box (${\text{AP}}_s^{\text{box}}$) of 29.0 %, and the Average Precision of Small-sized Mask (${\text{AP}}_s^{\text{mask}}$) of 27.1 % for four diseases, with detection recall reaching 76.8 % and segmentation recall reaching 73.4 %. Meanwhile. The model architecture demonstrates practical feasibility with 40.1 million (m) parameters and 169 Giga Floating-point Operations Per Second (GFLOP) computational complexity. The accuracy of disease grading reaches 92.07 %. In this study, a model based on Artificial Intelligence (AI) was implemented to address the challenges in leaf disease detection and segmentation. The proposed model, which integrates object detection and instance segmentation tasks, can be applied to accurately identify and grade leaf diseases, providing support for disease control strategies in agriculture.