Safflower picking points localization method during the full harvest period based on SBP-YOLOv8s-seg network

IF 7.7 1区农林科学 Q1 AGRICULTURE, MULTIDISCIPLINARY

Computers and Electronics in Agriculture Pub Date : 2024-11-25 DOI:10.1016/j.compag.2024.109646

He Zhang, Yun Ge, Hao Xia, Chao Sun

{"title":"Safflower picking points localization method during the full harvest period based on SBP-YOLOv8s-seg network","authors":"He Zhang, Yun Ge, Hao Xia, Chao Sun","doi":"10.1016/j.compag.2024.109646","DOIUrl":null,"url":null,"abstract":"<div><div>Visual recognition is crucial for robotic harvesting of safflower filaments in field. However, accurate detection and localization is challenging due to complex backgrounds, leaves and branches shielding, and variable safflower morphology. This study proposes a safflower picking points localization method during the full harvest period based on SBP-YOLOv8s-seg network. The method enhanced the accuracy by improving the performance of the detection and segmentation network and implementing phased localization. Specifically, SBP-YOLOv8s-seg network based on self-calibration was constructed for precise segmentation of safflower filaments and fruit balls. Additionally, different morphological features of safflower during the full harvest period were analyzed. The segmented masks underwent Principal Component Analysis (PCA) computation, region of interest (ROI) extraction, and contour fitting to extract the principal eigenvectors that express information about the filaments. To address the issue of picking position being invisible due to the occlusion of safflower necking, the picking points were determined in conjunction with the positional relationship between filaments and fruit balls. Experimental results demonstrated that the segmentation performance of SBP-YOLOv8s-seg network was superior to other networks, achieving a significant improvement in mean average precision (mAP) compared to YOLOv5s-seg, YOLOv6s-seg, YOLOv7s-seg, and YOLOv8s-seg, with improvements of 5.1 %, 2.3 %, 4.1 %, and 1.3 % respectively. The precision, recall and mAP of SBP-YOLOv8s-seg network in the segmentation task increased from 87.9 %, 79 %, and 84.4 % of YOLOv8s-seg to 89.1 %, 79.7 %, and 85.7 %. The accuracy of blooming safflower and decaying safflower calculated by the proposed method were 93.0 % and 91.9 %, respectively. The overall localization accuracy of safflower picking points was 92.9 %. Field experiments showed that the picking success rate was 90.7 %. This study provides a theoretical basis and data support for visual localization of safflower picking robot in the future.</div></div>","PeriodicalId":50627,"journal":{"name":"Computers and Electronics in Agriculture","volume":"227 ","pages":"Article 109646"},"PeriodicalIF":7.7000,"publicationDate":"2024-11-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computers and Electronics in Agriculture","FirstCategoryId":"97","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0168169924010378","RegionNum":1,"RegionCategory":"农林科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AGRICULTURE, MULTIDISCIPLINARY","Score":null,"Total":0}

引用次数: 0

Abstract

Visual recognition is crucial for robotic harvesting of safflower filaments in field. However, accurate detection and localization is challenging due to complex backgrounds, leaves and branches shielding, and variable safflower morphology. This study proposes a safflower picking points localization method during the full harvest period based on SBP-YOLOv8s-seg network. The method enhanced the accuracy by improving the performance of the detection and segmentation network and implementing phased localization. Specifically, SBP-YOLOv8s-seg network based on self-calibration was constructed for precise segmentation of safflower filaments and fruit balls. Additionally, different morphological features of safflower during the full harvest period were analyzed. The segmented masks underwent Principal Component Analysis (PCA) computation, region of interest (ROI) extraction, and contour fitting to extract the principal eigenvectors that express information about the filaments. To address the issue of picking position being invisible due to the occlusion of safflower necking, the picking points were determined in conjunction with the positional relationship between filaments and fruit balls. Experimental results demonstrated that the segmentation performance of SBP-YOLOv8s-seg network was superior to other networks, achieving a significant improvement in mean average precision (mAP) compared to YOLOv5s-seg, YOLOv6s-seg, YOLOv7s-seg, and YOLOv8s-seg, with improvements of 5.1 %, 2.3 %, 4.1 %, and 1.3 % respectively. The precision, recall and mAP of SBP-YOLOv8s-seg network in the segmentation task increased from 87.9 %, 79 %, and 84.4 % of YOLOv8s-seg to 89.1 %, 79.7 %, and 85.7 %. The accuracy of blooming safflower and decaying safflower calculated by the proposed method were 93.0 % and 91.9 %, respectively. The overall localization accuracy of safflower picking points was 92.9 %. Field experiments showed that the picking success rate was 90.7 %. This study provides a theoretical basis and data support for visual localization of safflower picking robot in the future.

查看原文本刊更多论文

基于 SBP-YOLOv8s-seg 网络的全收获期红花采摘点定位方法

视觉识别对于田间红花花丝的机器人收割至关重要。然而，由于背景复杂、枝叶遮挡以及红花形态多变，准确检测和定位具有挑战性。本研究提出了一种基于 SBP-YOLOv8s-seg 网络的全收获期红花采摘点定位方法。该方法通过提高检测和分割网络的性能以及实施分阶段定位，提高了定位精度。具体而言，构建了基于自校准的 SBP-YOLOv8s-seg 网络，用于精确分割红花花丝和果球。此外，还分析了整个收获期红花的不同形态特征。分割后的掩膜经过主成分分析（PCA）计算、感兴趣区域（ROI）提取和轮廓拟合，以提取表达花丝信息的主特征向量。为了解决由于红花颈部遮挡导致采摘位置不可见的问题，采摘点是结合花丝和果球之间的位置关系确定的。实验结果表明，SBP-YOLOv8s-seg 网络的分割性能优于其他网络，与 YOLOv5s-seg、YOLOv6s-seg、YOLOv7s-seg 和 YOLOv8s-seg 相比，平均精确度（mAP）显著提高，分别提高了 5.1%、2.3%、4.1% 和 1.3%。在分割任务中，SBP-YOLOv8s-seg 网络的精确度、召回率和 mAP 分别从 YOLOv8s-seg 的 87.9 %、79 % 和 84.4 % 提高到 89.1 %、79.7 % 和 85.7 %。建议方法计算的盛开红花和衰落红花的准确率分别为 93.0 % 和 91.9 %。红花采摘点的总体定位精度为 92.9%。田间试验表明，采摘成功率为 90.7%。这项研究为今后红花采摘机器人的可视化定位提供了理论依据和数据支持。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Computers and Electronics in Agriculture 工程技术-计算机：跨学科应用

CiteScore

15.30

自引率

14.50%

发文量

800

审稿时长

62 days

期刊介绍： Computers and Electronics in Agriculture provides international coverage of advancements in computer hardware, software, electronic instrumentation, and control systems applied to agricultural challenges. Encompassing agronomy, horticulture, forestry, aquaculture, and animal farming, the journal publishes original papers, reviews, and applications notes. It explores the use of computers and electronics in plant or animal agricultural production, covering topics like agricultural soils, water, pests, controlled environments, and waste. The scope extends to on-farm post-harvest operations and relevant technologies, including artificial intelligence, sensors, machine vision, robotics, networking, and simulation modeling. Its companion journal, Smart Agricultural Technology, continues the focus on smart applications in production agriculture.