Citrus pose estimation under complex orchard environment for robotic harvesting

IF 4.5 1区 农林科学 Q1 AGRONOMY
{"title":"Citrus pose estimation under complex orchard environment for robotic harvesting","authors":"","doi":"10.1016/j.eja.2024.127418","DOIUrl":null,"url":null,"abstract":"<div><div>The growth poses of citrus on trees are diverse. To ensure minimal loss during citrus harvesting, accurately estimating the pose of citrus is particularly important. To solve this problem, this research developed a real-time citrus pose estimation system based on neural networks and point cloud processing algorithms. Specifically, this method uses neural networks to identify citrus. After constructing the citrus point cloud, it is input into the Random Sample Consensus with Levenberg-Marquardt (RANSAC-LM) point cloud processing algorithm to obtain the citrus coordinates. Combined with citrus growth information, the pose is output. By analyzing the distribution of citrus poses, citrus poses convenient for end- effector harvesting are defined. To enhance the camera's ability to obtain information about citrus, a camera observation model is constructed to dynamically adjust the camera position. Through experiments, the appropriate deep learning target detection framework YOLO V5 is selected for citrus object detection. The precision (P), recall rate (R), and mean average precision (mAP) are 92.3 %, 79.1 %, and 88.5 % respectively. This network can handle detection tasks in real orchard environments. The original Random Sample Consensus (RANSAC) is improved by using the Levenberg-Marquardt (LM) nonlinear optimization method. Experimental results show that RANSAC-LM reduces the citrus center coordinate precision error from (0.2, 0.2, 2.3) mm to (0.1, 0.2, 1.4) mm, reduces the accuracy Spherical Error Probable (SEP) from 2.77 to 1.61, and finally reduces the citrus pose error from 5.72° to 2.43°. The efficiency of the proposed citrus pose estimation algorithm is 0.24 s. Deployed on a citrus picking robot, it verifies the feasibility of the algorithm and provides a new solution for the pose estimation problem of citrus harvesting robots.</div></div>","PeriodicalId":51045,"journal":{"name":"European Journal of Agronomy","volume":null,"pages":null},"PeriodicalIF":4.5000,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"European Journal of Agronomy","FirstCategoryId":"97","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1161030124003393","RegionNum":1,"RegionCategory":"农林科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AGRONOMY","Score":null,"Total":0}
引用次数: 0

Abstract

The growth poses of citrus on trees are diverse. To ensure minimal loss during citrus harvesting, accurately estimating the pose of citrus is particularly important. To solve this problem, this research developed a real-time citrus pose estimation system based on neural networks and point cloud processing algorithms. Specifically, this method uses neural networks to identify citrus. After constructing the citrus point cloud, it is input into the Random Sample Consensus with Levenberg-Marquardt (RANSAC-LM) point cloud processing algorithm to obtain the citrus coordinates. Combined with citrus growth information, the pose is output. By analyzing the distribution of citrus poses, citrus poses convenient for end- effector harvesting are defined. To enhance the camera's ability to obtain information about citrus, a camera observation model is constructed to dynamically adjust the camera position. Through experiments, the appropriate deep learning target detection framework YOLO V5 is selected for citrus object detection. The precision (P), recall rate (R), and mean average precision (mAP) are 92.3 %, 79.1 %, and 88.5 % respectively. This network can handle detection tasks in real orchard environments. The original Random Sample Consensus (RANSAC) is improved by using the Levenberg-Marquardt (LM) nonlinear optimization method. Experimental results show that RANSAC-LM reduces the citrus center coordinate precision error from (0.2, 0.2, 2.3) mm to (0.1, 0.2, 1.4) mm, reduces the accuracy Spherical Error Probable (SEP) from 2.77 to 1.61, and finally reduces the citrus pose error from 5.72° to 2.43°. The efficiency of the proposed citrus pose estimation algorithm is 0.24 s. Deployed on a citrus picking robot, it verifies the feasibility of the algorithm and provides a new solution for the pose estimation problem of citrus harvesting robots.
复杂果园环境下的柑橘姿态估计,用于机器人收割
柑橘在树上的生长姿态多种多样。为了确保在柑橘采收过程中损失最小,准确估计柑橘的姿态尤为重要。为解决这一问题,本研究开发了一种基于神经网络和点云处理算法的实时柑橘姿态估计系统。具体来说,该方法使用神经网络来识别柑橘。在构建柑橘点云后,将其输入随机样本共识与莱文伯格-马夸特(RANSAC-LM)点云处理算法,以获得柑橘坐标。结合柑橘生长信息,输出柑橘姿态。通过分析柑橘姿态的分布,确定了便于末端效应器收割的柑橘姿态。为了提高摄像头获取柑橘信息的能力,还构建了一个摄像头观测模型来动态调整摄像头位置。通过实验,为柑橘目标检测选择了合适的深度学习目标检测框架 YOLO V5。其精度(P)、召回率(R)和平均精度(mAP)分别为 92.3%、79.1% 和 88.5%。该网络可以处理真实果园环境中的检测任务。利用 Levenberg-Marquardt (LM) 非线性优化方法对原始随机抽样共识(RANSAC)进行了改进。实验结果表明,RANSAC-LM 将柑橘中心坐标精度误差从 (0.2, 0.2, 2.3) mm 降低到 (0.1, 0.2, 1.4) mm,将精度球形误差概率 (SEP) 从 2.77 降低到 1.61,并最终将柑橘姿态误差从 5.72° 降低到 2.43°。所提出的柑橘姿态估计算法的效率为 0.24 秒。在柑橘采摘机器人上的应用验证了该算法的可行性,并为柑橘采摘机器人的姿态估计问题提供了一种新的解决方案。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
European Journal of Agronomy
European Journal of Agronomy 农林科学-农艺学
CiteScore
8.30
自引率
7.70%
发文量
187
审稿时长
4.5 months
期刊介绍: The European Journal of Agronomy, the official journal of the European Society for Agronomy, publishes original research papers reporting experimental and theoretical contributions to field-based agronomy and crop science. The journal will consider research at the field level for agricultural, horticultural and tree crops, that uses comprehensive and explanatory approaches. The EJA covers the following topics: crop physiology crop production and management including irrigation, fertilization and soil management agroclimatology and modelling plant-soil relationships crop quality and post-harvest physiology farming and cropping systems agroecosystems and the environment crop-weed interactions and management organic farming horticultural crops papers from the European Society for Agronomy bi-annual meetings In determining the suitability of submitted articles for publication, particular scrutiny is placed on the degree of novelty and significance of the research and the extent to which it adds to existing knowledge in agronomy.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信