基于双目立体视觉的茶叶采摘机器人定位

IF 5.2 2区 计算机科学 Q2 ROBOTICS
Leiying He, Qianyao Zhuang, Yatao Li, Zhenghao Zhong, Jianneng Chen, Chuanyu Wu
{"title":"基于双目立体视觉的茶叶采摘机器人定位","authors":"Leiying He,&nbsp;Qianyao Zhuang,&nbsp;Yatao Li,&nbsp;Zhenghao Zhong,&nbsp;Jianneng Chen,&nbsp;Chuanyu Wu","doi":"10.1002/rob.22559","DOIUrl":null,"url":null,"abstract":"<div>\n \n <p>Localization of tea shoots is essential for achieving intelligent plucking. However, accurately identifying the plucking point within the unstructured field environment remains challenging. This study proposes a method for three-dimensional (3D) localization of tea shoots utilizing binocular stereo vision for robotic plucking in such environments. Initially, tea shoot masks from each binocular image are extracted using the You Only Look Once segmentation network and paired by calculating image similarity through the combined use of Scale-Invariant Feature Transform features and color histograms. The Selective AD-Census-HSI stereo-matching algorithm was subsequently developed specifically to generate disparity maps for instance-segmented tea shoots. This approach also incorporated enhancements in the initial cost calculation and the cross-construction modules to improve the algorithm's performance. The point cloud is generated via triangulation to identify the plucking points using V-shaped template matching. Disparity evaluation results indicate that the proposed stereo-matching algorithm enhances accuracy compared with the original AD-Census, especially in scenarios with significant luminance contrast between the left and right views. Results from the indoor 3D localization experiment show that the average localization error of the tea shoot plucking point is 5.78 mm. Lastly, a robotic tea shoot plucking experiment conducted in the field achieved a success rate of 62%. These results demonstrate that the proposed tea shoot localization method satisfies the requirements for robotic tea plucking, providing a novel solution for intelligent harvesting of tea.</p>\n </div>","PeriodicalId":192,"journal":{"name":"Journal of Field Robotics","volume":"42 6","pages":"2985-3002"},"PeriodicalIF":5.2000,"publicationDate":"2025-04-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Localization of Tea Shoots for Robotic Plucking Using Binocular Stereo Vision\",\"authors\":\"Leiying He,&nbsp;Qianyao Zhuang,&nbsp;Yatao Li,&nbsp;Zhenghao Zhong,&nbsp;Jianneng Chen,&nbsp;Chuanyu Wu\",\"doi\":\"10.1002/rob.22559\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div>\\n \\n <p>Localization of tea shoots is essential for achieving intelligent plucking. However, accurately identifying the plucking point within the unstructured field environment remains challenging. This study proposes a method for three-dimensional (3D) localization of tea shoots utilizing binocular stereo vision for robotic plucking in such environments. Initially, tea shoot masks from each binocular image are extracted using the You Only Look Once segmentation network and paired by calculating image similarity through the combined use of Scale-Invariant Feature Transform features and color histograms. The Selective AD-Census-HSI stereo-matching algorithm was subsequently developed specifically to generate disparity maps for instance-segmented tea shoots. This approach also incorporated enhancements in the initial cost calculation and the cross-construction modules to improve the algorithm's performance. The point cloud is generated via triangulation to identify the plucking points using V-shaped template matching. Disparity evaluation results indicate that the proposed stereo-matching algorithm enhances accuracy compared with the original AD-Census, especially in scenarios with significant luminance contrast between the left and right views. Results from the indoor 3D localization experiment show that the average localization error of the tea shoot plucking point is 5.78 mm. Lastly, a robotic tea shoot plucking experiment conducted in the field achieved a success rate of 62%. These results demonstrate that the proposed tea shoot localization method satisfies the requirements for robotic tea plucking, providing a novel solution for intelligent harvesting of tea.</p>\\n </div>\",\"PeriodicalId\":192,\"journal\":{\"name\":\"Journal of Field Robotics\",\"volume\":\"42 6\",\"pages\":\"2985-3002\"},\"PeriodicalIF\":5.2000,\"publicationDate\":\"2025-04-21\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Field Robotics\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1002/rob.22559\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ROBOTICS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Field Robotics","FirstCategoryId":"94","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/rob.22559","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ROBOTICS","Score":null,"Total":0}
引用次数: 0

摘要

茶苗的定位是实现智能采摘的关键。然而,在非结构化的油田环境中,准确地确定采油点仍然是一个挑战。本研究提出了一种在这种环境下利用双目立体视觉进行机器人采摘的茶叶三维定位方法。首先,使用You Only Look Once分割网络从每张双目图像中提取茶叶面具,并结合使用Scale-Invariant Feature Transform特征和颜色直方图计算图像相似度进行配对。随后开发了选择性AD-Census-HSI立体匹配算法,专门用于生成视差图,例如分割茶叶。该方法还对初始成本计算和交叉构造模块进行了改进,以提高算法的性能。通过三角剖分生成点云,利用v型模板匹配识别拔点。视差评估结果表明,与原始AD-Census相比,本文提出的立体匹配算法提高了精度,特别是在左右视图亮度对比明显的场景下。室内三维定位实验结果表明,茶叶采摘点的平均定位误差为5.78 mm。最后,在田间进行了机器人采茶试验,成功率为62%。结果表明,所提出的茶叶梢定位方法满足机器人采茶的要求,为茶叶智能采收提供了一种新的解决方案。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Localization of Tea Shoots for Robotic Plucking Using Binocular Stereo Vision

Localization of tea shoots is essential for achieving intelligent plucking. However, accurately identifying the plucking point within the unstructured field environment remains challenging. This study proposes a method for three-dimensional (3D) localization of tea shoots utilizing binocular stereo vision for robotic plucking in such environments. Initially, tea shoot masks from each binocular image are extracted using the You Only Look Once segmentation network and paired by calculating image similarity through the combined use of Scale-Invariant Feature Transform features and color histograms. The Selective AD-Census-HSI stereo-matching algorithm was subsequently developed specifically to generate disparity maps for instance-segmented tea shoots. This approach also incorporated enhancements in the initial cost calculation and the cross-construction modules to improve the algorithm's performance. The point cloud is generated via triangulation to identify the plucking points using V-shaped template matching. Disparity evaluation results indicate that the proposed stereo-matching algorithm enhances accuracy compared with the original AD-Census, especially in scenarios with significant luminance contrast between the left and right views. Results from the indoor 3D localization experiment show that the average localization error of the tea shoot plucking point is 5.78 mm. Lastly, a robotic tea shoot plucking experiment conducted in the field achieved a success rate of 62%. These results demonstrate that the proposed tea shoot localization method satisfies the requirements for robotic tea plucking, providing a novel solution for intelligent harvesting of tea.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Journal of Field Robotics
Journal of Field Robotics 工程技术-机器人学
CiteScore
15.00
自引率
3.60%
发文量
80
审稿时长
6 months
期刊介绍: The Journal of Field Robotics seeks to promote scholarly publications dealing with the fundamentals of robotics in unstructured and dynamic environments. The Journal focuses on experimental robotics and encourages publication of work that has both theoretical and practical significance.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信