基于双目视觉的柑橘采摘点检测和定位

IF 5.4 2区 农林科学 Q1 AGRICULTURE, MULTIDISCIPLINARY
Chaojun Hou, Jialiang Xu, Yu Tang, Jiajun Zhuang, Zhiping Tan, Weilin Chen, Sheng Wei, Huasheng Huang, Mingwei Fang
{"title":"基于双目视觉的柑橘采摘点检测和定位","authors":"Chaojun Hou, Jialiang Xu, Yu Tang, Jiajun Zhuang, Zhiping Tan, Weilin Chen, Sheng Wei, Huasheng Huang, Mingwei Fang","doi":"10.1007/s11119-024-10169-2","DOIUrl":null,"url":null,"abstract":"<p>Accurate localization of picking points in non-structural environments is crucial for intelligent picking of ripe citrus with a harvesting robot. However, citrus pedicels are too small and resemble other background objects in color, making it challenging to detect and localize the picking point of citrus fruits. This work presents a novel approach for detecting and localizing citrus picking points using binocular vision. First, the convolutional block attention module (CBAM) attention model is integrated into the backbone network of Mask R-CNN to increase the feature extraction for citrus pedicels, and the soft-non maximum suppression (Soft-NMS) strategy is used in the region proposal network to enhance the detection performance of citrus pedicel. Second, to accurately associate the citrus fruit with the best detected pedicel, a maximum discrimination criterion is proposed by integrating the confidence score of the detected pedicel and the degree of positional connectivity between the pedicel and the fruit. Finally, to reduce matching errors and improve computational efficiency, a rapid and robust matching method based on the normalized cross-correlation was applied to search the picking point within the line segment between the left and right images. The experimental results show that the precision, recall and F1-score for pedicel detection are 95.04%, 88.11%, and 91.44%, respectively, which are improvement of 13.00%, 7.84%, and 10.30% compared to the original Mask R-CNN. The mean absolute error (MAE) for the localizing the citrus picking point is 8.63 mm and the mean relative error (MRE) is 2.76%. The MRE was significantly reduced by at least 1.2% compared to the stereo matching methods belief-propagation (BP), semi-global block matching (SGBM), and block matching (BM), respectively. This study provides an effective method for the precise detection and localization of citrus picking point for a harvesting robot.</p>","PeriodicalId":20423,"journal":{"name":"Precision Agriculture","volume":null,"pages":null},"PeriodicalIF":5.4000,"publicationDate":"2024-07-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Detection and localization of citrus picking points based on binocular vision\",\"authors\":\"Chaojun Hou, Jialiang Xu, Yu Tang, Jiajun Zhuang, Zhiping Tan, Weilin Chen, Sheng Wei, Huasheng Huang, Mingwei Fang\",\"doi\":\"10.1007/s11119-024-10169-2\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>Accurate localization of picking points in non-structural environments is crucial for intelligent picking of ripe citrus with a harvesting robot. However, citrus pedicels are too small and resemble other background objects in color, making it challenging to detect and localize the picking point of citrus fruits. This work presents a novel approach for detecting and localizing citrus picking points using binocular vision. First, the convolutional block attention module (CBAM) attention model is integrated into the backbone network of Mask R-CNN to increase the feature extraction for citrus pedicels, and the soft-non maximum suppression (Soft-NMS) strategy is used in the region proposal network to enhance the detection performance of citrus pedicel. Second, to accurately associate the citrus fruit with the best detected pedicel, a maximum discrimination criterion is proposed by integrating the confidence score of the detected pedicel and the degree of positional connectivity between the pedicel and the fruit. Finally, to reduce matching errors and improve computational efficiency, a rapid and robust matching method based on the normalized cross-correlation was applied to search the picking point within the line segment between the left and right images. The experimental results show that the precision, recall and F1-score for pedicel detection are 95.04%, 88.11%, and 91.44%, respectively, which are improvement of 13.00%, 7.84%, and 10.30% compared to the original Mask R-CNN. The mean absolute error (MAE) for the localizing the citrus picking point is 8.63 mm and the mean relative error (MRE) is 2.76%. The MRE was significantly reduced by at least 1.2% compared to the stereo matching methods belief-propagation (BP), semi-global block matching (SGBM), and block matching (BM), respectively. This study provides an effective method for the precise detection and localization of citrus picking point for a harvesting robot.</p>\",\"PeriodicalId\":20423,\"journal\":{\"name\":\"Precision Agriculture\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":5.4000,\"publicationDate\":\"2024-07-28\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Precision Agriculture\",\"FirstCategoryId\":\"97\",\"ListUrlMain\":\"https://doi.org/10.1007/s11119-024-10169-2\",\"RegionNum\":2,\"RegionCategory\":\"农林科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"AGRICULTURE, MULTIDISCIPLINARY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Precision Agriculture","FirstCategoryId":"97","ListUrlMain":"https://doi.org/10.1007/s11119-024-10169-2","RegionNum":2,"RegionCategory":"农林科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AGRICULTURE, MULTIDISCIPLINARY","Score":null,"Total":0}
引用次数: 0

摘要

在非结构性环境中对采摘点进行精确定位,对于采摘机器人智能采摘成熟柑橘至关重要。然而,柑橘的果梗太小,而且颜色与其他背景物体相似,因此柑橘采摘点的检测和定位极具挑战性。本研究提出了一种利用双目视觉检测和定位柑橘采摘点的新方法。首先,将卷积块注意模块(CBAM)注意模型集成到 Mask R-CNN 的骨干网络中,以提高柑橘果梗的特征提取率,并在区域建议网络中使用软-非最大抑制(Soft-NMS)策略,以提高柑橘果梗的检测性能。其次,为了准确地将柑橘果实与最佳检测到的果梗联系起来,提出了一种最大判别准则,该准则综合了检测到的果梗的置信度得分以及果梗与果实之间的位置连接程度。最后,为了减少匹配误差和提高计算效率,采用了一种基于归一化交叉相关的快速鲁棒匹配方法,在左右图像之间的线段内搜索采摘点。实验结果表明,花梗检测的精确度、召回率和 F1 分数分别为 95.04%、88.11% 和 91.44%,与原始 Mask R-CNN 相比分别提高了 13.00%、7.84% 和 10.30%。柑橘采摘点定位的平均绝对误差(MAE)为 8.63 毫米,平均相对误差(MRE)为 2.76%。与立体匹配方法信念传播法(BP)、半全局块匹配法(SGBM)和块匹配法(BM)相比,平均相对误差至少减少了 1.2%。这项研究为采摘机器人精确检测和定位柑橘采摘点提供了一种有效的方法。
本文章由计算机程序翻译,如有差异,请以英文原文为准。

Detection and localization of citrus picking points based on binocular vision

Detection and localization of citrus picking points based on binocular vision

Accurate localization of picking points in non-structural environments is crucial for intelligent picking of ripe citrus with a harvesting robot. However, citrus pedicels are too small and resemble other background objects in color, making it challenging to detect and localize the picking point of citrus fruits. This work presents a novel approach for detecting and localizing citrus picking points using binocular vision. First, the convolutional block attention module (CBAM) attention model is integrated into the backbone network of Mask R-CNN to increase the feature extraction for citrus pedicels, and the soft-non maximum suppression (Soft-NMS) strategy is used in the region proposal network to enhance the detection performance of citrus pedicel. Second, to accurately associate the citrus fruit with the best detected pedicel, a maximum discrimination criterion is proposed by integrating the confidence score of the detected pedicel and the degree of positional connectivity between the pedicel and the fruit. Finally, to reduce matching errors and improve computational efficiency, a rapid and robust matching method based on the normalized cross-correlation was applied to search the picking point within the line segment between the left and right images. The experimental results show that the precision, recall and F1-score for pedicel detection are 95.04%, 88.11%, and 91.44%, respectively, which are improvement of 13.00%, 7.84%, and 10.30% compared to the original Mask R-CNN. The mean absolute error (MAE) for the localizing the citrus picking point is 8.63 mm and the mean relative error (MRE) is 2.76%. The MRE was significantly reduced by at least 1.2% compared to the stereo matching methods belief-propagation (BP), semi-global block matching (SGBM), and block matching (BM), respectively. This study provides an effective method for the precise detection and localization of citrus picking point for a harvesting robot.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Precision Agriculture
Precision Agriculture 农林科学-农业综合
CiteScore
12.30
自引率
8.10%
发文量
103
审稿时长
>24 weeks
期刊介绍: Precision Agriculture promotes the most innovative results coming from the research in the field of precision agriculture. It provides an effective forum for disseminating original and fundamental research and experience in the rapidly advancing area of precision farming. There are many topics in the field of precision agriculture; therefore, the topics that are addressed include, but are not limited to: Natural Resources Variability: Soil and landscape variability, digital elevation models, soil mapping, geostatistics, geographic information systems, microclimate, weather forecasting, remote sensing, management units, scale, etc. Managing Variability: Sampling techniques, site-specific nutrient and crop protection chemical recommendation, crop quality, tillage, seed density, seed variety, yield mapping, remote sensing, record keeping systems, data interpretation and use, crops (corn, wheat, sugar beets, potatoes, peanut, cotton, vegetables, etc.), management scale, etc. Engineering Technology: Computers, positioning systems, DGPS, machinery, tillage, planting, nutrient and crop protection implements, manure, irrigation, fertigation, yield monitor and mapping, soil physical and chemical characteristic sensors, weed/pest mapping, etc. Profitability: MEY, net returns, BMPs, optimum recommendations, crop quality, technology cost, sustainability, social impacts, marketing, cooperatives, farm scale, crop type, etc. Environment: Nutrient, crop protection chemicals, sediments, leaching, runoff, practices, field, watershed, on/off farm, artificial drainage, ground water, surface water, etc. Technology Transfer: Skill needs, education, training, outreach, methods, surveys, agri-business, producers, distance education, Internet, simulations models, decision support systems, expert systems, on-farm experimentation, partnerships, quality of rural life, etc.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信