语义感知的下一个最佳视图规划,用于高效搜索和检测任务相关的植物部分

IF 4.4 1区 农林科学 Q1 AGRICULTURAL ENGINEERING
Akshay K. Burusa, Joost Scholten, Xin Wang, David Rapado-Rincón, Eldert J. van Henten, Gert Kootstra
{"title":"语义感知的下一个最佳视图规划,用于高效搜索和检测任务相关的植物部分","authors":"Akshay K. Burusa,&nbsp;Joost Scholten,&nbsp;Xin Wang,&nbsp;David Rapado-Rincón,&nbsp;Eldert J. van Henten,&nbsp;Gert Kootstra","doi":"10.1016/j.biosystemseng.2024.09.018","DOIUrl":null,"url":null,"abstract":"<div><div>Searching and detecting the task-relevant parts of plants is important to automate harvesting and de-leafing of tomato plants using robots. This is challenging due to high levels of occlusion in tomato plants. Active vision is a promising approach in which the robot strategically plans its camera viewpoints to overcome occlusion and improve perception accuracy. However, current active-vision algorithms cannot differentiate between relevant and irrelevant plant parts and spend time on perceiving irrelevant plant parts. This work proposed a semantics-aware active-vision strategy that uses semantic information to identify the relevant plant parts and prioritise them during view planning. The proposed strategy was evaluated on the task of searching and detecting the relevant plant parts using simulation and real-world experiments. In simulation experiments, the semantics-aware strategy proposed could search and detect 81.8% of the relevant plant parts using nine viewpoints. It was significantly faster and detected more plant parts than predefined, random, and volumetric active-vision strategies that do not use semantic information. The strategy proposed was also robust to uncertainty in plant and plant-part positions, plant complexity, and different viewpoint-sampling strategies. In real-world experiments, the semantics-aware strategy could search and detect 82.7% of the relevant plant parts using seven viewpoints, under complex greenhouse conditions with natural variation and occlusion, natural illumination, sensor noise, and uncertainty in camera poses. The results of this work clearly indicate the advantage of using semantics-aware active vision for targeted perception of plant parts and its applicability in the real world. It can significantly improve the efficiency of automated harvesting and de-leafing in tomato crop production.</div></div>","PeriodicalId":9173,"journal":{"name":"Biosystems Engineering","volume":"248 ","pages":"Pages 1-14"},"PeriodicalIF":4.4000,"publicationDate":"2024-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Semantics-aware next-best-view planning for efficient search and detection of task-relevant plant parts\",\"authors\":\"Akshay K. Burusa,&nbsp;Joost Scholten,&nbsp;Xin Wang,&nbsp;David Rapado-Rincón,&nbsp;Eldert J. van Henten,&nbsp;Gert Kootstra\",\"doi\":\"10.1016/j.biosystemseng.2024.09.018\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>Searching and detecting the task-relevant parts of plants is important to automate harvesting and de-leafing of tomato plants using robots. This is challenging due to high levels of occlusion in tomato plants. Active vision is a promising approach in which the robot strategically plans its camera viewpoints to overcome occlusion and improve perception accuracy. However, current active-vision algorithms cannot differentiate between relevant and irrelevant plant parts and spend time on perceiving irrelevant plant parts. This work proposed a semantics-aware active-vision strategy that uses semantic information to identify the relevant plant parts and prioritise them during view planning. The proposed strategy was evaluated on the task of searching and detecting the relevant plant parts using simulation and real-world experiments. In simulation experiments, the semantics-aware strategy proposed could search and detect 81.8% of the relevant plant parts using nine viewpoints. It was significantly faster and detected more plant parts than predefined, random, and volumetric active-vision strategies that do not use semantic information. The strategy proposed was also robust to uncertainty in plant and plant-part positions, plant complexity, and different viewpoint-sampling strategies. In real-world experiments, the semantics-aware strategy could search and detect 82.7% of the relevant plant parts using seven viewpoints, under complex greenhouse conditions with natural variation and occlusion, natural illumination, sensor noise, and uncertainty in camera poses. The results of this work clearly indicate the advantage of using semantics-aware active vision for targeted perception of plant parts and its applicability in the real world. It can significantly improve the efficiency of automated harvesting and de-leafing in tomato crop production.</div></div>\",\"PeriodicalId\":9173,\"journal\":{\"name\":\"Biosystems Engineering\",\"volume\":\"248 \",\"pages\":\"Pages 1-14\"},\"PeriodicalIF\":4.4000,\"publicationDate\":\"2024-09-30\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Biosystems Engineering\",\"FirstCategoryId\":\"97\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S1537511024002204\",\"RegionNum\":1,\"RegionCategory\":\"农林科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"AGRICULTURAL ENGINEERING\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Biosystems Engineering","FirstCategoryId":"97","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1537511024002204","RegionNum":1,"RegionCategory":"农林科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AGRICULTURAL ENGINEERING","Score":null,"Total":0}
引用次数: 0

摘要

搜索和检测植物的任务相关部分对于使用机器人自动收获和摘除番茄植株的叶子非常重要。由于番茄植株的遮挡程度较高,因此这项工作极具挑战性。主动视觉是一种很有前途的方法,在这种方法中,机器人会战略性地规划其摄像头视点,以克服遮挡并提高感知精度。然而,目前的主动视觉算法无法区分相关和不相关的植物部分,因此会在感知不相关的植物部分上花费时间。这项研究提出了一种语义感知主动视觉策略,利用语义信息识别相关植物部分,并在视图规划过程中对其进行优先排序。通过模拟实验和真实世界实验,在搜索和检测相关植物部分的任务中对所提出的策略进行了评估。在模拟实验中,所提出的语义感知策略可以使用九个视角搜索和检测 81.8% 的相关植物部分。与不使用语义信息的预定义策略、随机策略和体积主动视觉策略相比,该策略的速度明显更快,检测到的植物部分也更多。所提出的策略对植物和植物部分位置的不确定性、植物的复杂性以及不同的视点采样策略也很稳健。在真实世界的实验中,在复杂的温室条件下,包括自然变化和遮挡、自然光照、传感器噪声以及相机姿势的不确定性,语义感知策略可以使用七个视点搜索并检测到 82.7% 的相关植物部分。这项工作的结果清楚地表明了使用语义感知主动视觉来有针对性地感知植物部分的优势及其在现实世界中的适用性。它能显著提高番茄作物生产中自动收获和去叶的效率。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Semantics-aware next-best-view planning for efficient search and detection of task-relevant plant parts
Searching and detecting the task-relevant parts of plants is important to automate harvesting and de-leafing of tomato plants using robots. This is challenging due to high levels of occlusion in tomato plants. Active vision is a promising approach in which the robot strategically plans its camera viewpoints to overcome occlusion and improve perception accuracy. However, current active-vision algorithms cannot differentiate between relevant and irrelevant plant parts and spend time on perceiving irrelevant plant parts. This work proposed a semantics-aware active-vision strategy that uses semantic information to identify the relevant plant parts and prioritise them during view planning. The proposed strategy was evaluated on the task of searching and detecting the relevant plant parts using simulation and real-world experiments. In simulation experiments, the semantics-aware strategy proposed could search and detect 81.8% of the relevant plant parts using nine viewpoints. It was significantly faster and detected more plant parts than predefined, random, and volumetric active-vision strategies that do not use semantic information. The strategy proposed was also robust to uncertainty in plant and plant-part positions, plant complexity, and different viewpoint-sampling strategies. In real-world experiments, the semantics-aware strategy could search and detect 82.7% of the relevant plant parts using seven viewpoints, under complex greenhouse conditions with natural variation and occlusion, natural illumination, sensor noise, and uncertainty in camera poses. The results of this work clearly indicate the advantage of using semantics-aware active vision for targeted perception of plant parts and its applicability in the real world. It can significantly improve the efficiency of automated harvesting and de-leafing in tomato crop production.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Biosystems Engineering
Biosystems Engineering 农林科学-农业工程
CiteScore
10.60
自引率
7.80%
发文量
239
审稿时长
53 days
期刊介绍: Biosystems Engineering publishes research in engineering and the physical sciences that represent advances in understanding or modelling of the performance of biological systems for sustainable developments in land use and the environment, agriculture and amenity, bioproduction processes and the food chain. The subject matter of the journal reflects the wide range and interdisciplinary nature of research in engineering for biological systems.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信