IEEE Robotics and Automation Letters最新文献_第9页

HPA-MPC: Hybrid Perception-Aware Nonlinear Model Predictive Control for Quadrotors With Suspended Loads

IF 4.6 2区计算机科学

IEEE Robotics and Automation Letters Pub Date : 2024-11-25 DOI: 10.1109/LRA.2024.3505816

Mrunal Sarvaiya;Guanrui Li;Giuseppe Loianno

{"title":"HPA-MPC: Hybrid Perception-Aware Nonlinear Model Predictive Control for Quadrotors With Suspended Loads","authors":"Mrunal Sarvaiya;Guanrui Li;Giuseppe Loianno","doi":"10.1109/LRA.2024.3505816","DOIUrl":"https://doi.org/10.1109/LRA.2024.3505816","url":null,"abstract":"Quadrotors equipped with cable-suspended loads represent a versatile, low-cost, and energy efficient solution for aerial transportation, construction, and manipulation tasks. However, their real-world deployment is hindered by several challenges. The system is difficult to control because it is nonlinear, underactuated, involves hybrid dynamics due to slack-taut cable modes, and evolves on complex configuration spaces. Additionally, it is crucial to estimate the full state and the cable's mode transitions in real-time using on-board sensors and computation. To address these challenges, we present a novel Hybrid Perception-Aware Nonlinear Model Predictive Control (HPA-MPC) control approach for quadrotors with suspended loads. Our method considers the complete hybrid system dynamics and includes a perception-aware cost to ensure the payload remains visible in the robot's camera during navigation. Furthermore, the full state and hybrid dynamics' transitions are estimated using onboard sensors. Experimental results demonstrate that our approach enables stable load tracking control, even during slack-taut transitions, and operates entirely onboard. The experiments also show that the perception-aware term effectively keeps the payload in the robot's camera field of view when a human operator interacts with the load.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 1","pages":"358-365"},"PeriodicalIF":4.6,"publicationDate":"2024-11-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142777879","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

SEDMamba: Enhancing Selective State Space Modelling With Bottleneck Mechanism and Fine-to-Coarse Temporal Fusion for Efficient Error Detection in Robot-Assisted Surgery

IF 4.6 2区计算机科学

IEEE Robotics and Automation Letters Pub Date : 2024-11-25 DOI: 10.1109/LRA.2024.3505818

Jialang Xu;Nazir Sirajudeen;Matthew Boal;Nader Francis;Danail Stoyanov;Evangelos B. Mazomenos

{"title":"SEDMamba: Enhancing Selective State Space Modelling With Bottleneck Mechanism and Fine-to-Coarse Temporal Fusion for Efficient Error Detection in Robot-Assisted Surgery","authors":"Jialang Xu;Nazir Sirajudeen;Matthew Boal;Nader Francis;Danail Stoyanov;Evangelos B. Mazomenos","doi":"10.1109/LRA.2024.3505818","DOIUrl":"https://doi.org/10.1109/LRA.2024.3505818","url":null,"abstract":"Automated detection of surgical errors can improve robotic-assisted surgery. Despite promising progress, existing methods still face challenges in capturing rich temporal context to establish long-term dependencies while maintaining computational efficiency. In this letter, we propose a novel hierarchical model named SEDMamba, which incorporates the selective state space model (SSM) into surgical error detection, facilitating efficient long sequence modelling with linear complexity. SEDMamba enhances selective SSM with a bottleneck mechanism and fine-to-coarse temporal fusion (FCTF) to detect and temporally localize surgical errors in long videos. The bottleneck mechanism compresses and restores features within their spatial dimension, thereby reducing computational complexity. FCTF utilizes multiple dilated 1D convolutional layers to merge temporal information across diverse scale ranges, accommodating errors of varying duration. Our work also contributes the first-of-its-kind, frame-level, in-vivo surgical error dataset to support error detection in real surgical cases. Specifically, we deploy the clinically validated observational clinical human reliability assessment tool (OCHRA) to annotate the errors during suturing tasks in an open-source radical prostatectomy dataset (SAR-RARP50). Experimental results demonstrate that our SEDMamba outperforms state-of-the-art methods with at least 1.82% AUC and 3.80% AP performance gains with significantly reduced computational complexity.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 1","pages":"232-239"},"PeriodicalIF":4.6,"publicationDate":"2024-11-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142761573","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

FASTNav: Fine-Tuned Adaptive Small-Language- Models Trained for Multi-Point Robot Navigation

IF 4.6 2区计算机科学

IEEE Robotics and Automation Letters Pub Date : 2024-11-25 DOI: 10.1109/LRA.2024.3506280

Yuxuan Chen;Yixin Han;Xiao Li

{"title":"FASTNav: Fine-Tuned Adaptive Small-Language- Models Trained for Multi-Point Robot Navigation","authors":"Yuxuan Chen;Yixin Han;Xiao Li","doi":"10.1109/LRA.2024.3506280","DOIUrl":"https://doi.org/10.1109/LRA.2024.3506280","url":null,"abstract":"With the rapid development of large language models (LLM), robots are starting to enjoy the benefits of new interaction methods that large language models bring. Because edge computing fulfills the needs for rapid response, privacy, and network autonomy, we believe it facilitates the extensive deployment of large models for robot navigation across various industries. To enable local deployment of language models on edge devices, we adopt some model boosting methods. In this paper, we propose \u0000<italic>FASTNav</i>\u0000 - a method for boosting lightweight LLMs, also known as small language models (SLMs), for robot navigation. The proposed method contains three modules: fine-tuning, teacher-student iteration, and language-based multi-point robot navigation. We train and evaluate models with FASTNav in both simulation and real robots, proving that we can deploy them with low cost, high accuracy and low response time. Compared to other model compression methods, FASTNav shows potential in the local deployment of language models and tends to be a promising solution for language-guided robot navigation on edge devices.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 1","pages":"390-397"},"PeriodicalIF":4.6,"publicationDate":"2024-11-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142777877","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

TARS: Tactile Affordance in Robot Synesthesia for Dexterous Manipulation

IF 4.6 2区计算机科学

IEEE Robotics and Automation Letters Pub Date : 2024-11-25 DOI: 10.1109/LRA.2024.3505783

Qiwei Wu;Haidong Wang;Jiayu Zhou;Xiaogang Xiong;Yunjiang Lou

{"title":"TARS: Tactile Affordance in Robot Synesthesia for Dexterous Manipulation","authors":"Qiwei Wu;Haidong Wang;Jiayu Zhou;Xiaogang Xiong;Yunjiang Lou","doi":"10.1109/LRA.2024.3505783","DOIUrl":"https://doi.org/10.1109/LRA.2024.3505783","url":null,"abstract":"In the field of dexterous robotic manipulation, integrating visual and tactile modalities to inform manipulation policies presents significant challenges, especially in non-contact scenarios where reliance on tactile perception can be inadequate. Visual affordance techniques currently offer effective manipulation-centric semantic priors focused on objects. However, most existing research is limited to using camera sensors and prior object information for affordance prediction. In this study, we introduce a unified framework called Tactile Affordance in Robot Synesthesia (TARS) for dexterous manipulation that employs robotic synesthesia through a unified point cloud representation. This framework harnesses the visuo-tactile affordance of objects, effectively merging comprehensive visual perception from external cameras with tactile feedback from local optical tactile sensors to handle tasks involving both contact and non-contact states. We simulated tactile perception in a simulation environment and trained task-oriented manipulation policies. Subsequently, we tested our approach on four distinct manipulation tasks, conducting extensive experiments to evaluate how different modules within our method optimize the performance of these manipulation policies.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 1","pages":"327-334"},"PeriodicalIF":4.6,"publicationDate":"2024-11-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142777880","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Trainable Hypervectors Encoding for Efficient 3D Loop-Closure Detection on Edge Devices

IF 4.6 2区计算机科学

IEEE Robotics and Automation Letters Pub Date : 2024-11-25 DOI: 10.1109/LRA.2024.3505820

Jeng-Lun Shieh;Shanq-Jang Ruan

{"title":"Trainable Hypervectors Encoding for Efficient 3D Loop-Closure Detection on Edge Devices","authors":"Jeng-Lun Shieh;Shanq-Jang Ruan","doi":"10.1109/LRA.2024.3505820","DOIUrl":"https://doi.org/10.1109/LRA.2024.3505820","url":null,"abstract":"Loop-closure detection plays a critical role in simultaneous localization and mapping (SLAM) systems. The primary task of loop-closure detection involves analyzing previously visited locations and correcting mapping errors, which typically stem from intrinsic noise in sensor data and accumulate over time. However, the burden of storing and querying/searching for previously visited information continues to increase with time. Consequently, reducing the amount of data stored becomes increasingly important. In this study, we propose a trainable hypervectors (THV) encoder, integrating quantization and a lookup table (LUT) to significantly enhance execution speed. Additionally, we employ a triangular mask in second-order pooling (SOP) for filtering extraneous features in the encoder and introduce binary quadruplet loss to efficiently train binary feature representations. We evaluate our method extensively on the KITTI, MulRan and Wild-Places datasets. The experiments demonstrate that our method substantially improves efficiency while maintaining accuracy. Moreover, our method effectively utilizes the 3D-NAND flash in-memory computing technique to improve execution performance.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 1","pages":"168-175"},"PeriodicalIF":4.6,"publicationDate":"2024-11-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142753865","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Adjustment Strategy Optimization and Design of the 3RPS-SPS Mechanism With Active and Passive Branches

IF 4.6 2区计算机科学

IEEE Robotics and Automation Letters Pub Date : 2024-11-25 DOI: 10.1109/LRA.2024.3506279

Yu Wang;Can Qiu;Xiaoyu He;Yundou Xu;Yongsheng Zhao

引用次数: 0

A Maneuverable Winding Gait for Snake Robots Based on a Delay-Aware Swing and Grasp Framework Combining Rules and Learning Methods

IF 4.6 2区计算机科学

IEEE Robotics and Automation Letters Pub Date : 2024-11-25 DOI: 10.1109/LRA.2024.3506274

Fengwei Sheng;Fuxi Wan;Chaoquan Tang;Xian Guo

{"title":"A Maneuverable Winding Gait for Snake Robots Based on a Delay-Aware Swing and Grasp Framework Combining Rules and Learning Methods","authors":"Fengwei Sheng;Fuxi Wan;Chaoquan Tang;Xian Guo","doi":"10.1109/LRA.2024.3506274","DOIUrl":"https://doi.org/10.1109/LRA.2024.3506274","url":null,"abstract":"Due to the high redundant degree of freedom characteristics of snake robots, their joint lever arms tend to be very long and often result in torque saturation, especially in the case of inter-tree motion. Traditional static planning methods based on curve segments connecting or wave functions are often limited by torque saturation and cannot meet the requirements of inter-tree motion of snake robots. Therefore, in this letter, a delay-aware swing and grasp framework combining rules and learning methods (DSG) is proposed for extending the inter-tree motion capability of snake robots. Specifically, first, to overcome the torque saturation problem, a joint torque direction determination rule is proposed to fully utilize the kinetic energy of the snake robot and enable the robot to move in the desired manner. Then, the DSG reduces the exploration space of the policy and guarantees the performance under delay, and based on which a maneuverable winding gait is designed to enable it to wrap around the target horizontal branch with maneuverability. Simulation and sufficient experiment results demonstrate that the proposed reinforcement learning (RL) controller has low torque requirement, high robustness under high delay, fast motion speed, and high generalizability.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 1","pages":"311-318"},"PeriodicalIF":4.6,"publicationDate":"2024-11-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142777624","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Beyond Gait: Seamless Knee Angle Prediction for Lower Limb Prosthesis in Multiple Scenarios

IF 4.6 2区计算机科学

IEEE Robotics and Automation Letters Pub Date : 2024-11-25 DOI: 10.1109/LRA.2024.3506220

Pengwei Wang;Yilong Chen;Wan Su;Jie Wang;Teng Ma;Haoyong Yu

{"title":"Beyond Gait: Seamless Knee Angle Prediction for Lower Limb Prosthesis in Multiple Scenarios","authors":"Pengwei Wang;Yilong Chen;Wan Su;Jie Wang;Teng Ma;Haoyong Yu","doi":"10.1109/LRA.2024.3506220","DOIUrl":"https://doi.org/10.1109/LRA.2024.3506220","url":null,"abstract":"Knee angle estimation plays a crucial role in the development of lower limb assistive devices, particularly prostheses. Current research in this area primarily focuses on stable gait movements, which limits applicability to real-world scenarios where human motion is far more complex. In this paper, we focus on estimating the knee angle in a broader range of activities beyond simple gait movements. By leveraging the synergy of whole-body dynamics, we propose a transformer-based probabilistic framework, the Angle Estimation Probabilistic Model (AEPM), which offers precise knee angle estimation across various daily movements. AEPM achieves an overall RMSE of 6.83 degrees, with an RMSE of 2.93 degrees in walking scenarios, outperforming the current state of the art with a 24.68% improvement in walking prediction accuracy. Additionally, our method can achieve seamless adaptation between different locomotion modes. Also, this model can be utilized to analyze the synergy between the knee and other joints. We reveal that the whole body movement has valuable information for knee movement, which can provide insights into designing sensors for prostheses.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 1","pages":"406-413"},"PeriodicalIF":4.6,"publicationDate":"2024-11-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142790253","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Are Doppler Velocity Measurements Useful for Spinning Radar Odometry?

IF 4.6 2区计算机科学

IEEE Robotics and Automation Letters Pub Date : 2024-11-22 DOI: 10.1109/LRA.2024.3505821

Daniil Lisus;Keenan Burnett;David J. Yoon;Richard Poulton;John Marshall;Timothy D. Barfoot

{"title":"Are Doppler Velocity Measurements Useful for Spinning Radar Odometry?","authors":"Daniil Lisus;Keenan Burnett;David J. Yoon;Richard Poulton;John Marshall;Timothy D. Barfoot","doi":"10.1109/LRA.2024.3505821","DOIUrl":"https://doi.org/10.1109/LRA.2024.3505821","url":null,"abstract":"Spinning, frequency-modulated continuous-wave (FMCW) radars with \u0000<inline-formula><tex-math>$360 ^{circ }$</tex-math></inline-formula>\u0000 coverage have been gaining popularity for autonomous-vehicle navigation. However, unlike ‘fixed’ automotive radar, commercially available spinning radar systems typically do not produce radial velocities due to the lack of repeated measurements in the same direction and the fundamental hardware setup. To make these radial velocities observable, we modified the firmware of a commercial spinning radar to use triangular frequency modulation. In this letter, we develop a novel way to use this modulation to extract radial Doppler velocity measurements from consecutive azimuths of a radar intensity scan, without any data association. We show that these noisy, error-prone measurements contain enough information to provide good ego-velocity estimates, and incorporate these estimates into different modern odometry pipelines. We extensively evaluate the pipelines on over \u0000<inline-formula><tex-math>$text{110 km}$</tex-math></inline-formula>\u0000 of driving data in progressively more geometrically challenging autonomous-driving environments. We show that Doppler velocity measurements improve odometry in well-defined geometric conditions and enable it to continue functioning even in severely geometrically degenerate environments, such as long tunnels.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 1","pages":"224-231"},"PeriodicalIF":4.6,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142761519","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

MotionGrasp: Long-Term Grasp Motion Tracking for Dynamic Grasping

IF 4.6 2区计算机科学

IEEE Robotics and Automation Letters Pub Date : 2024-11-22 DOI: 10.1109/LRA.2024.3504792

Nuo Chen;Xiao-Ming Wu;Guohao Xu;Jian-Jian Jiang;Zibo Chen;Wei-Shi Zheng

{"title":"MotionGrasp: Long-Term Grasp Motion Tracking for Dynamic Grasping","authors":"Nuo Chen;Xiao-Ming Wu;Guohao Xu;Jian-Jian Jiang;Zibo Chen;Wei-Shi Zheng","doi":"10.1109/LRA.2024.3504792","DOIUrl":"https://doi.org/10.1109/LRA.2024.3504792","url":null,"abstract":"Dynamic grasping, which aims to grasp moving objects in unstructured environment, is crucial for robotics community. Previous methods propose to track the initial grasps or objects by matching between the latest two frames. However, this neighbour-frame matching strategy ignores the long-term historical trajectory in tracking, resulting in accumulated error. To address this, we present a novel dynamic grasping framework, delicately taking the long-term trajectory into account in grasp tracking. To model the long-term trajectory well, we introduce the concept of Grasp Motion, the changes of grasps between frames, endowing the model with the dynamic modeling ability. Benefiting from the Grasp Motion, we are able to conduct accurate motion association, which associates the grasp generated in current frame to the long-term grasp trajectory and mitigates accumulated error. Moreover, since the generated grasps in current frame may not precisely align with the ground-truth grasp for the trajectory, which results in deviation when we put it into the trajectory for future association, we further design a motion alignment module to compensate it. Our experiments show that the MotionGrasp achieves great grasping performance in dynamic grasping, obtaining 20% increase compared to the previous SOTA method in the large-scale GraspNet-1billion dataset. Our experiments also verify that Grasp Motion is a key to the success of long-term modeling. The real-world experiments further verify the effectiveness of our method.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"10 1","pages":"796-803"},"PeriodicalIF":4.6,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142825933","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0