Anthony Opipari;Aravindhan K Krishnan;Shreekant Gayaka;Min Sun;Cheng-Hao Kuo;Arnie Sen;Odest Chadwicke Jenkins
{"title":"Configurable Embodied Data Generation for Class-Agnostic RGB-D Video Segmentation","authors":"Anthony Opipari;Aravindhan K Krishnan;Shreekant Gayaka;Min Sun;Cheng-Hao Kuo;Arnie Sen;Odest Chadwicke Jenkins","doi":"10.1109/LRA.2024.3486213","DOIUrl":"https://doi.org/10.1109/LRA.2024.3486213","url":null,"abstract":"This letter presents a method for generating large-scale datasets to improve class-agnostic video segmentation across robots with different form factors. Specifically, we consider the question of whether video segmentation models trained on generic segmentation data could be more effective for particular robot platforms \u0000<italic>if</i>\u0000 robot embodiment is factored into the data generation process. To answer this question, a pipeline is formulated for using 3D reconstructions (e.g. from HM3DSem (Yadav et al., 2023)) to generate segmented videos that are configurable based on a robot's embodiment (e.g. sensor type, sensor placement, and illumination source). A resulting massive RGB-D video panoptic segmentation dataset (MVPd) is introduced for extensive benchmarking with foundation and video segmentation models, as well as to support embodiment-focused research in video segmentation. Our experimental findings demonstrate that using MVPd for finetuning can lead to performance improvements when transferring foundation models to certain robot embodiments, such as specific camera placements. These experiments also show that using 3D modalities (depth images and camera pose) can lead to improvements in video segmentation accuracy and consistency.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11409-11416"},"PeriodicalIF":4.6,"publicationDate":"2024-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636257","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Serial Robot Kinematic Calibration Using a Novel Linear Finite Screw Deviation Model","authors":"Jaehyung Kim;Min Cheol Lee","doi":"10.1109/LRA.2024.3480521","DOIUrl":"https://doi.org/10.1109/LRA.2024.3480521","url":null,"abstract":"This letter introduces a novel method for calibrating serial robots using the Linearized Finite Screw Deviation (LFSD) model, aiming to minimize errors within the robot's workspace with a few end-effector postures. In contrast to prior studies that require several end-effector posture measurements, our proposed method presents a more resource-efficient approach suitable for practical applications. The proposed LFSD model stands out by enabling the identification of axis deviations with significantly fewer end-effector points compared to previous studies. This reduction in measurement points not only enhances applicability but also reduces the potential possibilities of noise arising from end-effector measurements. Moreover, the method's simple affine structure extends its applicability to the calibration of redundant robots. To validate the effectiveness of our proposed method, simulations and experiments with wire-encoder-based stereotactic devices were conducted on serial robots with five and six degrees of freedom.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"9 12","pages":"11433-11440"},"PeriodicalIF":4.6,"publicationDate":"2024-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10716461","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636429","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yash Chitalia;Achraj Sarma;Timothy A. Brumfiel;Nancy J. Deaton;Maxina Sheft;Jaydev P. Desai
{"title":"Model-Based Design of the COAST Guidewire Robot for Large Deflection","authors":"Yash Chitalia;Achraj Sarma;Timothy A. Brumfiel;Nancy J. Deaton;Maxina Sheft;Jaydev P. Desai","doi":"10.1109/LRA.2023.3286125","DOIUrl":"10.1109/LRA.2023.3286125","url":null,"abstract":"Minimally invasive endovascular procedures involve the manual placement of a guidewire, which is made difficult by vascular tortuosity and the lack of precise tip control. Steerable guidewire systems have been developed with tendon-driven, magnetic, and concentric tube actuation strategies to enable precise tip control, however, selecting machining parameters for such robots does not have a strict procedure. In this letter, we develop a systematic design procedure for selecting the tube pairs of the \u0000<underline>CO</u>\u0000axially \u0000<underline>A</u>\u0000ligned \u0000<underline>ST</u>\u0000eerable (COAST) guidewire robot. This includes the introduction of a mechanical model that accounts for micromachining-induced pre-curvatures with the goal of determining design parameters that reduce combined distal tip pre-curvature and minimize abrupt changes in actuated tip position for the COAST guidewire robot through selection of the best flexural rigidity between the tube pairs. We present adjustments in the kinematics modeling of COAST robot tip bending motion, and use these to characterize the bending behavior of the COAST robot for varying geometries of the micromachined tubes, with an average RMSE value for the tip position error of 0.816 mm in the validation study.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"8 9","pages":"5345-5352"},"PeriodicalIF":5.2,"publicationDate":"2023-06-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10151923","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10158369","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Margaret Rox;Daniel S. Esser;Mariana E. Smith;Tayfun Efe Ertop;Maxwell Emerson;Fabien Maldonado;Erin A. Gillaspie;Alan Kuntz;Robert J. Webster
{"title":"Toward Continuum Robot Tentacles for Lung Interventions: Exploring Folding Support Disks","authors":"Margaret Rox;Daniel S. Esser;Mariana E. Smith;Tayfun Efe Ertop;Maxwell Emerson;Fabien Maldonado;Erin A. Gillaspie;Alan Kuntz;Robert J. Webster","doi":"10.1109/LRA.2023.3267006","DOIUrl":"10.1109/LRA.2023.3267006","url":null,"abstract":"Toward the future goal of creating a lung surgery system featuring multiple tentacle-like robots, we present a new folding concept for continuum robots that enables them to squeeze through openings smaller than the robot's nominal diameter (e.g., the narrow space between adjacent ribs). This is facilitated by making the disks along the robot's backbone foldable.We also demonstrate that such a robot can feature not only straight, but also curved tendon routing paths, thereby achieving a diverse family of conformations. We find that the foldable robot performs comparably, from a kinematic perspective, to an identical non-folding continuum robot at varying deployment lengths. This work paves the way for future applications with a continuum robot that can fold and fit through smaller openings, with the potential to reduce invasiveness during surgical tasks.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"8 6","pages":"3494-3501"},"PeriodicalIF":5.2,"publicationDate":"2023-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10101847","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9708382","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yanzhou Wang;Ka-Wai Kwok;Kevin Cleary;Russell H. Taylor;Iulian Iordachita
{"title":"Flexible Needle Bending Model for Spinal Injection Procedures","authors":"Yanzhou Wang;Ka-Wai Kwok;Kevin Cleary;Russell H. Taylor;Iulian Iordachita","doi":"10.1109/LRA.2023.3239310","DOIUrl":"10.1109/LRA.2023.3239310","url":null,"abstract":"An in situ needle manipulation technique used by physicians when performing spinal injections is modeled to study its effect on needle shape and needle tip position. A mechanics-based model is proposed and solved using finite element method. A test setup is presented to mimic the needle manipulation motion. Tissue phantoms made from plastisol as well as porcine skeletal muscle samples are used to evaluate the model accuracy against medical images. The effect of different compression models as well as model parameters on model accuracy is studied, and the effect of needle-tissue interaction on the needle remote center of motion is examined. With the correct combination of compression model and model parameters, the model simulation is able to predict needle tip position within submillimeter accuracy.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"8 3","pages":"1343-1350"},"PeriodicalIF":5.2,"publicationDate":"2023-01-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10024841","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10164493","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zicong Wu;Mikel De Iturrate Reyzabal;S.M.Hadi Sadati;Hongbin Liu;Sebastien Ourselin;Daniel Leff;Robert K. Katzschmann;Kawal Rhode;Christos Bergeles
{"title":"Towards a Physics-Based Model for Steerable Eversion Growing Robots","authors":"Zicong Wu;Mikel De Iturrate Reyzabal;S.M.Hadi Sadati;Hongbin Liu;Sebastien Ourselin;Daniel Leff;Robert K. Katzschmann;Kawal Rhode;Christos Bergeles","doi":"10.1109/LRA.2023.3234823","DOIUrl":"10.1109/LRA.2023.3234823","url":null,"abstract":"Soft robots that grow through eversion/apical extension can effectively navigate fragile environments such as ducts and vessels inside the human body. This letter presents the physics-based model of a miniature steerable eversion growing robot. We demonstrate the robot's growing, steering, stiffening and interaction capabilities. The interaction between two robot-internal components is explored, i.e., a steerable catheter for robot tip orientation, and a growing sheath for robot elongation/retraction. The behavior of the growing robot under different inner pressures and external tip forces is investigated. Simulations are carried out within the SOFA framework. Extensive experimentation with a physical robot setup demonstrates agreement with the simulations. The comparison demonstrates a mean absolute error of 10–20% between simulation and experimental results for curvature values, including catheter-only experiments, sheath-only experiments and full system experiments. To our knowledge, this is the first work to explore physics-based modelling of a tendon-driven steerable eversion growing robot. While our work is motivated by early breast cancer detection through mammary duct inspection and uses our MAMMOBOT robot prototype, our approach is general and relevant to similar growing robots.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"8 2","pages":"1005-1012"},"PeriodicalIF":5.2,"publicationDate":"2023-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10008022","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9200147","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xihan Ma;Wen-Yi Kuo;Kehan Yang;Ashiqur Rahaman;Haichong K. Zhang
{"title":"A-SEE: Active-Sensing End-Effector Enabled Probe Self-Normal-Positioning for Robotic Ultrasound Imaging Applications","authors":"Xihan Ma;Wen-Yi Kuo;Kehan Yang;Ashiqur Rahaman;Haichong K. Zhang","doi":"10.1109/LRA.2022.3218183","DOIUrl":"10.1109/LRA.2022.3218183","url":null,"abstract":"Conventional manual ultrasound (US) imaging is a physically demanding procedure for sonographers. A robotic US system (RUSS) has the potential to overcome this limitation by automating and standardizing the imaging procedure. It also extends ultrasound accessibility in resource-limited environments with the shortage of human operators by enabling remote diagnosis. During imaging, keeping the US probe normal to the skin surface largely benefits the US image quality. However, an autonomous, real-time, low-cost method to align the probe towards the direction orthogonal to the skin surface without pre-operative information is absent in RUSS. We propose a novel end-effector design to achieve self-normal-positioning of the US probe. The end-effector embeds four laser distance sensors to estimate the desired rotation towards the normal direction. We then integrate the proposed end-effector with a RUSS system which allows the probe to be automatically and dynamically kept to normal direction during US imaging. We evaluated the normal positioning accuracy and the US image quality using a flat surface phantom, an upper torso mannequin, and a lung ultrasound phantom. Results show that the normal positioning accuracy is 4.17 \u0000<inline-formula><tex-math>$pm$</tex-math></inline-formula>\u0000 2.24 degrees on the flat surface and 14.67 \u0000<inline-formula><tex-math>$pm$</tex-math></inline-formula>\u0000 8.46 degrees on the mannequin. The quality of the RUSS collected US images from the lung ultrasound phantom was equivalent to that of the manually collected ones.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"7 4","pages":"12475-12482"},"PeriodicalIF":5.2,"publicationDate":"2022-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9932673","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9722499","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Deep-Learning to Map a Benchmark Dataset of Non-Amputee Ambulation for Controlling an Open Source Bionic Leg","authors":"Minjae Kim;Levi J. Hargrove","doi":"10.1109/LRA.2022.3194323","DOIUrl":"10.1109/LRA.2022.3194323","url":null,"abstract":"Powered lower-limb prosthetic devices may be becoming a promising option for amputation patients. Although various methods have been proposed to produce gait trajectories similar to those of non-disabled individuals, implementing these control methods is still challenging. It remains unclear whether these methods provide appropriate, safe, and intuitive locomotion as intended. This letter proposes the direct mapping of the voluntary movement of a residual limb (i.e., thigh) to the desired impedance parameters for amputated limbs (i.e., knee and ankle). The proposed model was learned from the gait trajectories of intact limb individuals from a publicly available biomechanics dataset, and was applied to control the prosthetic leg without post-tuning the network. Thus, the proposed method does not require training time with individuals with amputation nor configuration time for its use, and it provides a closely resembling gait trajectory of the intact limb. For preliminary testing, three able-bodied subjects participated in bypass tests. The proposed model accomplished intuitive and reliable level-ground walking at three different step lengths: self-selected, long-, and short-step lengths. The results indicate that intact benchmark data with different sensor configurations can be directly used to train the model to control prosthetic legs.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"7 4","pages":"10597-10604"},"PeriodicalIF":5.2,"publicationDate":"2022-07-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10010674/pdf/nihms-1828893.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9497825","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Onder Erin;Suraj Raval;Trevor J. Schwehr;Will Pryor;Yotam Barnoy;Adrian Bell;Xiaolong Liu;Lamar O. Mair;Irving N. Weinberg;Axel Krieger;Yancy Diaz-Mercado
{"title":"Enhanced Accuracy in Magnetic Actuation: Closed-Loop Control of a Magnetic Agent With Low-Error Numerical Magnetic Model Estimation","authors":"Onder Erin;Suraj Raval;Trevor J. Schwehr;Will Pryor;Yotam Barnoy;Adrian Bell;Xiaolong Liu;Lamar O. Mair;Irving N. Weinberg;Axel Krieger;Yancy Diaz-Mercado","doi":"10.1109/LRA.2022.3191047","DOIUrl":"10.1109/LRA.2022.3191047","url":null,"abstract":"Magnetic actuation holds promise for wirelessly controlling small, magnetic surgical tools and may enable the next generation of ultra minimally invasive surgical robotic systems. Precise torque and force exertion are required for safe surgical operations and accurate state control. Dipole field estimation models perform well far from electromagnets but yield large errors near coils. Thus, manipulations near coils suffer from severe (10x) field modeling errors. We experimentally quantify closed-loop magnetic agent control performance by using both a highly erroneous dipole model and a more accurate numerical magnetic model to estimate magnetic forces and torques for any given robot pose in 2D. We compare experimental measurements with estimation errors for the dipole model and our finite element analysis (FEA) based model of fields near coils. With five different paths designed for this study, we demonstrate that FEA-based magnetic field modeling reduces positioning root-mean-square (RMS) errors by 48% to 79% as compared with dipole models. Models demonstrate close agreement for magnetic field direction estimation, showing similar accuracy for orientation control. Such improved magnetic modelling is crucial for systems requiring robust estimates of magnetic forces for positioning agents, particularly in force-sensitive environments like surgical manipulation.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"7 4","pages":"9429-9436"},"PeriodicalIF":5.2,"publicationDate":"2022-07-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9762677/pdf/nihms-1825983.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10433584","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Characterizing Prosthesis Control Fault During Human-Prosthesis Interactive Walking Using Intrinsic Sensors","authors":"Amirreza Naseri;Ming Liu;I-Chieh Lee;Wentao Liu;He Huang","doi":"10.1109/LRA.2022.3186503","DOIUrl":"10.1109/LRA.2022.3186503","url":null,"abstract":"The physical interactions between wearable lower limb robots and humans have been investigated to inform effective robot design for walking augmentation. However, human-robot interactions when internal faults occur within robots have not been systematically reported, but it is essential to improve the robustness of robotic devices and ensure the user’s safety. This letter aims to (1) present a methodology to characterize the behavior of the robotic transfemoral prosthesis as an effective wearable robot platform while interacting with the users in the presence of internal faults, and (2) identify the potential data sources for accurate detection of the prosthesis fault. We first obtained the human perceived response in terms of their walking stability when the prosthesis control fault (inappropriate intrinsic control output/command) was emulated/applied in level-ground walking. Then the measurements and their features, obtained from the transfemoral prosthesis, were examined for the emulated faults that elicited a sense of instability in human users. The optimal features that contributed the most in separating faulty interaction from the normal walking condition were determined using two machine-learning-based approaches: One-Class Support Vector Machine (OCSVM) and Mahalanobis Distance (MD) classifier. The OCSVM anomaly detector could achieve an average sensitivity of 85.7% and an average false alarm rate of 1.7% with a reasonable detecting time of 147.6 ms for detecting emulated control errors among all subjects. The result demonstrates the potential of using machine-learning-based schemes in identifying prosthesis control faults based on intrinsic sensors on the prosthesis. This study presents a procedure to study human-robot fault tolerance and inform the future design of robust prosthesis control.","PeriodicalId":13241,"journal":{"name":"IEEE Robotics and Automation Letters","volume":"7 3","pages":"8307-8314"},"PeriodicalIF":5.2,"publicationDate":"2022-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9881473/pdf/nihms-1822376.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9735608","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}