Zhen Deng;Xiaoxiao Wei;Chuanchuan Pan;Guotao Li;Ying Hu
{"title":"Shared Control of Tendon-Driven Continuum Robots Using Visibility-Guaranteed Optimization for Endoscopic Surgery","authors":"Zhen Deng;Xiaoxiao Wei;Chuanchuan Pan;Guotao Li;Ying Hu","doi":"10.1109/TMRB.2024.3381371","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3381371","url":null,"abstract":"Tendon-driven continuum robots (TDCRs) with mechanical compliance have gained popularity in natural orifice transluminal endoscopic surgery (NOTES). Teleoperation problems of the TDCRs involve performance objectives in addition to the visibility constraint. Handling the coupling between potentially conflicting objectives and the visibility constraint remains challenging for surgeons when operating TDCRs. This paper presents a shared control method to assist in the teleoperation of the TDCRs, which guarantees visual targets remain within the field of view (FoV) of the TDCR. The visibility constraint is explicitly defined using a zeroing control barrier function, which is specified in terms of the forward invariance of a visible set. To ensure accuracy, the Jacobian matrix of the system is approximated online using sensing data. Then, the visibility constraint, along with the robot’s physical constraints, is integrated into a quadratic program (QP) framework. This allows for the optimization of the control input of the operator subject to constraints, thus preserving visibility. Finally, simulations and experiments were conducted to demonstrate the effectiveness of the proposed approach under two teleoperation modes. The results show that the proposed method achieved a reduction of approximately 70% in ITP and 43% in MAE compared to direct teleoperation.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140820373","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dean M. Corva;Brenna Parke;Alyssa West;Egan H. Doeven;Scott D. Adams;Susannah J. Tye;Parastoo Hashemi;Michael Berk;Abbas Z. Kouzani
{"title":"SmartStim: An Artificial Intelligence Enabled Deep Brain Stimulation Device","authors":"Dean M. Corva;Brenna Parke;Alyssa West;Egan H. Doeven;Scott D. Adams;Susannah J. Tye;Parastoo Hashemi;Michael Berk;Abbas Z. Kouzani","doi":"10.1109/TMRB.2024.3381341","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3381341","url":null,"abstract":"Deep brain stimulation (DBS) has demonstrated therapeutic efficacy in the treatment of neurological and psychiatric disorders. Currently, DBS devices employ an ‘open-loop’ configuration, requiring manual adjustment of electrical stimulation to address patient needs. For this reason, closed-loop DBS is being developed, delivering appropriate treatment on-demand based on internal signal monitoring. A key challenge in current research is the complexity of interpreting the measured signals and delivering appropriate interventions, currently no miniaturised closed-loop DBS device has on-board artificial intelligence (AI) to meet this need. This paper presents a new miniaturised device, named SmartStim, that uses AI to monitor dynamically changing brain biomarkers. In addition, the AI decides if the output stimulator is required for treatment. This device has two key components: the hardware module (neural sensor unit, processor, and neurostimulator) and a software module (data processing, AI, and firmware). The neural sensor unit is comprised of two subcomponents. The first is a potentiostat that can perform impedance analysis, and the second is a dedicated fast scan cyclic voltammetry (FSCV) front-end that can perform scan rates up to 1000 V/s. This device can output current-controlled stimulation waveforms in a frequency range of 5 Hz – 200 Hz, a current range of \u0000<inline-formula> <tex-math>$1~mu text{A}$ </tex-math></inline-formula>\u0000 to 10 mA, with active charge balancing. Five experiments were conducted to validate SmartStim: static resistive load test, emulated brain resistance test, static electrochemical cell test, impedance test, and dynamic serotonin test. These experiments confirm the potential for SmartStim to identify neurochemical patterns in a mouse brain using AI.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140820296","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"PPFormer: A Novel Model for Polyp Segmentation in Digestive Endoscopy","authors":"Wenxin Chen;Kaifeng Wang;Chao Qian;Xue Li;Changsheng Li;Xingguang Duan","doi":"10.1109/TMRB.2024.3381330","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3381330","url":null,"abstract":"Polyp segmentation is a pivotal task in the field of medical image processing. We devised a more effective deep learning model (PPFormer) that seamlessly integrates pyramid pooling module with transformer. This integration significantly improves the model’s ability to restore intricate details during the decoding phase. Additionally, we rethinked the importance of multi-scale feature maps within the model and thoughtfully designed two pruning strategies to target the elimination of redundant and mis-segmented feature maps, resulting in improved segmentation quality. In this paper, we aim to explore methods to enhance the performance of the polyp segmentation model. We conducted experiments on three different polyp segmentation datasets, and the model presented in this paper consistently exhibited exceptional performance. Through visual experiments, the model demonstrated an enhanced capacity to handle the edge of the polyp, indicating an improved capability to restore image details during the decoding process. In terms of quantitative metrics, PPFormer achieved outstanding results in segmentation-related indicators. For example, it obtained mIoU scores of 91.67%, 92.09%, and 93.19% on the Kvasir-SEG, CVC-ClinicDB, and CVC-300 datasets, respectively.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140820247","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Prognosis of Tissue Stiffness Through Multilayer Perceptron Technique With Adaptive Learning Rate in Minimal Invasive Surgical Procedures","authors":"Bulbul Behera;M. Felix Orlando;R. S. Anand","doi":"10.1109/TMRB.2024.3377371","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3377371","url":null,"abstract":"Flexible needles are navigated through anatomical pathways to reach deep seated tissues for minimally invasive surgical procedures. During such risky navigation, anatomical obstacles and the target malignant tissue regions could be dislodged due to excessive stress upon needle-tissue interaction. Hence, knowledge about the interactive forces is essential to execute a safe needle steering procedure during percutaneous cancerous treatments. This paper proposes an adaptive learning rate based multilayer perceptron technique for determining Young’s modulus of tissue at each stage of navigation and then utilizing this value to predict the deflection of flexible needle in tissue environment. To validate the accuracy of predictions, an energy-based model is incorporated into the analysis. Simulation results demonstrate that the proposed model can efficiently predict Young’s modulus in just 0.59 secs. To further validate the efficacy of this novel methodology, extensive experimental studies are conducted, including rigorous statistical analysis using ANOVA with a 5% accuracy level. The effectiveness of neural networks is underscored through a two-sample t-test across 5 different trials, revealing consistently low mean absolute errors, typically below 1.5 kPa, except in trial 3. These findings highlight the reliability of the proposed novel technique in predicting Young’s modulus and ensuring accurate needle deflection predictions.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140820290","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Christian Tamantini;Francesca Cordella;Nevio Luigi Tagliamonte;Ilenia Pecoraro;Iolanda Pisotta;Alessandra Bigioni;Federica Tamburella;Matteo Lorusso;Marco Molinari;Loredana Zollo
{"title":"A Data-Driven Fuzzy Logic Method for Psychophysiological Assessment: An Application to Exoskeleton-Assisted Walking","authors":"Christian Tamantini;Francesca Cordella;Nevio Luigi Tagliamonte;Ilenia Pecoraro;Iolanda Pisotta;Alessandra Bigioni;Federica Tamburella;Matteo Lorusso;Marco Molinari;Loredana Zollo","doi":"10.1109/TMRB.2024.3377453","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3377453","url":null,"abstract":"Multimodal physiological monitoring and related estimation of the PsychoPhysiological (PP) state play an essential role in investigating the physical and cognitive workload of people executing a motor task. The aim of this work was to develop a data-driven Fuzzy Logic method to estimate four PP indicators, i.e., Energy Expenditure, Fatigue, Attention, and Stress, and test it in a study including ten healthy participants walking while assisted by a lower limb treadmill-based exoskeleton. PP indicators were compared with participants’ self-reported evaluation of the human-robot interaction experience following the administration of a dedicated questionnaire. Results from a correlation analysis demonstrated that the output of the Fuzzy Logic method was consistent with the participants’ subjective assessment.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10477280","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140820291","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Robotic Systems Design in Endovascular Treatment","authors":"Naner Li;Yiwei Wang;Huan Zhao;Han Ding","doi":"10.1109/TMRB.2024.3377374","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3377374","url":null,"abstract":"Minimally invasive endovascular interventional surgery has become the primary surgical method for treating cardiovascular and cerebrovascular diseases. Robotic technology has been proposed as a means to enhance surgical procedures and improve physician experience. Despite the development of related technologies and the proposal of many robotic endovascular intervention systems and commercially available products in recent decades, widespread adoption in routine practice has been obstructed. To mature commercial robotic endovascular intervention systems, improvements in structure design, haptic feedback, autonomy, and other areas are crucial. Autonomous navigation and untethered microrobots have the potential to revolutionize the future of robotic endovascular intervention system innovation. This review highlights the challenges in the design of robotic systems for endovascular treatment, offers perspectives for future robotic system design, and encourages collaboration between engineers and physicians to expedite the use of these systems in clinical practice and explore new possibilities.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140820398","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Examination of Biofeedback to Support the Use of Upper-Extremity Exoskeletons Under Proportional Myoelectric Control","authors":"Xiangyu Peng;Leia Stirling","doi":"10.1109/TMRB.2024.3377278","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3377278","url":null,"abstract":"Exoskeletons have the potential to assist individuals in completing daily tasks and augment industrial workers in labor-intensive jobs. While previous studies have shown the capability of powered upper limb exoskeletons to reduce muscle effort and maintain task performance in continuous cyclical movements, their effectiveness in natural movements that contain both dynamic and static tasks remains uncertain. This study aimed to investigate the impact of visual and haptic electromyography (EMG) biofeedback on participants \u0000<inline-formula> <tex-math>$(n=36)$ </tex-math></inline-formula>\u0000 while they performed a target position matching task with a powered upper limb exoskeleton. Our hypothesis was that users could benefit from the biofeedback to minimize muscle effort and use the exoskeleton more effectively. However, the results indicated that the biofeedback did not reduce muscle effort in participants, but it had a positive impact on the smoothness of participants’ extension movements. The challenge of reducing muscle effort appeared to stem from participants experiencing difficulty in relaxing their muscles, even when the exoskeleton provided support for the task or maintained the desired posture. Nevertheless, participant feedback supported that biofeedback might enhance their satisfaction with exoskeleton usage, which is a crucial factor in promoting long-term acceptance. These findings provide a foundation for future research in user training methods and controller development for exoskeletons.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140820233","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Bin Li;Bo Lu;Hongbin Lin;Yaxiang Wang;Fangxun Zhong;Qi Dou;Yun-Hui Liu
{"title":"On the Monocular 3-D Pose Estimation for Arbitrary Shaped Needle in Dynamic Scenes: An Efficient Visual Learning and Geometry Modeling Approach","authors":"Bin Li;Bo Lu;Hongbin Lin;Yaxiang Wang;Fangxun Zhong;Qi Dou;Yun-Hui Liu","doi":"10.1109/TMRB.2024.3377357","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3377357","url":null,"abstract":"Image-guided needle pose estimation is crucial for robotic autonomous suturing, but it poses significant challenges due to the needle’s slender visual projection and dynamic surgical environments. Current state-of-the-art methods rely on additional prior information (e.g., in-hand grasp, accurate kinematics, etc.) to achieve sub-millimeter accuracy, hindering their applicability in varying surgical scenes. This paper presents a new generic framework for monocular needle pose estimation: Visual learning network for efficient geometric primitives extraction and novel geometry model for accurate pose recovery. To capture needle’s primitives precisely, we introduce a morphology-based mask contour fusion mechanism in a multi-scale manner. We then establish a novel state representation for needle pose and develop a physical projection model to derive its relationship with the primitives. An anti-occlusion objective is formulated to jointly optimize the pose and bias of inference primitives, achieving sub-millimeter accuracy under occlusion scenarios. Our approach requires neither CAD model nor circular shape assumption and can extensively estimate poses of other small planar axisymmetric objects. Experiments on ex-/in-vivo scenarios validate the accuracy of estimated intermediate primitives and final poses of needles. We further deploy our framework on the dVRK platform for automatic and precise needle manipulations, demonstrating the feasibility for use in robotic surgery.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140820361","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Explorations of Autonomous Prosthetic Grasping via Proximity Vision and Deep Learning","authors":"E. Mastinu;A. Coletti;J. van den Berg;C. Cipriani","doi":"10.1109/TMRB.2024.3377530","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3377530","url":null,"abstract":"The traumatic loss of a hand is usually followed by significant psychological, functional and rehabilitation challenges. Even though much progress has been reached in the past decades, the prosthetic challenge of restoring the human hand functionality is still far from being achieved. Autonomous prosthetic hands showed promising results and wide potential benefit, a benefit that must be still explored and deployed. Here, we hypothesized that a combination of a radar sensor and a low-resolution time-of-flight camera can be sufficient for object recognition in both static and dynamic scenarios. To test this hypothesis, we analyzed via deep learning algorithms HANDdata, a human-object interaction dataset with particular focus on reach-to-grasp actions. Inference testing was also performed on unseen data purposely acquired. The analyses reported here, broken down to gradually increasing levels of complexity, showed a great potential of using such proximity sensors as alternative or complementary solution to standard camera-based systems. In particular, integrated and low-power radar can be a potential key technology for next generation intelligent and autonomous prostheses.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10472622","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140820250","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Steerable Cross-Axis Notched Continuum Manipulator for Endobronchial Intervention","authors":"Xiaojie Ai;Yilin Cai;Anzhu Gao;Weidong Chen","doi":"10.1109/TMRB.2024.3377359","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3377359","url":null,"abstract":"Achieving the balance between stiffness and range of motion (ROM) in continuum manipulators is a primary design challenge. To tackle this design trade-off, this paper introduces a novel notched-tube continuum manipulator (NTCM) called the Steerable Cross-axis Notched (SCAN) manipulator. It achieves this by integrating asymmetric cross-axis notches into a pair of concentric nitinol tubes. Two pairs of cross-tilted beams are positioned within each segment, thereby extending the length of the flexural members. When compared to traditional NTCM with vertically configured beams (termed as v-NTCM), the SCAN manipulator (SCANM) exhibits a greater maximum bending angle for the same level of bending stiffness. Furthermore, the SCANM exhibits greater bending stiffness in comparison to the v-NTCM with the same maximum bending angle. Subsequently, a mechanical model for the SCANM, accounting for external tip load and tendon friction, is developed. Additionally, a multi-objective optimization is carried out to identify the optimal structural performance. Through model analysis and comparisons, this paper also elucidates the distinct advantages offered by the SCANM. Model verification experiments and stiffness testing experiments are conducted to quantify both the model’s accuracy and stiffness of the SCANM. Finally, an endobronchial grasping and a laser ablation experiment are conducted to demonstrate the practical feasibility of the SCANM for clinical applications.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140820244","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}