Elaheh Arefinia;Jayender Jagadeesan;Rajni V. Patel
{"title":"Machine-Learning-Based Multi-Modal Force Estimation for Steerable Ablation Catheters","authors":"Elaheh Arefinia;Jayender Jagadeesan;Rajni V. Patel","doi":"10.1109/TMRB.2024.3407590","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3407590","url":null,"abstract":"Catheter-based cardiac ablation is a minimally invasive procedure for treating atrial fibrillation (AF). Electrophysiologists perform the procedure under image guidance during which the contact force between the heart tissue and the catheter tip determines the quality of lesions created. This paper describes a novel multi-modal contact force estimator based on Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). The estimator takes the shape and optical flow of the deflectable distal section as two modalities since frames and motion between frames complement each other to capture the long context in the video frames of the catheter. The angle between the tissue and the catheter tip is considered a complement of the extracted shape. The data acquisition platform measures the two-degrees-of-freedom contact force and video data as the catheter motion is constrained in the imaging plane. The images are captured via a camera that simulates single-view fluoroscopy for experimental purposes. In this sensor-free procedure, the features of the images and optical flow modalities are extracted through transfer learning. Long Short-Term Memory Networks (LSTMs) with a memory fusion network (MFN) are implemented to consider time dependency and hysteresis due to friction. The architecture integrates spatial and temporal networks. Late fusion with the concatenation of LSTMs, transformer decoders, and Gated Recurrent Units (GRUs) are implemented to verify the feasibility of the proposed network-based approach and its superiority over single-modality networks. The resulting mean absolute error, which accounted for only 2.84% of the total magnitude, was obtained by collecting data under more realistic circumstances in contrast to previous research studies. The decrease in error is considerably better than that achieved by individual modalities and late fusion with concatenation. These results emphasize the practicality and relevance of utilizing a multi-modal network in real-world scenarios.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 3","pages":"1004-1016"},"PeriodicalIF":3.4,"publicationDate":"2024-03-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141965833","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Wenkang Fan;Wenjing Jiang;Hong Shi;Hui-Qing Zeng;Yinran Chen;Xiongbiao Luo
{"title":"Triple-Supervised Convolutional Transformer Aggregation for Robust Monocular Endoscopic Dense Depth Estimation","authors":"Wenkang Fan;Wenjing Jiang;Hong Shi;Hui-Qing Zeng;Yinran Chen;Xiongbiao Luo","doi":"10.1109/TMRB.2024.3407384","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3407384","url":null,"abstract":"Accurate deeply learned dense depth prediction remains a challenge to monocular vision reconstruction. Compared to monocular depth estimation from natural images, endoscopic dense depth prediction is even more challenging. While it is difficult to annotate endoscopic video data for supervised learning, endoscopic video images certainly suffer from illumination variations (limited lighting source, limited field of viewing, and specular highlight), smooth and textureless surfaces in surgical complex fields. This work explores a new deep learning framework of triple-supervised convolutional transformer aggregation (TSCTA) for monocular endoscopic dense depth recovery without annotating any data. Specifically, TSCTA creates convolutional transformer aggregation networks with a new hybrid encoder that combines dense convolution and scalable transformers to parallel extract local texture features and global spatial-temporal features, while it builds a local and global aggregation decoder to effectively aggregate global features and local features from coarse to fine. Moreover, we develop a self-supervised learning framework with triple supervision, which integrates minimum photometric consistency and depth consistency with sparse depth self-supervision to train our model by unannotated data. We evaluated TSCTA on unannotated monocular endoscopic images collected from various surgical procedures, with the experimental results showing that our methods can achieve more accurate depth range, more complete depth distribution, more sufficient textures, better qualitative and quantitative assessment results than state-of-the-art deeply learned monocular dense depth estimation methods.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 3","pages":"1017-1029"},"PeriodicalIF":3.4,"publicationDate":"2024-03-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141964814","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Daniel Costa;Gianni Borghesan;Mouloud Ourak;António Pedro Aguiar;Yuyu Cai;Emmanuel Vander Poorten
{"title":"Robotic Ultrasound-Guided Instrument Localization in Fetoscopy","authors":"Daniel Costa;Gianni Borghesan;Mouloud Ourak;António Pedro Aguiar;Yuyu Cai;Emmanuel Vander Poorten","doi":"10.1109/TMRB.2024.3407330","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3407330","url":null,"abstract":"Fetoscopic Endoluminal Tracheal Occlusion (FETO) is a minimally invasive fetal surgery (MIFS) aimed at mitigating the effects of Congenital Diaphragmatic Hernia (CDH). During FETO, a latex balloon is introduced in the fetal trachea using a fetoscope. Typically, this surgery is performed under ultrasound guidance which is provided by a sonographer who manually operates the ultrasound probe. This manual operation imposes a considerable physical and cognitive demand, placing a burden on the sonographer. This paper proposes a robotic ultrasound-based instrument tracking system that automates the probe position control while ensuring continuous visibility of the fetoscope in ultrasound images. The development of the proposed system is achieved with the completion of two tasks. Firstly, a series of fetoscope localization algorithms are developed and compared. Secondly, a task-based control for a robotic ultrasound system is developed. The localization algorithms’ performance is evaluated on annotated ultrasound datasets. The OEU-Net algorithm is selected based on this evaluation and is implemented in the instrument tracking system. The performance assessment of the tracking system shows that it is capable of tracking the fetoscope with a mean error below 4 mm. Thus, the developed system represents a significant advancement toward automatic robotic assistance for ultrasound guidance during FETO.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 3","pages":"806-817"},"PeriodicalIF":3.4,"publicationDate":"2024-03-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141965880","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Gregorio Tagliabue;Vishal Raveendranathan;Amedeo Gariboldi;Lennard Y. Hut;Andrea Zucchelli;Raffaella Carloni
{"title":"MyKnee: Mechatronic Design of a Novel Powered Variable Stiffness Prosthetic Knee","authors":"Gregorio Tagliabue;Vishal Raveendranathan;Amedeo Gariboldi;Lennard Y. Hut;Andrea Zucchelli;Raffaella Carloni","doi":"10.1109/TMRB.2024.3407194","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3407194","url":null,"abstract":"Powered prosthetic legs have the potential of significantly enhancing the mobility, independence, and overall quality of life of individuals with lower-limb amputation. Unfortunately, powered prosthesis are followed by the issue of their weight and limited battery life when compared to passive or semi-active prosthesis, which, conversely, lack of complex movement capabilities. In this paper, we present an innovative design and the development of a powered prosthetic knee joint, which is actuated by means of a compact variable stiffness actuator. This innovative and promising technology can provide adaptability to different activities of daily living, while also ensuring energy efficiency and maintaining a lightweight design. The key feature of this novel powered knee joint lies in the use of a mechanism that can vary the stiffness of the joint through newly designed non-linear elastic elements. By applying advanced finite element analysis in the design process, a robust device has been realized that could readily comply with the ISO 10328.2016 standard for structural integrity. This made the knee joint suitable for future clinical trials with people with above-knee amputation.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 3","pages":"1190-1201"},"PeriodicalIF":3.4,"publicationDate":"2024-03-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141965575","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Review of Proprioceptive Feedback Strategies for Upper-Limb Myoelectric Prostheses","authors":"Olivier Lecompte;Sofiane Achiche;Abolfazl Mohebbi","doi":"10.1109/TMRB.2024.3407532","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3407532","url":null,"abstract":"Upper extremity prostheses have seen significant technological advances in recent years, primarily with the advent of myoelectric prostheses and other designs incorporating mechatronic elements. Although they do not replicate the functionality of the natural hand, users now have a way of communicating their movement intentions to the prosthesis. However, the lack of physiological feedback from the device to the user can hinder proper integration of the prosthesis, and can be a contributing factor in the rejection of the technology. This is why experts point out that sensory feedback is one of the main missing features of commercial prostheses. The literature surrounding the restoration of somatosensation primarily discusses strategies to emulate tactile perception, but few address proprioceptive perception, which is the ability to perceive limb position and movement. Yet, proprioception has been shown to be a crucial element in object manipulation. This article offers an in-depth look into the literature surrounding proprioceptive perception restoration strategies for users of upper limb prostheses by identifying and comparing the documented strategies in relation to the concept of an optimal sensory feedback restoration device.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 3","pages":"930-939"},"PeriodicalIF":3.4,"publicationDate":"2024-03-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141965885","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
David G. Black;Amir Hossein Hadi Hosseinabadi;Nicholas Rangga Pradnyawira;Mika Nogami;Septimu E. Salcudean
{"title":"Low-Profile 6-Axis Differential Magnetic Force/Torque Sensing","authors":"David G. Black;Amir Hossein Hadi Hosseinabadi;Nicholas Rangga Pradnyawira;Mika Nogami;Septimu E. Salcudean","doi":"10.1109/TMRB.2024.3407350","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3407350","url":null,"abstract":"Force/torque sensing on hand-held tools enables control of applied forces, which is often essential in both tele-robotics and remote guidance of people. However, existing force sensors are either bulky, complex, or have insufficient load rating. This paper presents a novel 6 axis force-torque sensor based on differential magnetic field readings in a collection of low-profile sensor modules placed around a tool or device. The instrumentation is easy to install but nonetheless achieves good performance. A detailed mathematical model and optimization-based design procedure are also introduced. The modeling, simulation, and optimization of the force sensor are described and then used in the electrical and mechanical design and integration of the sensor into an ultrasound probe. Through a neural network-based nonlinear calibration, the sensor achieves average root-mean-square test errors of 0.41 N and 0.027 Nm compared to an off-the-shelf ATI Nano25 sensor, which are 0.80% and 1.16% of the full-scale range respectively. The sensor has an average noise power spectral density of less than 0.0001 N/\u0000<inline-formula> <tex-math>$sqrt {text {Hz}}$ </tex-math></inline-formula>\u0000, and a 95% confidence interval resolution of 0.0086 N and 0.063 Nmm. The practical readout rate is 1.3 kHz over USB serial and it can also operate over Bluetooth or Wi-Fi. This sensor can enable instrumentation of manual tools to improve the performance and transparency of teleoperated or autonomous systems.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 3","pages":"992-1003"},"PeriodicalIF":3.4,"publicationDate":"2024-03-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141965149","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ming Xuan Chua;Yoshiro Okubo;Shuhua Peng;Thanh Nho Do;Chun Hui Wang;Liao Wu
{"title":"Analysis of Fatigue-Induced Compensatory Movements in Bicep Curls: Gaining Insights for the Deployment of Wearable Sensors","authors":"Ming Xuan Chua;Yoshiro Okubo;Shuhua Peng;Thanh Nho Do;Chun Hui Wang;Liao Wu","doi":"10.1109/TMRB.2024.3407239","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3407239","url":null,"abstract":"A common challenge in Bicep Curls rehabilitation is muscle compensation, where patients adopt alternative movement patterns when the primary muscle group cannot act due to injury or fatigue, significantly decreasing the effectiveness of rehabilitation efforts. The problem is exacerbated by the growing trend toward transitioning from in-clinic to home-based rehabilitation, where constant monitoring and correction by physiotherapists are limited. Developing wearable sensors capable of detecting muscle compensation becomes crucial to address this challenge. This study aims to gain insights into the optimal deployment of wearable sensors through a comprehensive study of muscle compensation in Bicep Curls. We collect upper limb joint kinematics and surface electromyography signals (sEMG) from eight muscles in 12 healthy subjects during standard and fatigue stages. Two muscle synergies are derived from sEMG signals and are analyzed comprehensively along with joint kinematics. Our findings reveal a shift in the relative contribution of forearm muscles to shoulder muscles, accompanied by a significant increase in activation amplitude for both synergies. Additionally, more pronounced movement was observed at the shoulder joint during fatigue. These results suggest focusing on the shoulder muscle activities and joint motions when deploying wearable sensors to effectively detect compensatory movements.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 3","pages":"1147-1157"},"PeriodicalIF":3.4,"publicationDate":"2024-03-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141965580","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Soft Robotic Gastroscope for Low/Middle-Income Countries: Design and Preliminary Validation","authors":"Xuyang Ren;Yu Huan;Martina Finocchiaro;Matteo Cianchetti;Giacomo Lo Secco;Shuxin Wang;Paolo Dario;Anastasios Koulaouzidis;Alberto Arezzo;Gastone Ciuti","doi":"10.1109/TMRB.2024.3407364","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3407364","url":null,"abstract":"To reduce incidence and mortality, screening of the gastric cavity is crucial to diagnose early-stage cancers. Most cases are concentrated in low/middle-income countries (LMICs), where medical resources are limited. In this paper, we propose a miniaturized, disposable, and low-cost soft robotic gastroscope designed for screening in LMICs. The robotic platform is composed of i) a frontal soft-core module, ii) a flexible multi-lumen tether, and iii) an intuitive control handle, to provide a) a 180 deg bending angle, b) a 360 deg axial rotation, and c) linear movements with a 15 mm fine adjustment. Thanks to a single internal bending chamber, the diameter of the soft-core module and the tether are reduced to 7.2 mm and 4.3 mm, respectively. Mechanical performance, operational functionalities, and clinical dependability were successfully evaluated through in-vitro and ex-vivo experiments. In summary, given i) low-cost (i.e., ~25 USD), ii) low invasiveness, iii) high portability, and iv) intuitive control, the disposable soft gastroscope might have considerable clinical potential for widening gastric cancer screening in LMICs.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 3","pages":"1063-1072"},"PeriodicalIF":3.4,"publicationDate":"2024-03-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141966289","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Advancements in Soft Wearable Robots: A Systematic Review of Actuation Mechanisms and Physical Interfaces","authors":"Sajjad Hussain;Fanny Ficuciello","doi":"10.1109/TMRB.2024.3407374","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3407374","url":null,"abstract":"Soft actuators and robotic devices designed for rehabilitation and assistance are a rapidly growing field of research. Their inherent flexibility enhances comfort and usability without restricting the user’s natural range of motion. However, despite these advantages, there are still several challenges that need to be addressed before these systems can be commercialized. This paper presents a comprehensive review of the latest developments in soft wearable robots, also known as exosuits. Soft exosuits are composed of two main components: actuation mechanisms (how forces/torques are generated) and physical interfaces (how and where the robot is anchored to the body). This paper reviews the advances in these two areas, while categorizing exosuits based on the intended assisted joint, assisted degrees of freedom (DOF), and device type. The systematic literature review follows the PRISMA guidelines to summarize the relevant studies and investigate their related physical interface, actuation mechanism and its design. Several limitations were identified in these areas, and insights into potential future research directions are presented. In the future, the goal should be to develop an untethered assistive device that can provide assistance to multiple joints while having a low form factor, an intuitive and natural interface, and being comfortable for the user.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 3","pages":"903-929"},"PeriodicalIF":3.4,"publicationDate":"2024-03-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10542466","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141965169","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nauman Hafeez;Nikolaos Boulgouris;Philip Begg;Richard Irving;Chris Coulson;Hao Wu;Huan Jia;Xinli Du
{"title":"Real-Time Insertion Depth Tracking of Cochlear Implant Electrode Array With Bipolar Complex Impedance and Machine Intelligence","authors":"Nauman Hafeez;Nikolaos Boulgouris;Philip Begg;Richard Irving;Chris Coulson;Hao Wu;Huan Jia;Xinli Du","doi":"10.1109/TMRB.2024.3407355","DOIUrl":"https://doi.org/10.1109/TMRB.2024.3407355","url":null,"abstract":"Cochlear implants have significantly improved hearing for many as the most successful prosthesis, however, hearing outcomes vary. Uncertainty during electrode array (EA) insertion, including trauma and depth control, is one factor. To minimize radiation exposure from imaging methods like CT scans, this in-vitro study investigates the use of bipolar electrode impedance and artificial intelligent models to determine EA insertion depth. Complex impedance data was collected by inserting a commercial EA into a scaled-up 2D scala tympani model using a robotic feeder system. A support vector machine model produced a 98% classification accuracy for final insertion depth estimation. A CNN-LSTM hybrid model yielded 0.85 R-squared and 1.72 mm mean absolute error in depth estimation at each millimeter during a 25 mm insertion. This approach to depth assessment based on impedance may help with cochlear implant procedures and find use in other medical implant applications.","PeriodicalId":73318,"journal":{"name":"IEEE transactions on medical robotics and bionics","volume":"6 3","pages":"1245-1255"},"PeriodicalIF":3.4,"publicationDate":"2024-03-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141965685","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}