{"title":"An explainable deep learning model for automated classification and localization of microrobots by functionality using ultrasound images","authors":"Ferhat Sadak","doi":"10.1016/j.robot.2024.104841","DOIUrl":"10.1016/j.robot.2024.104841","url":null,"abstract":"<div><div>The rapid advancements of untethered microrobots offer exciting opportunities in fields such as targeted drug delivery and minimally invasive surgical procedures. However, several challenges remain, especially in achieving precise localization and classification of microrobots within living organisms using ultrasound (US) imaging. Current US-based detection algorithms often suffer from inaccurate visual feedback, causing positioning errors. This paper presents a novel explainable deep learning model for the localization and classification of eight different types of microrobots using US images. We introduce the Attention-Fused Bottleneck Module (AFBM), which enhances feature extraction and improves the performance of microrobot classification and localization tasks. Our model consistently outperforms baseline models such as YOLOR, YOLOv5-C3HB, YOLOv5-TBH, YOLOv5 m, and YOLOv7. The proposed model achieved mean Average Precision (mAP) of 0.861 and 0.909 at an IoU threshold of 0.95 which is 2% and 1.5% higher than the YOLOv5 m model in training and testing, respectively. Multi-thresh IoU analysis was performed at IoU thresholds of 0.6, 0.75, and 0.95, and demonstrated that the microrobot localization accuracy of our model is superior. A robustness analysis was performed based on high and low frequencies, gain, and speckle in our test data set, and our model demonstrated higher overall accuracy. UsingScore-CAM in our framework enhances interpretability, allowing for transparent insights into the model’s decision-making process. Our work signifies a notable advancement in microrobot classification and detection, with potential applications in real-world scenarios using the newly available USMicroMagset dataset for benchmarking.</div></div>","PeriodicalId":49592,"journal":{"name":"Robotics and Autonomous Systems","volume":"183 ","pages":"Article 104841"},"PeriodicalIF":4.3,"publicationDate":"2024-10-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142572215","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ivan Moskalenko , Anastasiia Kornilova , Gonzalo Ferrer
{"title":"Visual place recognition for aerial imagery: A survey","authors":"Ivan Moskalenko , Anastasiia Kornilova , Gonzalo Ferrer","doi":"10.1016/j.robot.2024.104837","DOIUrl":"10.1016/j.robot.2024.104837","url":null,"abstract":"<div><div>Aerial imagery and its direct application to visual localization is an essential problem for many Robotics and Computer Vision tasks. While Global Navigation Satellite Systems (GNSS) are the standard default solution for solving the aerial localization problem, it is subject to a number of limitations, such as, signal instability or solution unreliability that make this option not so desirable. Consequently, visual geolocalization is emerging as a viable alternative. However, adapting <em>Visual Place Recognition</em> (VPR) task to aerial imagery presents significant challenges, including weather variations and repetitive patterns. Current VPR reviews largely neglect the specific context of aerial data. This paper introduces a methodology tailored for evaluating VPR techniques specifically in the domain of aerial imagery, providing a comprehensive assessment of various methods and their performance. However, we not only compare various VPR methods, but also demonstrate the importance of selecting appropriate zoom and overlap levels when constructing map tiles to achieve maximum efficiency of VPR algorithms in the case of aerial imagery. The code is available on our GitHub repository — <span><span>https://github.com/prime-slam/aero-vloc</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49592,"journal":{"name":"Robotics and Autonomous Systems","volume":"183 ","pages":"Article 104837"},"PeriodicalIF":4.3,"publicationDate":"2024-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142553696","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mingxuan Ding , Qinyun Tang , Kaixin Liu , Xi Chen , Dake Lu , Changda Tian , Liquan Wang , Yingxuan Li , Gang Wang
{"title":"Advancements in amphibious robot navigation through wheeled odometer uncertainty extension and distributed information fusion","authors":"Mingxuan Ding , Qinyun Tang , Kaixin Liu , Xi Chen , Dake Lu , Changda Tian , Liquan Wang , Yingxuan Li , Gang Wang","doi":"10.1016/j.robot.2024.104839","DOIUrl":"10.1016/j.robot.2024.104839","url":null,"abstract":"<div><div>The advancement and safeguarding of the water-land interface region is of paramount importance, and amphibious robots with the capacity for autonomous operation can play a pivotal role in this domain. However, the inability of the majority of reliable navigation sensors to adapt to the water-land interface environment presents a significant challenge for amphibious robots, as obtaining positional information is crucial for autonomous operation. To address this issue, we have proposed a positioning and navigation framework, designated as NAWR (Navigation Algorithm for Amphibious Wheeled Robots), with the objective of enhancing the navigation capabilities of amphibious robots. Firstly, a method for representing the odometer's confidence based on a simplified wheel-terrain interaction model has been developed. This method quantitatively assesses the reliability of each odometer by estimating the slip rate. Secondly, we have introduced an improved split covariance intersection filter (I-SCIF), which maximizes the utilization of navigation information sources to enhance the accuracy of positional estimation. Finally, we will integrate these two methods to form the NAWR framework and validate the effectiveness of the proposed methods through multiple robot field trials. The results from both field trials and ablation tests collectively demonstrate that the modules and overall approach within the NAWR framework effectively enhance the navigation capabilities of amphibious robots.</div></div>","PeriodicalId":49592,"journal":{"name":"Robotics and Autonomous Systems","volume":"183 ","pages":"Article 104839"},"PeriodicalIF":4.3,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142586363","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Michele Perrelli, Francesco Lago, Salvatore Garofalo, Luigi Bruno, Domenico Mundo, Giuseppe Carbone
{"title":"A critical review and systematic design approach for innovative upper-limb rehabilitation devices","authors":"Michele Perrelli, Francesco Lago, Salvatore Garofalo, Luigi Bruno, Domenico Mundo, Giuseppe Carbone","doi":"10.1016/j.robot.2024.104835","DOIUrl":"10.1016/j.robot.2024.104835","url":null,"abstract":"<div><div>This paper conducts a thorough literature review and assessment of prevailing upper-limb rehabilitation devices, scrutinizing their strengths and limitations. The focus of this work is mainly on soft exosuit devices but some rigid and hybrid exoskeleton devices are also discussed as a comparative mean. Subsequently, this manuscript delineates explicit design guidelines with the intent of fostering a systematic approach toward innovation in the realm of upper-limb rehabilitation technology. Through an examination of current concepts and technological paradigms, this study seeks to contribute nuanced insights aimed at optimizing both efficacy and user experience in rehabilitation device design. The culmination of this critical analysis results in the proposal of a systematic design procedure to inform and influence the trajectory of specific user-tailored innovations within the domain of upper-limb rehabilitation devices.The proposed approach enables the identification of features and weaknesses in existing devices, facilitating also the design of innovative solutions for unsolved issues in the field of wearable robotics. A design example is presented to clarify the proposed design procedure.</div></div>","PeriodicalId":49592,"journal":{"name":"Robotics and Autonomous Systems","volume":"183 ","pages":"Article 104835"},"PeriodicalIF":4.3,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142527644","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zhifeng Huang , Runqiao Zhou , Ruiyuan Huang , Jun Ota
{"title":"Differential kinematics of a single-point-suspended manipulator with center-of-mass shift compensation","authors":"Zhifeng Huang , Runqiao Zhou , Ruiyuan Huang , Jun Ota","doi":"10.1016/j.robot.2024.104820","DOIUrl":"10.1016/j.robot.2024.104820","url":null,"abstract":"<div><div>The single cable-suspended manipulator is suitable for special occasions such as aerial operation tasks of unmanned aerial vehicles (UAVs) and deep-well search and rescue. However, due to the lack of complete constraints at the base, the manipulator will have errors in end position due to the center-of-mass offset during the motion. In this paper, the model of CoM shifting is established, and the Jacobian matrix is improved based on this model, to realize the differential kinematics solution for the single cable-suspended manipulator. In addition, by introducing the constraint of CoM shift in the Jacobian matrix, it makes it possible to synchronize the planning of the motion of the end and the center of mass. This can effectively avoid the wobbling of the manipulator in the presence of elasticity or instability at the suspension point. Both simulation and prototype experiments effectively verify the effectiveness of the proposed method. Using the method of this paper, the average error of the trajectories in the z-axis and x-axis can be reduced from 27.0 ± 2.6 mm to 5.6 ± 3.4 mm, and 43.0 ± 64.2 mm to 3.3 ± 4.8 mm, respectively.</div></div>","PeriodicalId":49592,"journal":{"name":"Robotics and Autonomous Systems","volume":"183 ","pages":"Article 104820"},"PeriodicalIF":4.3,"publicationDate":"2024-10-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142553195","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Bo Fu , Yuming Chen , Yi Quan , Xilin Zhou , Chaoshun Li
{"title":"Bidirectional artificial potential field-based ant colony optimization for robot path planning","authors":"Bo Fu , Yuming Chen , Yi Quan , Xilin Zhou , Chaoshun Li","doi":"10.1016/j.robot.2024.104834","DOIUrl":"10.1016/j.robot.2024.104834","url":null,"abstract":"<div><div>Ant colony optimization (ACO) is a common approach for addressing mobile robot path planning problems. However, it still encounters some challenges including slow convergence speed, susceptibility to local optima, and a tendency to falling into traps. We propose a bidirectional artificial potential field-based ant colony optimization (BAPFACO) algorithm to solve these issues. First, the bidirectional artificial potential field is introduced to initialize the grid environment model and restrict direction selection to jump out of the trap. Second, an adaptive heuristic function is presented to strengthen directionality of the algorithm and reduce the turning times. Third, a pseudo-random state transition rule based on potential difference between starting and ending nodes is developed to accelerate convergence speed. Finally, an improved pheromone update strategy incorporating pheromone diffusion mechanism and elite ants update strategy is proposed to help getting out of local optima. To demonstrate the advantages of BAPFACO, the validation of the performance in six different complexity environments and comparative experiments with other conventional search algorithms and ACO variants are conducted. The results of experiment show that compared to various ACO variants, BAPFACO have advantages in terms of reducing the turning times, shortening path length, improving convergence speed and avoiding ant loss. In complex environments, compared to IHMACO, the average path length enhancement percentage (<em>PLE</em>) of BAPFACO is 20.98%, the average iterations enhancement percentage (<em>IE</em>) of BAPFACO is 20.00% and the average turning times enhancement percentage (<em>TE</em>) of BAPFACO is 49.43%. These results firmly demonstrate the efficiency and practicality of the BAPFACO algorithm for mobile robot in path planning.</div></div>","PeriodicalId":49592,"journal":{"name":"Robotics and Autonomous Systems","volume":"183 ","pages":"Article 104834"},"PeriodicalIF":4.3,"publicationDate":"2024-10-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142658822","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Inspection robot GPS outages localization based on error Kalman filter and deep learning","authors":"Yansheng Li, Haoyang Yu, Lingli Xiao, Yiyang Yuan","doi":"10.1016/j.robot.2024.104824","DOIUrl":"10.1016/j.robot.2024.104824","url":null,"abstract":"<div><div>In urban environments, inspection robots face complex terrain and variable motion states, posing high demands on their positioning systems. Although the integration of Micro-Electro-Mechanical Systems Inertial Navigation Systems (MEMS-INS) with the Global Positioning System (GPS) provides continuous positioning information, high buildings and tunnels in cities can block GPS signals, leading to signal interruptions and increased positioning errors. During GPS outages, MEMS-INS gradually accumulates errors, severely affecting positioning accuracy. To address this issue, this paper proposes an adaptive error state Kalman Filter (AESKF), which employs an adaptive mechanism to eliminate the noise impact of MEMS-INS and reduce reliance on the process model. Additionally, a deep learning framework based on the Self-Attention mechanism of the Transformer and a custom loss function Long Short-Term Memory (LSTM) module is proposed to predict position increments of the inspection robot. Combining AESKF with Transformer-LSTM achieves optimized positioning accuracy of the inspection robot during GPS outages in dynamic urban environments. Simulation and practical experimental results demonstrate that the combination of AESKF and Transformer-LSTM significantly improves positioning accuracy. Compared to other mature methods, the Root Mean Square Error (RMSE) of positioning is reduced by up to 83.64 % in the north direction and 89.56 % in the east direction. When the GPS signal interruption lasts for 10 s and 60 s, the maximum position error standard deviation (STD) is 0.1186 m and 1.0417 m, respectively.</div></div>","PeriodicalId":49592,"journal":{"name":"Robotics and Autonomous Systems","volume":"183 ","pages":"Article 104824"},"PeriodicalIF":4.3,"publicationDate":"2024-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142553697","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Wendong Wang , Chenyang Wang , Xiaoqing Yuan , Songyun Xie , Jinming Liu
{"title":"Design of exoskeleton brain-like intelligent trajectory prediction model based on echo state network","authors":"Wendong Wang , Chenyang Wang , Xiaoqing Yuan , Songyun Xie , Jinming Liu","doi":"10.1016/j.robot.2024.104836","DOIUrl":"10.1016/j.robot.2024.104836","url":null,"abstract":"<div><div>Traditional rehabilitation training methods face significant challenges, such as low repeatability and a shortage of skilled physicians. Exoskeleton robots have been recognized by rehabilitation experts as valuable tools in addressing these issues. However, current auxiliary training devices suffer from limited human-computer interaction capabilities, single-mode training, and basic passive functionalities. To enhance the effectiveness of rehabilitation training, particularly in predicting human movement trajectories, this study presents a brain-like intelligent trajectory prediction model. This model, inspired by bionics, follows the physiological structure and control mechanisms of the human brain to improve human-robot cooperative control in rehabilitation exoskeletons. Utilizing an Echo State Network (ESN), the model establishes a computational framework that mirrors the motor neuron activity of the cerebellum, brainstem, and spinal cord. In conjunction with the Spiking Cerebellar Model Network (SCMN), a brain-like trajectory prediction model was developed that incorporates pulsatile neurons, simulating the transmission and synaptic processes observed in biological neural networks. This approach enhances computational efficiency and physiological interpretability, addressing the limitations of existing neural network models. Experimental results demonstrate that the proposed brain-like control model effectively predicts the movement trajectories of upper limb rehabilitation exoskeletons, offering a novel theoretical and practical framework for bionic control in rehabilitation robotics.</div></div>","PeriodicalId":49592,"journal":{"name":"Robotics and Autonomous Systems","volume":"183 ","pages":"Article 104836"},"PeriodicalIF":4.3,"publicationDate":"2024-10-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142527722","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Vibration suppression of redundantly controlled cable-driven parallel robots","authors":"Xiaotong Zhao, Jingli Du, KunPeng Zhao","doi":"10.1016/j.robot.2024.104838","DOIUrl":"10.1016/j.robot.2024.104838","url":null,"abstract":"<div><div>Cable-driven parallel robots (CDPRs) use flexible cables to connect the end-effector to a fixed base, which is prioritized for large workspace and fast operation speed, but the presence of flexible cables creates a challenge for high-precision control of CDPRs. The mass and elasticity of the cables need to be considered to model the CDPRs in a large workspace more accurately. In this paper, the dynamics of the CDPRs are modeled using the finite element method. In order to more accurately predict the simulation results of the discrete-time model at the actual control frequency, the hierarchical model predictive control (H-MPC) algorithm is proposed with an internal mapping module for mapping control signals and an external prediction module for predictive control. In the control process, we designed a physics-informed neural network (PINN) to predict the state of end-cable elements. Under the same hardware conditions, the H-MPC algorithm effectively reduces the vibration of the end-effector during operation compared to the model predictive control (MPC) algorithm. Our proposed algorithm is validated under various trajectories, and the results show that the H-MPC algorithm can mitigate the vibration condition of the end-effector. We provide new solutions and ideas for the research in high precision control and vibration control of CDPRs. Our H-MPC algorithms are also easier to deploy in industrial controls.</div></div>","PeriodicalId":49592,"journal":{"name":"Robotics and Autonomous Systems","volume":"183 ","pages":"Article 104838"},"PeriodicalIF":4.3,"publicationDate":"2024-10-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142527725","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Management of a fleet of autonomous underwater gliders for area coverage: From simulation to real-life experimentation","authors":"Aurélien Merci , Cédric Anthierens , Nadège Thirion-Moreau , Yann Le Page","doi":"10.1016/j.robot.2024.104825","DOIUrl":"10.1016/j.robot.2024.104825","url":null,"abstract":"<div><div>This article deals with underwater gliders whether there are operated in a fleet or individually. They constitute the most affordable and energy-saving autonomous observation/data acquisition platform, making long-duration ocean exploration missions possible. In this article, theoretical researches are led to solve the path planning problem of multi-point exploration missions of this type of vehicle. We focus on the area coverage type missions <em>i.e.</em> all points of a given area must be visited only once. We suggest a new path planning method for area coverage <em>i.e.</em> the fleet of glider is sized and the optimized glider trajectories are calculated according to selected criteria (mission duration, energy consumption or traveled distance). Our proposed approach combines weighted graph theory with our underwater glider simulator whose main interest is to be capable of integrating time-varying 3D environmental data (4D). Our method is tested in simulation and then in a dynamic real-life context (Mediterranean Sea) on Alseamar’s SeaExplorer autonomous underwater gliders. Finally, a comparison with the expertise of a glider pilot and a more conventional approach, exploiting only the distance between the waypoints in the operation area, confirms the relevance and effectiveness of the suggested method. The experimental mission demonstrates the interest and benefits of the approach and the ease of operational implementation in an industrial context.</div></div>","PeriodicalId":49592,"journal":{"name":"Robotics and Autonomous Systems","volume":"183 ","pages":"Article 104825"},"PeriodicalIF":4.3,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142527721","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}