An explainable deep learning model for automated classification and localization of microrobots by functionality using ultrasound images

Abstract

The rapid advancements of untethered microrobots offer exciting opportunities in fields such as targeted drug delivery and minimally invasive surgical procedures. However, several challenges remain, especially in achieving precise localization and classification of microrobots within living organisms using ultrasound (US) imaging. Current US-based detection algorithms often suffer from inaccurate visual feedback, causing positioning errors. This paper presents a novel explainable deep learning model for the localization and classification of eight different types of microrobots using US images. We introduce the Attention-Fused Bottleneck Module (AFBM), which enhances feature extraction and improves the performance of microrobot classification and localization tasks. Our model consistently outperforms baseline models such as YOLOR, YOLOv5-C3HB, YOLOv5-TBH, YOLOv5 m, and YOLOv7. The proposed model achieved mean Average Precision (mAP) of 0.861 and 0.909 at an IoU threshold of 0.95 which is 2% and 1.5% higher than the YOLOv5 m model in training and testing, respectively. Multi-thresh IoU analysis was performed at IoU thresholds of 0.6, 0.75, and 0.95, and demonstrated that the microrobot localization accuracy of our model is superior. A robustness analysis was performed based on high and low frequencies, gain, and speckle in our test data set, and our model demonstrated higher overall accuracy. UsingScore-CAM in our framework enhances interpretability, allowing for transparent insights into the model’s decision-making process. Our work signifies a notable advancement in microrobot classification and detection, with potential applications in real-world scenarios using the newly available USMicroMagset dataset for benchmarking.