Automatic Endoscopic Navigation for Monocular Depth and Ego-Motion Estimation in Wireless Capsule Endoscopy Through Transformer Network

Abstract

Gastrointestinal (GI) cancers are among the most prevalent globally. Wireless capsule endoscopy (WCE), a minimally invasive technology, offers a promising alternative for diagnosing and treating GI diseases. Accurate depth estimation from WCE but remains challenging due to the complexity of the GI environment and limited datasets. In this paper, we propose an automatic endoscopic navigation system for monocular depth and ego-motion estimation in wireless capsule endoscopy (WCE) through a Transformer-based encoder-decoder network. Minimally invasive surgeries, including gastrointestinal (GI) procedures, face unique challenges such as restricted field of view, illumination variation, and texture sparsity, which complicate depth estimation and pose estimation tasks. Traditional Structure from Motion (SfM) and SLAM methods are often inadequate for GI scenes due to these inherent complexities. To address these issues, we introduce a novel self-supervised neural network framework that integrates a dual-attention mechanism within a modified ResNet. This model simultaneously predicts depth maps and ego-motion from monocular GI images, without requiring ground truth depth data. Our approach enhances feature extraction through spatial and channel-wise attention, allowing the network to capture both local and global contextual information. Furthermore, a multi-scale structural similarity index combined with L1 loss function is employed to improve the accuracy of depth estimation in challenging endoscopic environments. The model leverages a multi-interval frame sampling strategy to simulate diverse ego-motion scenarios, making it robust to low frame rate inputs typically seen in WCE. For ego-motion estimation on the ColonSim dataset, our model achieves an Absolute Trajectory Error (ATE) of 0.09 m at 30 FPS, outperforming the next-best model, SC-SfMLearner, by 44.4%. Additionally, for depth estimation, our model records an Absolute Relative Error (Abs Rel) of 0.33, a Squared Relative Error (Sq Rel) of 0.27, and a Root Mean Square Error (RMSE) of 0.94 on the EndoSLAM dataset.