Trimming-then-augmentation: Towards robust depth and odometry estimation for endoscopic images

Abstract

Depth and odometry estimation for endoscopic imaging is an essential task for robot assisted endoluminal intervention. Due to the difficulty of obtaining sufficient in vivo ground truth data, unsupervised learning is preferred in practical settings. Existing methods, however, are hampered by imaging artifacts and the paucity of unique anatomical markers, coupled with tissue motion and specular reflections, leading to the poor accuracy and generalizability. In this work, a trimming-then-augmentation framework is proposed. It uses a “mask-then-recover” training strategy to firstly mask out the artifact regions and then reconstruct the depth and pose information based on the global perception of a convolutional network. Subsequently, an augmentation module is used to provide stable correspondence between endoscopic image pairs. A task-specific loss function guides the augmentation module to adaptively establish stable feature pairs for improving the overall accuracy of subsequent 3D structural reconstruction. Detailed validation has been performed with results showing that the proposed method can significantly improve the accuracy of existing state-of-the-art unsupervised methods, demonstrating the effectiveness of the method and its resilience to image artifacts, in addition to its stability when applied to in vivo settings.