Global region reidentification for camera relocalization in video-based surgical navigation.

Abstract

Purpose: Vision-based navigation systems rely on the registered camera poses in the CT space to guide surgeons. However, while it is possible to provide an approximate initialization, this registration becomes outdated as the endoscopic camera leaves and reenters the anatomy. Endoscopic camera relocalization is the process of determining the position of an endoscope relative to an anatomical reference after reinsertion. However, accurately reidentifying the global surgical scene and estimating camera pose have proven challenging due to the varying appearance of endoscopic sequences.

Methods: We present a training-free approach to accurately reidentify the region of interest (ROI) and estimate the camera position of a query image after reinsertion. This method utilizes previously observed images with known poses and a CT scan. By combining advanced foundation models with classical techniques, we globally reidentify a prior image of the ROI, which is then used for image-based feature matching and pose recovery via the Perspective-n-Point algorithm.

Results: We conducted experiments on eight sequences from three cadaver studies. Our results show that our method accurately reidentifies when the endoscope reaches the ROI and identifies suitable image pairs for PnP-based pose estimation. It achieves an average translation error of 1.74 mm and a rotational error of 0.09 radians, making it suitable for reinitialization in image-based navigation without human intervention.

Conclusion: Our work presents a training-free approach for detecting when the endoscope reenters the ROI and estimating the camera's pose after reinsertions. The approach demonstrates promising results contributing toward enabling pose reinitialization for vision-based surgical applications.