ASR-NeSurf: Alleviating structural redundancy in neural surface reconstruction for deformable endoscopic tissues by validity probability

Abstract

Accurate reconstruction of dynamic mesh models of human deformable soft tissues in surgical scenarios is critical for a variety of clinical applications. However, due to the challenges of limited sparse views, weak image texture information, uneven illumination intensity and large lens distortion in endoscopic video, the traditional 3D reconstruction methods based on depth estimation and SLAM fail to accurate surface reconstruction. Existing neural radiance field methods, such as Endosurf, have been developed for this problem, while these methods still suffer from inaccurate generation of mesh models with structural redundancy due to limited sparse views. In this paper, we propose a novel neural surface reconstruction method for deformable soft tissues from endoscopic videos, named ASR-NeSurf. Specifically, our approach modifies the volume rendering process by introducing the neural validity probability field to predict the probability of redundant structures. Further, unbiased validity probability volume rendering is employed to generate high-quality geometry and appearance. Experiments on three public datasets with variation of sparse-view and different degrees of deformation demonstrate that ASR-NeSurf significantly outperforms the state-of-the-art neural-field-based method, particularly in reconstructing high-fidelity mesh models.