A Time-of-Flight Stereoscopic Endoscope for Anatomical 3D Reconstruction

This paper presents a novel endoscope design for laparoscopic surgery that has been specifically tailored to provide both a stereoscopic view to the surgeon and a high-accuracy 3D reconstruction for an advanced visualization of the anatomical environment. The former helps the main surgeon in teleoperating a robotic minimally-invasive system (R-MIS) while the latter provides necessary data to upcoming autonomous surgical procedure implementations in a manner akin to the current development of autonomous driving systems. To this aim, we created an initial prototype that incorporates a pair of high-quality, chip-on-tip RGB cameras with a Time-of-Flight (ToF) 3D sensor in a sufficiently compact design to allow its usage in intra-luminal operations. The combination of these sensors provides a reliable 3D model of the anatomical structures at close and far distances within the workspace to effectively overcome the issues presented by current laparoscopy stereo endoscopes, for which the depth estimation is hindered by the reduced baseline distance between the cameras. Moreover, the application to current robotic platforms presents innate mathematical issues when applying hand-eye calibration techniques for localization. We finally developed a calibration procedure that merges both color and depth information. The endoscope design is fully validated through the reconstruction of a planar surface, achieving a depth, latitudinal, and longitudinal orientation precision of 3.3mm, −0.02rad, −0.025rad respectively.