A Simultaneous Polyp and Lumen Detection Framework Toward Autonomous Robotic Colonoscopy

Colorectal Cancer is one of the deadliest diseases with a high incidence and mortality worldwide. Robotic colonoscopes have been extensively developed to provide alternative solutions for colon screening. Nevertheless, most robotic colonoscopes remain a low autonomy level, which leads to non-intuitive manipulation and limits their clinical translation. This paper proposes a deep learning-based framework for simultaneous polyp and lumen detection, which aims to automates robotic colonoscopes to achieve intelligent and autonomous manipulations in two aspects of navigation and diagnosis. Two fully annotated datasets, including a real colon dataset, with 40186 images, and a colon phantom dataset, are developed to facilitate polyp and lumen detection in both clinical and laboratory environments. Benchmarking of various object detection models achieve an average precision of 0.827, and an average recall of 0.866. Experimental validation is conducted in both a commercialized colon phantom and an ex-vivo porcine colon using an electromagnetically actuated soft-tethered colonoscope as a case study, with the results indicate that the colonoscope can successfully perform autonomous navigation and automatic polyp detection under the proposed unified framework. This work promotes clinical applications of robotic colonoscopy by enhancing its autonomy level with artificial intelligence techniques.