Virtual reality navigation system of nasal endoscopy with real surface texture information

The 3D information of medical images can effectively assist surgeons in diagnosis and surgical planning. In this paper, the methods of monocular endoscopic reconstruction of the inner surface of the nasal cavity and nasal endoscopic surgery path planning are proposed, and a virtual nasal endoscopic navigation system with real surface texture information is realized, which provides an intuitive reference for surgeons' diagnosis and surgery. First, by extracting the ASIFT feature points of the nasal endoscopic images, the structure from motion and multi-view stereo reconstruction are performed to construct a point cloud model. The point cloud model is used to reconstruct the triangle surface and texture mapping to obtain a 3D model of the nasal cavity with real surface texture information. Secondly, in view of the lack of scale information of the obtained monocular nasal reconstruction model, a variable-scale registration method between the obtained monocular nasal reconstruction model and the CT image reconstruction model is proposed. The FPFH algorithm is used to achieve feature description and matching, and coarse registration is achieved; the variable scale iterative closest point algorithm is used to achieve fine registration. Then, considering the cylinder shape of the nasal endoscope, a collision-free 3D planning map is constructed, and the RRT algorithm is used to realize the collision-free path planning under the complex nasal cavity space constraints. Finally, a virtual nasal endoscopy navigation system with real surface texture information is constructed, and virtual nasal endoscopy experiments are carried out to verify the effectiveness of the navigation system.