Free-Viewpoint Augmented Reality Navigation for Laparoscopic Surgery Based on Virtual Markers And SLAM

Minimally invasive surgery (MIS) has been widely applied in the field of surgery due to its clinical benefits such as low invasiveness and low risk of infection [1]. The surgeon's understanding of the anatomy is a key factor that influences intraoperative complications in MIS [2]. In laparoscopic cholecystectomy (LC), for example, complications due to incomplete exposure of anatomical structures such as the cystic duct and common bile duct account for 92.9% of all LC complications [3]. The key step in LC is to reveal Calot's triangle properly, which is more difficult in cases of heavy inflammation of the gallbladder, encapsulation of surrounding tissues and anatomical variation. This may result in misinterpretation of the anatomy by the surgeon and thus increases the incidence of bile duct injury (BDI) of medical origin [4]. Video see-through augmented reality (VST-AR) navigation [5], a new technology is introduced to address these issues. As a new research hotspot in the field of laparoscopic augmented reality navigation, this technology visualizes the surgical target and key anatomical structures by means of a video transparency overlay to enhance intraoperative perception and improve the safety of the surgery. The core technical issue of VST-AR is the registration, which refers to the matching of a virtual 3D organ model reconstructed preoperatively by CT or MR to the laparoscopic image. The registration is divided into two phases: initial registration and tracking. This study proposes a VST-AR navigation framework based on virtual markers and SLAM. The concept of virtual markers is introduced to achieve rapid non- invasive registration of the virtual scene to the real scene. Based on the patient's body surface features, the proposed navigation framework achieves dynamic tracking of the laparoscope pose and free viewpoint transparency of hidden anatomical structures. In the case of laparoscopic cholecystectomy, for example, the key anatomical structures, such as the gallbladder and the cystic duct, are overlaid onto their corresponding positions in the liver, so as to support the surgeon with the refinement of the operation.