Augmented Tangibility Surgical Navigation Using Spatial Interactive 3-D Hologram zSpace with OsiriX and Bio-Texture 3-D Organ Modeling

Abstract

We developed a new spatial navigation system for medical informatics by interactive superimposing 3-D hologram and 3D printing technology. Interactive stereo display was used for the existence of an interaction between the users and stereo images of the patient's anatomy depicted on the display in the form of tracking the user's head and hand/arm position. Sensing the user's head position created motion parallax information, an immersive depth cue that can be added to the binocular parallax already present in the display. We also developed new technology of bio-texture modeling by multi-material 3D printing to form 3D organ textures and structures. Based on patient-specific MDCT data sets, regions of interest were segmented using DICOM viewer OsiriX application. After generating 3D surface models of the organ and STL file out of the patient's 3D data, the inkjet 3D printer created a 3D multi-material organ replica. Sensing the user's hand or arm position using motion sensor attached the patient's life size 3-D printed organ model, allowed the user to manipulate the spatial attributes of the virtual and real printed organs, which can enhance spatial reasoning and augmented tangibility. These tangible organ replication provide better anatomical reference tool as a tailormade simulation and navigation, and contribute to medical safety and accuracy, less-invasiveness and improvement of the medical education for students and trainees.