Towards portable image guidance and automatic patient registration using an RGB-D camera and video projector

PURPOSE: Surgical navigation has remained a challenge in emergent procedures with mobile targets. Recent advances in RGB-Depth (RGB-D) camera technology have expanded the opportunities for adopting computer vision solutions in computer-assisted surgery. The purpose of this study was to demonstrate the capacity of an RGB-D camera rigidly fixed to a video projector to perform optical marker tracking, depth-based patient registration, and projected target visualization using open-source software. METHODS: The accuracy of marker tracking and system calibration was tested by projecting points onto the corners of a geometrically patterned marker, which was imaged in several locations and orientations. The depth-based registration and practical projection accuracy of the system was evaluated by targeting specific locations on a simulated patient and measuring the distance of the projected points from their targets. RESULTS: The average Euclidean distance between the marker corners and projected points was 5.34 mm (SD 2.93 mm), and the target projection error in the manikin trial following depth-based registration was 4.01 mm (SD 1.51 mm). On average, the distance of the captured point cloud from the manikin model after registration was 1.83 mm (SD 0.15 mm), while the fiducial registration error associated with registering a commercial tracking system to the manikin was 2.47 mm (SD 0.63 mm). CONCLUSION: This study highlights the potential application of RGB-D cameras calibrated to portable video projectors as inexpensive, rapidly deployable navigation systems for use in a variety of procedures, and demonstrates that their accuracy in performing patient registration is suitable for many bedside interventions.