Stereoscopic augmented reality system for supervised training on minimal invasive surgery robots

Abstract

Training in the use of robot-assisted surgery systems is necessary before a surgeon is able to perform procedures using these systems because the setup is very different from manual procedures. In addition, surgery robots are highly expensive to both acquire and maintain --- thereby entailing the need for efficient training. When training with the robot, the communication between the trainer and the trainee is limited, since the trainee often cannot see the trainer. To overcome this issue, this paper proposes an Augmented Reality (AR) system where the trainer is controlling two virtual robotic arms. These arms are virtually superimposed on the video feed to the trainee, and can therefore be used to demonstrate and perform various tasks for the trainee. Furthermore, the trainer is presented with a 3D image through a stereoscopic display. Because of the added depth perception, this enables the trainer to better guide and help the trainee. A prototype has been developed using low-cost materials and the system has been evaluated by surgeons at Aalborg University Hospital. User feedback indicated that a 3D display for the trainer is very useful as it enables the trainer to better monitor the procedure, and thereby enhances the training experience. The virtual overlay was also found to work as a good and illustrative approach for enhanced communication. However, the delay of the prototype made it difficult to use for actual training.