Augmented Reality with Hand Gestures Control for Electronic Medical Record

Abstract

Medical records are generally used in medical institution to keep track of patients’ information and their health conditions. Information such as family members, past surgeries and medication taken are recorded and accessible for future references. Medical images of the patient, such as Magnetic Resonance Imaging (MRI) scan and Computed Tomography (CT scan) are available in a 2D printout and are shown to the patient. With the advancement of technology and augmented reality platform, this medical image can be brought to life to enhance the understanding of it. This paper presents the work on developing an application that employs Augmented Reality (AR) technology to enhance users’ experience with Electronic Medical Record (EMR). It allows patients to have an in-depth look at their 3D organs as well as their past medical record. With the integration of LEAP motion device, the 3D organs can be controlled using hand gestures. Three 3D models (kidney, heart and liver) were developed to demonstrate the feasibility of using AR as an alternative solution for storing users’ medical records. Based on the users’ feedback obtained through user acceptance test sessions, the responses are very encouraging.