Virtual-Physical Interaction System

Abstract

We present a novel human-computer interaction (HCI) system called Virtual-Physical Interaction System(VPIS): On the one hand, offering a realistic product experience process in a Virtual Reality(VR) environment. On the other hand, the corresponding process is controlled by manipulating a tangibly product model in a physical world. Thus, the VPIS with a Virtual-Physical combination enhances user experience in HCI field, which can apply in several application scenarios (Aided Design). However, the implementation of this system is a challenging task that subsumes several fundamental problems as sub-tasks: target detection and tracking, gesture detection and recognition. In order to achieve the expected goal of VPIS, the system is based on the physical hardware (HoloLens, Leap Motion and 3D printer) and the existing computer vision(CV) algorithms.