Towards an FPGA-Based Smart Camera for Virtual Reality Applications

Abstract

Virtual reality (VR) is an experience taking place within simulated and immersive environments. Although in recent years several virtual reality applications such as, virtual reality gaming, medical educational and military training applications have been developed; one important limitation still remains for the tracking sensor. Commercial headsets such as the Oculus Rift or HTC Vive have tracking sensors which project active signals to the user's body and limits the motion understanding. To address this problem, we propose a novel passive sensor (which consist of an FPGA-based smart camera) which computes the optical flow an estimates semantic information about the user movement inside the camera fabric. Then, using these semantic information as feedback for the virtual reality engine; accurate tracking without active signals being projected to the user's body and with the capability to implement several cameras in order to achieve a better movement understanding is possible. Preliminary results are encourageous, demonstrating the possibility of a visual-based tracking approach suitable for virtual reality applications.