An engine of virtual reality mixing environment based on real-time modeling and interaction

Abstract

Mixed reality (MR) is a further research hotspot with the development of virtual reality (VR). Seamless fusion and real-time interaction between virtual environment and real users or objects are two most important problems urgent to be solved for a MR system. In this paper an engine of virtual reality mixing environment E-VRME is proposed based on real-time modeling and interaction and three applications are demonstrated. The engine runs on our self-built hardware platform DreamWorld and has three modules. Graphics module is designed for VE rendering, modeling module for 3D reconstruction and V-R interaction module for collision detection and feedback. An octree-based visual hull modeling method is used for constructing the mesh models of all the users from their multi-view images. Then, marching cubes algorithm is applied for triangulation and texture mapping is carried out. E-VRME integrates virtual and real information into a seamless unit and it supports any kinds of real-time MR applications. Experimental results and three different application systems proved its usefulness and effectiveness.