Integrating a Real-Time Captured Object into Mixed Reality

Abstract

Web-based mixed reality (MR) that combines remote real objects with a virtual environment to form a realistic 3D MR space has received much interest in cyberworlds. This paper introduces a novel method of integrating a live video object that is taken by multiple fixed cameras in a remote natural environment into a MR space. Using the captured live video frames, the view of the remote object corresponding to the user's current viewpoint in the MR space is generated in realtime based on the proposed fast view morphing algorithm, which ensures a natural image transition without reconstructing the 3D model of the object. During the user's free navigation in the MR space, the real-time generated view of the object is presented in that space, which results in a convincing impression that the remote object exists in the local realistic 3D MR space. Experimental results in both a desktop MR system and a CAVE-like MR system have demonstrated the validity of the proposed approach