Live Room Merger: A Real-Time Augmented Reality System for Merging Two Room Scenes

Abstract

A real-time augmented reality system is built to replace the background of the user's room (the 'observer room' or 'local room') with a 360-degree live video of another room (the 'remote room'). The user can see the merged room captured by an RGBD camera mounted on the VR headset. A 360-degree image of the remote room is converted into a simple box-like room structure model in real time. The model is loaded into Unity and replaces the background of the observer room, with the result then displayed in a VR head-mount device.