A Real-time Background Replacement System Based on Estimated Depth for AR Applications

Recent technological advances in Virtual Reality (VR) and Augmented Reality (AR) enable users to experience a high-quality virtual world. The AR technology is attracting attention in various fields and is also used in the entertainment field such as museums. However, the existing AR technology generally requires specialized sensors such as Light Detection And Ranging (LiDAR) sensors and feature points, which require cost in terms of time and money. The authors have proposed a real-time background removal method and an AR system based on the estimated depth of the captured image to provide a virtual space experience using mobile devices such as smartphones. This paper describes an AR virtual space system that dynamically changes the replaced background based on motion information transmitted from the user's device.