Advanced self-contained object removal for realizing real-time Diminished Reality in unconstrained environments

Abstract

While Augmented Reality has always been restricted to adding artificial content to the real environment, Diminished Reality allows for removing real world content. Existing approaches however, either require complex setups or are not applicable in real-time. In this paper we present our approach for removing real-world objects from a live video stream of the user's real environment. Our approach is based on a simple setup and neither requires any pre-processing nor any information on the structure and location of the objects to be removed or on their background. Our approach is based on the identification of the objects to be removed combined with an image completion and synthesis algorithm. The performance of our approach is one to two magnitudes better than that of previous work in the area of image completion, providing real-time object cancellation on standard laptop or tablet computers.