Frequency-based environment matting

Abstract

Environment matting is a technique to extract the environment matte which is used to describe how an object reflects and refracts the environment light. In this paper, we propose a novel environment matting method to obtain the environment matte of a real scene. Previous methods use different backdrops as the calibration patterns and search for the environment matte in the spatial domain. In our method, however, a series of background images displayed on a screen sequentially in time are interpreted as signals. The frequency similarity of these signals is used as the searching criterion. The frequencies of these signals are not changed when they interact with the foreground objects and thus can be used to extract the environment matte. While using correspondence in the spatial domain in existing approaches is prone to error, using frequency correspondence is not. Thus, our approach is robust to noise and can easily deal with some of the complex light transport phenomena which cannot be easily handled using current methods. The experimental results are very encouraging.