Practical, real-time studio matting using dual imagers

Abstract

This paper presents a practical system for capturing high-resolution video mattes using cameras that contain two imagers on one optical axis. The dual imagers capture registered frames that differ only by defocus or polarization at pixels corresponding to special background 'gray-screens.' This system eliminates color spill and other drawbacks of blue-screen matting while preserving many of its desirable properties (e.g., unassisted, real-time, natural illumination) over more recent methods, and achieving higher precision output for Bayer-filter digital cameras. Because two imagers capture more information than one, we are able to automatically process scenes that would require manual retouching with blue-screen matting. The dual-imager system successfully pulls mattes for scenes containing thin hair, liquids, glass, and reflective objects; mirror reflections produce incorrect results. We show result comparisons for these scenes against blue-screen matting and describe materials and patterns for building a capture system.