Real-time image based rendering from uncalibrated images

Abstract

We present a novel real-time image-based rendering system for generating realistic novel views of complex scenes from a set of uncalibrated images. A combination of structure-and-motion and stereo techniques is used to obtain calibrated cameras and dense depth maps for all recorded images. These depth maps are converted into restrictive quadtrees, which allow for adaptive, view-dependent tessellations while storing per-vertex quality. When rendering a novel view, a subset of suitable cameras is selected based upon a ranking criterion. In the spirit of the unstructured lumigraph rendering approach a blending field is evaluated, although the implementation is adapted on several points. We alleviate the need for the creation of a geometric proxy for each novel view while the camera blending field is sampled in a more optimal, non-uniform way and combined with the per-vertex quality to reduce texture artifacts. In order to make real-time visualization possible, all critical steps of the visualization pipeline are programmed in a highly optimized way on commodity graphics hardware using the OpenGL Shading Language. The proposed system can handle complex scenes such as large outdoor scenes as well as small objects with a large number of acquired images.