3D reconstruction of natural scenes with view-adaptive multi-texturing

Abstract

We present a 3D reconstruction and modeling system that operates on a number of input photographs that show a natural scene. Approaches from computer graphics and image processing are combined and performance is shown via experiments. Furthermore, reconstruction quality is analyzed w.r.t. the number and distribution of textures, used for reconstruction. The reconstruction pipeline starts with image acquisition, which consists of a number of photographs of the scene that are sequentially taken at different positions. Since the photographs are not acquired concurrently, they are influenced by different illumination conditions that we mandate to be preserved in the final 3D representation. In the second step, object segmentation is applied and camera calibration provided. This allows the application of shape-from-silhouette approaches, namely a hierarchical voxel approach, where different resolution layers are organized within an octree structure. For applying texture mapping, the voxel model is transformed into a wireframe, which provides smoothing of the object's surface and also reduces the number of surface primitives. Finally, a subset of original images is mapped onto the 3D geometry to provide texture information. Here, view-adaptive multi-texturing is used to preserve natural illumination. Intermediate views are interpolated automatically using adaptive real-time weight calculations for original textures.