Reconstruction of Dense Correspondences

Abstract

This chapter concentrates on dense image correspondence estimation with a special focus on stereo. Images are the basic input for a vast majority of algorithms dealing with the reconstruction of the real world. To analyze a scene from a collection of images it becomes inevitable to put these images into correspondence. These correspondences then form the basis for many subsequent analyses, including camera calibration, stereo and 3D reconstruction, motion information, scene flow and others. While some of these tasks like camera calibration require only sparse correspondences between the images, Chapter 7, others require per-pixel correspondence, also known as dense correspondence estimation. Humans are extremely good at solving the correspondence problem which most of them do all the time during depth perception. Basically, the eyes serve as two cameras, slightly displaced, with respect to each other, that capture the surrounding from two different viewpoints. When focusing on an object at a certain distance one has already computed an estimate of the distance in the brain and therefore of the object’s position in space. It turns out the same problem is quite difficult for a computer and has been researched for several decades now. The difficulty in correspondence estimation is caused by several factors: images are often corrupted by sensor noise, e.g. when recorded in a poorly lit environment Section 1.1; the captured scene signal is discretized and represented by some finite image resolution; not every pixel actually has a correspondencing partner in the other views as it might be occluded; and ambiguities due to the absence of texture are difficult to solve. If one can solve the dense correspondence problem a variety of different applications becomes possible especially in the field of computer vision. Robot navigation and autonomous cars require depth perception to avoid obstacles [Giachetti et al. 98, Kastrinaki et al. 03]. Quality assurance in industrial applications is often based on stereo algorithms to detect cracks