Virtual view synthesis from uncalibrated stereo cameras

This paper presents a new approach to the synthesis of a novel view from two images captured by a non-calibrated stereo system. Here view synthesis employs epipolar constraints associated with a two camera configuration. A fundamental matrix is used to obtain features in the synthesized view via reprojection of corresponding features in the source images. Unlike classical methods which are based on inferring three dimensional structure of the scene or use dense correspondence between the source images to produce the new synthesized image, this method requires only sparse correspondence between source image features. Perspective image warping techniques then render the remaining image points via interpolation. The approach permits interactive view synthesis in: immersive telepresence systems, realistic virtual worlds and overlay of objects at different positions on live video of dynamic scenes for augmented reality display systems. Method efficiency is illustrated with examples of synthetic and real scenes.