A geometric invariant for visual recognition and 3D reconstruction from two perspective/orthographic views

Abstract

The author addresses the problem of reconstructing 3D space in a projective framework from two views, and the problem of artificially generating novel views of the scene from two given views. He shows that with the correspondences coming from four non-coplanar points in the scene and the corresponding epipoles, one can define and reconstruct (using simple linear methods) a projective invariant, referred to as projective depth, that can be used later to reconstruct the projective or affine structure of the scene, or directly to generate novel views of the scene. The derivation has the advantage that the viewing transformation matrix need not be recovered in the course of computations (i.e., he computes structure without motion).<>