Shape tensors for efficient and learnable indexing

Abstract

Multi-point geometry: The geometry of 1 point in N images under perspective projection has been thoroughly investigated, identifying bilinear, trilinear, and quadrilinear relations between the projections of 1 point in 2, 3 and 4 frames respectively. The dual problem-the geometry of N points in 1 image-has been studied mostly in the context of object recognition, often assuming weak perspective or affine projection. We provide here a complete description of this problem. We employ a formalism in which multiframe and multi-point geometries appear in symmetry. Points and projections are interchangeable. We then derive bilinear equations for 6 points (dual to 3-frame geometry), trilinear equations for 7 points (dual to 3-frame geometry), and quadrilinear equations for 8 points (dual to the epipolar geometry). We show that the quadrilinear equations are dependent on the the bilinear and trilinear equations, and we show that adding more points will not generate any new equation. The new equations are used to design new algorithms for the reconstruction of shape from many frames, and for learning invariant relations for indexing into a database.