NRSfM using local rigidity

Abstract

In this paper we show that typical nonrigid structure can often be approximated well as locally rigid sub-structures in time and space. Specifically, we assume that: 1) the structure can be approximated as rigid in a short local time window and 2) some point- pairs stay relatively rigid in space, maintaining a fixed distance between them during the sequence. First, we use the triangulation constraints in rigid SfM over a sliding time window to get an initial estimate of the nonrigid 3D structure. Then we automatically identify relatively rigid point-pairs in this structure, and use their length-constancy simultaneously with triangulation constraints to refine the structure estimate. Local factorization inherently handles small camera motion, short sequences and significant natural occlusions gracefully, performing better than nonrigid factorization methods. We show more stable and accurate results as compared to the state-of-the art on even short sequences starting from 15 frames only, containing camera rotations as small as 2° and up to 50% contiguous missing data.