Monocular 3D Reconstruction of Multiple Non-Rigid Objects by Union of Non-linear Spatial-Temporal Subspaces

Abstract

Non-rigid structure from motion (NRSFM) is an ill-posed problem and has attracted lots of attention in computer vision. Because the NRSfM is ill-posed, a variety of priors, such as the low-rank shape basis, isometric constraints and the low-rank representation, have been employed to make the problem solvable. However, when multiple non-rigid objects are taken into account, the problem becomes more challenging. In such a difficult case, modelling the point trajectories as a union of linear spatial and temporal subspaces via low-rank representation techniques is an effective attempt. Nevertheless, the linear low-rank representation technique is not good at modelling the heavily non-linear variation of the data in both spatial and temporal. In NRSfM, it stands for more complex deformation and shape configuration. In this paper, we propose an approach to solve this problem. Relying on the kernelized low-rank representation technique, we model the point trajectories as a union of nonlinear subspaces and formulate the reconstruction as an optimization problem which can be solved by alternating direction multiplier method (ADMM). Benefitting from the union of nonlinear subspaces model, our method produces the more accurate reconstructions against the state-of-the-art methods on several sequences from CMU MoCap datasets.