Camera pose in SfT and NRSfM under isometric and weaker deformation models

Abstract

Camera pose is a very natural concept in 3D vision in the rigid setting. It is however much more difficult to work with in deformable settings. Consequently, numerous deformable reconstruction methods simply ignore camera pose. We analyse the concept of pose in deformable settings and prove that it is unconstrained with the existing formulations, properly justifying the existing pose-less methods reconstructing structure only. We explain this result intuitively by the impossibility to define an intrinsic coordinate frame to a general deforming object. The proposed analysis uses the isometric deformation model and extends to the weaker models including conformality and equiareality We propose a novel prior to rescue camera pose estimation in deformable settings, which attributes the deforming object’s dominant rigid-body motion to the camera. We show that adding this prior to any existing formulation fully constrains camera pose and leads to elegant two-step solution methods, involving deformable structure reconstruction using a base method in the first step, and absolute orientation or Procrustes analysis in the second step. We derive the proposed approach for the template-based and template-less settings, respectively implemented using Shape-from-Template (SfT) and Non-Rigid Structure-from-Motion (NRSfM) as base methods and validate them experimentally, showing that the computed pose is qualitatively and quantitatively plausible.