Joint dense 3D interpretation and multiple motion segmentation of temporal image sequences: a variational framework with active curve evolution and level sets

Abstract

The aim of this study is to introduce a novel method for the simultaneous motion segmentation and dense 3D interpretation of temporal sequences of monocular images. The problem is to recover simultaneously 3D structure, 3D motion, and a motion-based segmentation from the image sequence spatio-temporal variations. Motion in space is considered relative to the viewing system so that both the viewing system and environmental objects are allowed to move. The problem is stated as a 3D motion segmentation problem with simultaneous depth estimation within the regions of segmentation. The Euler-Lagrange equations of minimization of the objective functional lead to curve evolution PDE implemented via level sets.