Silhouette labeling and tracking in calibrated omnidirectional video sequences

Abstract

In this paper, we present a methodology for labeling and tracking human silhouettes in indoor videos acquired by omnidirectional (fish-eye) cameras. The proposed methodology is based on a fisheye camera model that employs a spherical optical element and central projection, which has been calibrated to allow extraction of 3D geometry clues as described in [11]. The proposed algorithm requires input from a video segmentation algorithm, generating segmented human silhouettes. The history of a person's real position, as well as his appearance in the form of R, G, B color values are utilized in the described methodology. According to initial experimentation, the proposed algorithm is able to track efficiently multiple silhouettes with prolonged partial or full occlusions and it can calculate the trajectory of each silhouette. The algorithm can operate in the presence of imperfect segmentation, with the persons moving in any direction with respect to the camera, thus producing radically different shapes and color appearances.