Edge-and-motion-based semantic video object extraction and application to scene composition

Abstract

The automatic segmentation of natural video for content-based applications is considered. In particular, a segmentation algorithm that uses both motion and edge information is proposed. It applies the edge-linking approach to determine accurately the boundaries of objects and bidirectional motion estimation to track robustly the motion of objects and changes in shape. The design of the algorithm considers computational complexity. Since composition using segmented objects is further considered. A method of enlarging or shrinking objects for such a purpose is described and evaluated. The method is amenable to spatial-scalable coding.