TV program segmentation using multi-modal information fusion

Abstract

A TV program segmentation algorithm is presented by the fusion of the multi-modal information in the large-scale videos. As "Inter-Programs" are generally inserted into the TV videos repeatedly, the macro structures of the videos can be effectively and automatically generated by identifying the video-audio features of the special sequences. The Electronic Program Guide (EPG) is used to organize the structures into the programs. Three sections are included in the algorithm, namely, the video-based non-supervised duplicate sequence detection, the audio-based special clip retrieval and the EPG-based 24-hour program segmentation. The algorithm has been tested in 60-day different-type TV videos. The F-measures of the multi-modal fusion and video-based duplicated sequence detection achieve the rates of over 98% and 96% respectively. These results show that the proposed method is highly efficient and effective for the TV Program segmentation.