Video Summarization Using Feature Vector Clustering

Abstract

With ever-growing utilization of online and offline videos and increasing video content, Video Summarization serves as the best aid for video browsing. It involves domain explicit semantic comprehension of a video and understanding of user expectations. Generally, video summarization systems include extracting video features, analyzing the visual variations and selecting video frames. Over the years, various methodologies have been developed for the same. Different supervised and unsupervised algorithms have been established and these models have been trained on various factors or various rewards. The challenges these methods face stand as a motivation for the approach this paper discusses. Like in many cases, summary frames may be repeated if some scene or concept appears more than once. This paper presents a novel approach based on clustering of video frames based on their feature vectors. The clustering takes into consideration the semantic factor of video frames. Each concept cluster gives a representative frame which then forms the summary set, here concept cluster refers to the independent entity present in a video which can be easily distinguished by another concept or entity. This entity can be a scene of a mountain or different persons. It also aims to increase system performance by removing the redundancy. The system is developed using a CNN for feature extraction and a clustering algorithm that takes into consideration the similarity factor between these vectors. The model is evaluated on the measures Precision and Recall and tested on the VSUMM dataset. The results outperform some of the established methodologies and serve the summarization purpose.