Video Summarization Based on Fusing Features and Shot Segmentation

Abstract

Video summarization is a technique that creates short summaries from original videos while retaining the main representative information. Traditional video summarization models based on deep learning mostly use frames as the basic processing unit, which cannot handle long videos due to hardware limitations. In this paper, we compress the frame-level features into shot-level features using a feature extractor based on Convolutional Neural Network (CNN), which can improve the training accuracy and reduce computation. At the same time, we propose a feature fusion algorithm based on the capsule network, which combines the RGB features and Light Flow features of the video into the deep features with adaptive weights to enhance the original video features. Experiment results on two benchmark datasets (TVsum and SumMe) validate the effectiveness of our method.