HyV-Summ: Social media video summarization on custom dataset using hybrid techniques

Abstract

The proliferation of social networking platforms such as YouTube, Facebook, Instagram, and X has led to an exponential growth in multimedia content, with billions of videos uploaded every hour. Efficient management of such vast amount of data necessitates advanced summarization techniques in order to eliminate irrelevant and redundant information. A summarized video, containing the most distinct frames or key frames, provides a concise representation of the original content. Existing deep learning and non-deep learning techniques for video summarization have certain limitations. Deep learning methods are complex and resource-intensive, while non-deep learning algorithms often fail to extract informative features from vast social media videos. This paper addresses the issue by proposing a novel hybrid technique, named Hybrid Video Summarization (HyV-Summ), which integrates deep and non-deep learning techniques to leverage their respective strengths by focusing only on social media content. We developed a custom dataset, SocialSum to train our proposed model HyV-Summ, since existing benchmark datasets like TVSum and SumMe contain diverse types of content not specific to social media videos. We provide a comparative analysis of existing techniques and datasets with our proposed techniques and dataset. The results demonstrate that HyV-Summ outperforms existing techniques, such as Long Short Term Memory (LSTM)-based and Generative Adversarial Network (GAN)-based summarization by achieving higher F1-scores while applied on both the SocialSum dataset and available datasets.