Interactive and real-time generation of home video summaries on mobile devices

Abstract

With the proliferation of mobile devices and multimedia, videos have become an indispensable part of life-logs for personal experiences. In this paper, we present a real-time and interactive application for home video summarization on mobile devices. The main challenge of this method is lack of information about the video content in the following frames, which we term "partial-context" in this paper. First of all, real-time segmentation algorithm based on partial-context is applied to decompose the captured video into segments in line with the change in dominant camera motion. Secondly, the main challenge to conventional video summarization is the semantic understanding of the video content. Thus, we leverage the fact that it is easy to get user input on a mobile device and attack this problem through the user interaction. The user preference is learned and modeled by a Gaussian Mixture Model (GMM), which is updated each time when users manually select key frames. Evaluation results demonstrate that our system significantly improves user experience and provides an efficient automatic/semi-automatic video summarization solution for mobile users.