Scene-Based Video Analytics Studio

Abstract

The amount of the internet video has been growing rapidly in recent years. Efficient video indexing and retrieval, therefore, is becoming an important research and system-design issue. Reliably extracting metadata from video as indexes is one major step toward efficient video management. There are numerous video types, and everyone can define new video types of his own. We believe an open video analysis framework should help when one needs to automatically process various types of videos. More, the nature of video can be so different that we may end up having a dedicated video analysis module for each video type. It is infeasible to design a system to automatically process every type of video. In the paper, we propose a scene-based video analytic studio that comes with (1) an open video analysis framework where the video analysis modules are developed and deployed as plug-ins, (2) an authoring tool where videos can be manually tagged, and (3) an HTML5-based video player the play backs videos using the metadata we generate. In addition, it provides a runtime environment with standard libraries and proprietary rule-based automaton modules to facilitate the plug-in development. At the end, we will show its application to click able (shoppable) videos, which we plan to apply to our e-learning projects.