Hierarchical organization for medical video summarization using latent visual and semantic analysis

Abstract

The large increase of medical video archives demands effective organization for efficient retrieval and browsing. In this paper, we present a novel framework of video summarization based on the latent low-level visual and high-level semantic analysis. First, we investigate the concept hierarchy of the medical videos. Secondly, we collect textual semantic information around videos with an image-by-word matrix analysis process. Then, keyframe based video summarization is constructed by affinity propagation clustering and video content mining. Finally, we organize the extracted shots with hierarchical pattern, and tag keyframes with semantic labels. Our approach takes advantage of both visual content and textual information for video abstract. Preliminary experiment results show that our proposed approach could perform well.