Enhanced video handling based on audio analysis

Abstract

Soundtracks of videos contain a rich source of content-based information. In this paper, we propose an audio-based approach to video indexing and handling. Audio data is analysed by means of frequency analysis, and music and voice are independently detected even if they occur together. The method is implemented on a system called Video in Time as an example of creating reasonable condensed versions of dramas or movies by excerpting meaningful video segments. Users can select the desired replaying time from several different levels, depending on how much time can be afforded for viewing. Detection rates for music and voice are evaluated and experiences with the system are mentioned.