Nonexclusive audio segmentation and indexing as a pre-processor for audio information mining

Abstract

Much content related information can be extracted from recorded soundtracks, such as those of multimedia files. The soundtracks might be heuristically classified into three categories namely speech, music and ambient or event sounds. Research in the past focused on algorithms to classify audio clips in an exclusive manner. However, soundtracks from media content are often presented as overlapped mixtures of all these three types of sounds. Nonexclusive segmentation and indexing are therefore essential pre-processors for effective audio information mining and metadata generation. This paper emphasizes the importance of nonexclusive indexing and segmentation methods, identifies the challenges and proposes a universal architecture for nonexclusive segmentation and indexing as a pre-processor for audio information mining, metadata extraction and scene analysis. Related feature selection, pattern recognition and signal processing algorithms are presented and testing results discussed.