Advanced approaches to speaker diarization of audio documents

Abstract

Speaker diarization is the process of annotating an audio document with information about the speaker identity of speech segments along with their start and end time. Assuming that audio input consists of speech only or that non-speech segments have been already identified by another method, the task of speaker diarization is to find “who spoke when”. Since there is no prior information about the number of speakers, the main approach is to apply segment clustering. According to the clustering algorithm used, speaker diarization systems can be divided into two groups: 1) based on agglomerative clustering, and 2) based on on-line clustering. Agglomerative clustering is an off-line approach and is used in most of the current systems because it gives accurate results and can be fine tuned by performing several processing passes over the data. This, however, comes at the cost of high computational load which increases exponentially with the number of segments and the requirement of having the whole audio document available in advance. In contrast, on-line clustering based systems have almost constant computational load, work on-line in real time with small latency, but are generally less accurate than off-line systems. As we show in this paper, when using advanced on-line learning methods and original design, on-line systems can make less errors than off-line systems and can even work faster than real time with very low latency.