Face clustering in videos: GMM-based hierarchical clustering using Spatio-Temporal data

Abstract

In recent years, an increase in multimedia data generation and efficient forms of storage have given rise to needs like quick browsing, efficient summarization and techniques for information retrieval. Face Clustering, together with other technologies such as speech recognition, can effectively solve these problems. Applications such as video indexing, major cast detection and video summarization greatly benefit from the development of accurate face clustering algorithms. Since videos represent a temporally ordered collection of faces, it is only natural to use the knowledge of the temporal ordering of these faces, in conjunction with the spatial features extracted from them, to obtain optimal clusterings. This paper is aimed at developing a novel clustering algorithm, by modifying the highly successful hierarchical agglomerative clustering (HAC) process, so that it includes an effective initialization mechanism, via an initial temporal clustering and Gaussian Mixture Model based cluster splitting, and introduces a temporal aspect during cluster combination, in addition to the spatial distances. Experiments show that it significantly outperforms HAC while being equally flexible.