Combining vocal and visual cues in an identity verification system using K-NN based classifiers

The contribution of this paper is twofold: (1) to formulate a fusion problem encountered in the design of a multi-modal identity verification system as a particular classification problem, (2) to propose a simple classifier to solve this problem. The multi-modal identity verification system under consideration is built of d modalities in parallel, each one delivering as output a scalar number, called the score, stating how well the claimed identity is verified. A fusion module receiving as input the d scores has to take a binary decision: accept or reject identity. We have solved this fusion problem using a classic k-nearest-neighbor (k-NN) classifier. The most important problem encountered with this simple classifier is the unbalance between the number of reference points in either class. Adapting the classic k-NN classifier using distance weighting and vector quantization principles enables to reduce the influence and the number of impostor reference points respectively. This constitutes the originality of this paper. The performances of these different fusion modules have been evaluated on a multi-modal database, containing both vocal and visual modalities.