Biometric characteristics of hand gestures through joint decomposition of cross-subject and cross-session biases

Abstract

Hand gestures are natural in human–computer interfaces for their use in gesture-based control, as well as personalized devices with privacy preservation and device personalization. However, some hand gestures bear biometric traits, whereas some others generalize well across subjects. The ‘entanglement’ between these concepts is in gestures with tight intra-subject clusters (in a feature space), and low inter-subject separation. This puts forth a key requirement: quantifying the biometric ‘goodness’ of a gesture so as to segregate between the gestures prudent for authentication and recognition-level generalization. We propose a biometric characterization framework that estimates how uniquely and with how much variability, any unseen/seen subject would perform a particular gesture. We leverage a theoretical understanding of personality and stability biases present in cross-user (CU) and cross-session (CS) gesture recognition experiments. Structuring upon the biases that influence performance in these experiments, we derive mathematical relationships that quantify the biometric goodness of a gesture. In order to disentangle bias for identity understanding, we introduce two strategies based upon ‘bias mitigation’ and ‘bias intensification’. We empirically validate the proposed framework through experiments on three datasets. The proposed framework is generic and does not require user identity information. Additionally, this can operate over any existing hand-gesture recognition pipelines.