A Multimodal Biometric Recognition System based on Finger Snapping and Information Fusion

Biometrics, physiological and/or behavioral traits used to distinguish different individuals, have now been widely used in authentication sites because they have shown advantage over passwords and token-based verification methods. However, biometric systems that use static and unimodal biometric information are susceptible to spoofing, which poses a security threat. Finger snapping, which is dynamic and can be collected multimodally, has the potential to overcome the current challenges, but a good information fusion method, model, as well as dataset is needed. In this study, three main questions are proposed and studied: (1) Is snapping a viable biometric? (2) Is feature-level information fusion of snapping practical? (3) How do traditional ML models and DL models differ in their performances on a snapping dataset? To answer the three questions, a preparative dataset consisting of kinetic information of snaps from 33 participants, as well as a main dataset consisting of kinetic and acoustic information of snaps from 50 participants have been built using a self-constructed data collection system. The preparative dataset is used to decide which subset of sensors provides the most useful information, and the main dataset is used to answer the above three questions. After testing, it is discovered that (1) snapping is a highly feasible biometric, (2) the kinetic and acoustic data can be fused at the feature level, and (3) The traditional ML model tested has greater potential over DL models due to the higher AUROC metric (0.967 ± 0.034 versus 0.943 ± 0.032), but has a lower weighted accuracy score with the default decision threshold than the DL model (87.5% ± 4.04% versus 89.9% ± 2.74%).