Multi-Modal Aesthetic System for Person Identification

Abstract

Aesthetic preference can be described as one's taste or fondness for a particular subject. This information has become ubiquitous as online communities and social media have grown increasingly integrated with daily life. The domain of social-behavioral biometrics analyzes the interactions, relations, and communications of individuals rather than traditional physical traits. Recent research has demonstrated that a person's visual aesthetic preferences possess discriminatory value for person identification. This paper introduces the first audio and visual multi-modal aesthetic identification system that utilizes both user-liked images and songs for an accurate identity prediction with score-level fusion. The developed multimodal system achieves an accuracy of 99.4% on the proprietary audio-visual dataset, outperforming unimodal systems.