OU-CA: A Face–Voice Multimodal Benchmark for Continuous User Authentication

Unlike conventional login authentication systems, which verify a user’s identity only once at the beginning of a session, continuous authentication enhances security by continuously verifying the user throughout the session. In this paper, we introduce a novel multimodal face-voice benchmark dataset for continuous user authentication. It contains data from 1,169 subjects, collected through web-based conversational games conducted between paired participants. During the games, participants’ facial videos and corresponding voice data were automatically recorded. Specifically, five cameras surrounding the display captured faces from different viewing angles to increase pose variations. Moreover, we provide baseline evaluations using state-of-the-art face and speaker recognition methods on our database. We further propose a novel disentanglement cluster network for continuous face authentication, achieving state-of-the-art performance on both our database and the widely used IJB-B and IJB-C datasets. We expect that our database will facilitate further research on continuous authentication and the development of more secure authentication systems.