Salient Face Prediction without Bells and Whistles

Abstract

Salient face prediction in multiple-face videos is a fundamental task in machine vision. It finds usage in various applications like video editing and human-machine interactions. The field has seen significant progress in recent years, backed by large datasets comprising specifically of multi-face videos. As the first contribution, we present promise in a visual-only baseline, achieving state-of-the-art results for salient face prediction. Our work motivates reconsideration towards sophisticated multimodal, multi-stream architectures. We further show that a simple upstream task like active speaker detection can give a reasonable baseline and match prior tailored models for detecting salient faces. Moreover, we bring to light the inconsistencies in evaluation strategies, highlighting a need for standardization. We propose using a ranking-based evaluation for the task. Overall, our work motivates a fundamental course correction before re-initiating the search for novel architectures and frameworks.