A comprehensive evaluation of audio-visual behavior in various modes of interviews in the wild

Behavioral interviews are a crucial part of any candidate selection process. Automatic assessment of communication skill in such interviews is a key issue in Social Computing. With the evolution of technology, interactive online platforms have enabled conducting interviews online either in the form of video interviews (asynchronous interviews without an interviewer) or online face-to-face interviews without the need of collocation of both interviewer and interviewee. In this work, we automatically assess the communication skill score of job applicants giving real interface-based interviews and online face-to-face interviews with a model trained on the video interviews collected in a controlled environment. Web-based platforms are developed to collect interviews asynchronously and face-to-face. The web-based platform for face-to-face is Skype-like that allows recording and streaming the audio-visual content of both interviewer and interviewee. We address this problem by capturing the audio-visual behavior of the participant in various modes. To this end, we have collected 251 video interviews (asynchronous) in controlled environment for training, 38 real video interviews in the wild and 45 face-to-face interviews in the wild for evaluation. This work is novel in evaluating the interviews in the wild, with the model trained in a complementary environment. Results obtained indicate that models trained on data collected in controlled environment generalize for the interviews conducted in real uncontrolled environment, with diversity in people and audio recording hardware.