Affective, Cognitive and Behavioural Engagement Detection for Human-robot Interaction in a Bartending Scenario

Abstract

Guaranteeing people’s engagement during an interaction is very important to elicit positive and effective emotions in public service scenarios. A robot should be able to detect its interlocutor’s level and mode of engagement to accordingly modulate its behaviours. However, there is not a generally accepted model to annotate and classify engagement during an interaction. In this work, we consider engagement as a multidimensional construct with three relevant dimensions: affective, cognitive and behavioural. To be automatically evaluated by a robot, such a complex construct requires the selection of the proper interaction features among a large set of possibilities. Moreover, manually collecting and annotating large datasets of real interactions are extremely time-consuming and costly. In this study, we collected the recordings of human-robot interactions in a bartending scenario, and we compared different feature selection and regression models to find the features that characterise a user’s engagement in the interaction, and the model that can efficiently detect them. Results showed that the characterisation of each dimension separately in terms of features and regression obtains better results with respect to a model that directly combines the three dimensions.