Dog emotion recognition from images in the wild: DEBIw dataset and first results

Abstract

Emotions play a transcendental role in the behavior of dogs. Their emotional response to certain stimuli and situations can be decisive in their actions. Automatic recognition of dog emotions gives ethologists and trainers the ability to monitor dog reactions and help profile them more accurately. On the other hand, providing dog-computer interaction systems with the ability to know the emotional state of the canine user can help improve the objectives of the interaction. This work presents the creation of a new database of images of dogs representing emotions such as (aggression, anxiety, contentment and fear) and a method to classify them automatically. This database consists of 15,599 images downloaded from the Internet directly. Each image was manually labeled by multiple taggers using a web-based interface. Using a variety of state-of-the-art image classification approaches, including an AutoML solution that performed the best (0.67 of the macro average f1 measure), the plausibility of automatic dog emotion recognition was assessed. This is remarkable, given that the input to the classification model is an image downloaded from the Internet without applying any cleaning, segmentation, characterization, or key point marking technique. The proposed methodology can serve as a non-invasive, easy-to-instrument, and easy-to-retrain means for the implementation of dog emotion-aware computational systems. More importantly, the created dataset will allow the in-depth study of this relevant problem. * Both authors contributed equally to this research.