Automatic canine emotion recognition through multimodal approach

Abstract

This study introduces a comprehensive multimodal approach for analyzing and classifying emotions in dogs, combining visual, inertial, and physiological data to improve emotion recognition performance. The research focuses on the dimensions of valence and arousal to categorize dog emotions into four quadrants: playing, frustration, abandonment, and petting. A custom-developed device (PATITA) was used for synchronized data collection to which a feature extraction process based on windowing was done. Dimensionality reduction and feature selection techniques were applied to identified most relevant features across data types. Then, several unimodal and multimodal classification models, including Naïve Bayes, SVM, ExtraTrees, and kNN, were trained and evaluated. Experimental results demonstrated the superiority of the multimodal approach, with ExtraTrees classifier consistently yielding the best results (F1-score = 0.96), using the reduced feature set. In conclusion, this work presents a robust multimodal framework for canine emotion recognition, providing a foundation for future studies to refine techniques and overcome current limitations, particularly through more sophisticated models and expanded data collection.