Evaluating the Causal Effect of Multimedia and Affective Temperament in Felt Emotion and Liking

Abstract

In this paper, we propose an approach to evaluate the causal effect of videos on subjects who watched movies from the LIRIS-ACCEDE dataset and from whom the following information was collected: affective temperaments, gender, and electroencephalography (EEG) signals. The affective temperament was obtained by analyzing their answers to the AFECT questionnaire. Evidence was collected from specialized literature to design a Structural Causal Model to be subjected to Do-Calculus Causal Inference. Video concepts were extracted to characterize the major video content after k-means clustering. Information from 15 volunteers was analyzed and the effects of video content, affective temperament, and gender on emotion response and liking were computed. Higher Order Crossings (HOC) were extracted from EEG signals and the features were clustered and used as intermediate evidence of affective influence. This research provides answers for the following questions about the specific watched videos: (i) How does gender affect the felt emotion and liking? (ii) How does the affective temperament of a person affect felt emotion and liking? and (iii) How does the content of a video affect felt emotion and liking? The main contribution of this paper is in the proposed methodology which can be applied to any similar dataset to investigate the causal relationships of video content and affective temperament on the emotion of the audience.