A Prediction Method for Dimensional Sentiment Analysis of the Movie and TV Drama based on Variable-length Sequence Input

Time continuous emotion prediction problem has always been one of the difficulties in affective video content analysis. The current research mainly designs a temporally continuous long video emotion prediction method by dividing the long video into short video segments of fixed duration. These methods ignore the time dependencies between short video clips and the mood changes in short video clips. Therefore, combined with the related concepts of film and television narrative structure in cinematic language, this paper defines a prediction method for dimensional sentiment analysis of the movie and TV drama based on variable sequence length inputs. First, this paper defines a method for partitioning variable-length audiovisual sequences that set subunits of dimensional emotion prediction as variable sequence-length inputs. Then, a method for extracting and combining audio and visual features of each variable-length audiovisual sequence is proposed. Finally, a prediction network for dimensional emotion is designed based on variable sequence length inputs. This paper focuses on dimensional sentiment prediction and evaluates the proposed method on the extended COGNIMUSE dataset. The method achieves comparable performance to other methods while increasing the prediction speed, with the Mean Square Error (MSE) reduced from 0.13 to 0.11 for arousal and from 0.19 to 0.13 for valence.