User-oriented affective video content analysis

Abstract

The article looks into a new direction in multimedia content analysis: the extraction and modeling of the affective content of an arbitrary video. The affective content is viewed as the amount of feeling/emotion contained in and mediated by a video toward a viewer. The ability to automatically extract video content of this nature will lead to a high level of personalization in broadcast delivery to private users, as well as considerably broadening the possibilities of efficiently handling and presenting large amounts of audio-visual data stored in emerging video databases. The technique we have developed uses the so-called "dimensional approach to affect" concept underlined by psychophysiology studies. Our computational method sets to represent the affective content as feature points in the so-called 2D emotion space. We manage to obtain time curves that represent the two affect dimensions (arousal and valence) for a video, considered respectively, from low-level video characteristics. Combining the two time curves results in the so-called affect curve that is regarded as a reliable representation of transitions from one feeling to another along a video, as perceived by a viewer. We illustrate the success of our technique on excerpts taken from an action movie and a typical soccer game, respectively.