Comparison of multidimensional decoding of affect from audio, video and audiovideo recordings.

Abstract

Video tapes were made of six encoders portraying a warm and a cold experimenter. 60 decoders who were either exposed to the video, audio or both the video and audio portions of the stimuli rated the presentations on the Nowlis Affect Scales. The degree of consistency among judgments and the degree to which the communication channels conveyed and facilitated the differentiation of the warm and cold presentations of the experimenters were calculated. Decoders were least consistent in decoding presentations conveyed in the audio channel; combined audio-video judgments were no more consistent than judgments based on the video channel alone. While there were significant (p < .01) multivariate and univariate experimenter behavior x channel interactions on 7 of the 11 iNowlis scales, no consistent pattern across encoders was evidenced. Rather, some encoders showed variations in feelings principally through visually mediated stimuli, others through the tone of the voice. The results are discussed in the context of quantitative versus qualitative differences among the communication channels.