Feature space mutual information in speech-video sequences

Abstract

We present an approach to directly study mutual relationships between audio and video signals for multimedia applications. The presented approach is mathematically based on information theory and is closely related to information theoretic classification. We show that very simple features of the audio- resp. video-channel can already contain lots of mutual information between both modalities. The mathematical approach is very general though and not restricted to the presented multimedia application.