A Multimodal Meeting Browser that Implements an Important Utterance Detection Model based on Multimodal Information

Abstract

This paper proposes a multimodal meeting browser with a CNN model that estimates important utterances based on co-occurrence of verbal and nonverbal behaviors in multi-party conversations. The proposed browser was designed to visualize important utterances and to make it easier to observe the nonverbal behaviors of the conversation participants. A user study was conducted to examine whether the proposed browser supports the user to correctly understand the content of the discussion. By comparing a text-based browser and a simple video player, it was found that the proposed browser was more efficient than the video player and allowed the user to obtain a more accurate understanding of the discussion than the text-based browser.