PACMR: Progressive Adaptive Crossmodal Reinforcement for Multimodal Apparent Personality Traits Analysis

Abstract

Multimodal apparent personality traits analysis is a challenging issue due to the asynchrony among modalities. To address this issue, this paper proposes a Progressive Adaptive Crossmodal Reinforcement (PACMR) approach for multimodal apparent personality traits analysis. PACMR adopts a progressive reinforcement strategy to provide a multi-level information exchange among different modalities for crossmodal interactions, resulting in reinforcing the source and target modalities simultaneously. Specifically, PACMR introduces an Adaptive Modality Reinforcement Unit (AMRU) to adaptively adjust the weights of self-attention and crossmodal attention for capturing reliable contextual dependencies of multimodal sequence data. Experiment results on the public First Impressions dataset demonstrate the effectiveness of the proposed method.