Face Attribute Recognition with Multimodal Fusion

Abstract

Facial attribute recognition is a challenging question, especially the potential high-level semantic features between attributes are a research focus of attribute recognition research. In this paper, a face attribute recognition network is proposed based on multimodal fusion, which is completely data-driven and incorporates image features and advanced semantic features of attributes: 1) We design an image feature extraction network with an underlying shared network that does not restrict the specific form. Meanwhile, a layer-parameter orthogonalization operation is unveiled, which can increase the expressive power of the features. 2) Using graph convolutional networks (GCN), we build a semantic feature extraction network to extract advanced semantic features of face attributes and mine correlations between attributes. 3) To enhance the representation of features, we propose a multimodal fusion module to fuse two complementary representations of image features and semantic features. 4) We construct fused feature classification networks using attribute-independent high-level networks to predict multiple attributes separately. We have conducted extensive experiments to demonstrate the effectiveness of the proposed method and have obtained results that are competitive with state-of-the-art methods.