A Manifold Semantic Canonical Correlation Framework for Effective Feature Fusion

Abstract

In this paper, we present a manifold semantic canonical correlation (MSCC) framework with application to feature fusion. In the proposed framework, a manifold method is first employed to preserve the local structural information of multi-view feature spaces. Afterwards, a semantic canonical correlation algorithm is integrated with the manifold method to accomplish the task of feature fusion. Since the semantic canonical correlation algorithm is capable of measuring the global correlation across multiple variables, both the local structural information and the global correlation are incorporated into the proposed framework, resulting in a new feature representation of high quality. To demonstrate the effectiveness and the generality of the proposed solution, we conduct experiments on audio emotion recognition and object recognition by utilizing classic and deep neural network (DNN) based features, respectively. Experimental results show the superiority of the proposed solution on feature fusion.