Multi-modal subspace learning with dropout regularization for cross-modal recognition and retrieval

Abstract

There has been a surge of efforts in cross-modal recognition and retrieval in recent multimedia research. Towards this goal, we investigate a multi-modal subspace learning algorithm together with the Dropout regularizer. Inspired by the regularization for neural networks, we propose to aritificially remove the effect of certain amount of feature bins using the probabilistic approach to prevent the linear subspace learning from over-fitting. The novel regularizer is well integrated into the multi-modal learning algorithm which maximizes the between-class scatter while minimizing the within-class scatter in the projected latent space. The new objective function can be solved efficiently as the generalized eigenvalue problem. Experimental results have shown that superior performance can be obtained in both face-sketch recognition and cross-modal retrieval applications.