Adaptive Margin Ranking for Supervised Cross-modal Retrieval

Cross-modal retrieval is to achieve flexible query between different modalities. Many approaches solve the problem by learning a common feature space under to separate the multimodal instances from different categories. But it is challenge to design an effective projecting function. In this paper, we propose a novel cross-modal retrieval method, called Adaptive Margin Ranking for Supervised Cross-modal Retrieval (AMRS). In the solution, we design a neural network as the nonlinear mapping function. To maximize the discrimination of multimodal feature in common representation space, we keep away the samples with different semantic by an adaptive margin, and jointly force the modality invariance to eliminate cross-modal discrepancy. Experimental results on widely used benchmark datasets show that the proposed method is effective in cross-modal learning.