Latent structure-oriented asymmetric hashing for cross-modal retrieval

Abstract

Cross-modal hashing has attracted considerable attention in cross-modal retrieval due to its excellent computational efficiency and retrieval performance. Most existing methods aim to map multimodal data into a common representation space where either semantic similarity or instance similarity is preserved. However, these methods do not consider the potential clustering structure of instances that characterizes sample separability, resulting in degraded retrieval performance. Furthermore, capturing the consistent instance similarity by effectively fusing similarities of different modalities remains an essential problem to be addressed. To tackle these issues, this paper proposes a novel latent structure-oriented asymmetric cross-modal Hashing method (LSOAH) for cross-modal retrieval. Specifically, LSOAH formulates the common representation learning with orthogonal decomposition, where each modality-specific instance is projected and decomposed into a modality-specific base matrix and a common cluster indicator matrix, and where the indicator matrix is concatenated with the hash code via an asymmetric mechanism. Additionally, we utilize Hadamard product on graphs from different modalities to explore the consistent instance similarity, and embed it in the common representation. Finally, a unified objective function is presented to enable the simultaneous exploration of the cluster structure, instance similarity and semantic similarity, as well as the hash code learning, upon which an alternating optimization algorithm is developed with theoretically proven convergence. Experimental results on three benchmark datasets confirm the superiority of the proposed LSOAH for cross-modal retrieval.