Revising similarity relationship hashing for unsupervised cross-modal retrieval

Abstract

Previous methods have made promising progress, but there are still some limitations in narrowing the gap between modalities and exploring and preserving intrinsic multimodal semantics. Furthermore, there has been a failure to effectively incorporate the hash codes to correct poorly trained instance pairs during the training process. To overcome the above-mentioned issues, we propose a novel unsupervised hash learning framework, Revising Similarity Relationship Hashing (RSRH). Firstly, we constructed a feature cross-reconstruction module to narrow the gap between modalities. In addition, we build a multimodal fusion similarity map that nonlinearly combines intra- and inter-modal similarity maps to generate multimodal representations with complementary relationships. Finally, we propose a multimodal fusion graph update module for updating poorly trained instance pairs, improving retrieval performance. Experimental data show that our method outperforms many current mainstream hashing methods in performance, and its effectiveness and superiority have been fully validated.