Deep adaptive gradient-triplet hashing for cross-modal retrieval

Abstract

Due to its low storage and high computing efficiency, deep cross-modal hashing has a wide application prospect in large-scale cross-modal retrieval. However, fixed gradients and manually set similarity margins in traditional triplet loss hinder the model’s ability to adapt to varying sample difficulties, leading to poor discrimination of hard negatives and degraded hash code quality, especially when positive-negative distances exceed the preset margin. In addition, most deep cross-modal hashing methods learn both similarity and quantization, and the interaction between the two can break the embedding, resulting in sub-optimal hash codes. In this paper, the Deep Adaptive Gradient-triplet Hashing (DAGtH) framework is proposed to embed heterogeneous modalities data into a discrimative discrete space and capture neighborhood relationships in primitive space. Specially, by setting suitable gradients to triples with different hardness, a new adaptive gradient-triplet loss is proposed to preserve the consistency of neighborhood relationships in original space, prompting intra-class compactness and inter-class separability of heterogeneous modalities. Meanwhile, by dividing the learning process into two parts, the Householder quantization loss is introduced into cross-modal retrieval to reduce lossy compression from the quantization. First, performing similarity learning in the embedding space. Second, the orthogonal transformation is optimized to reduce the distance between embedded and discrete binary code. To validate the effectiveness of our proposed DAGtH framework, comprehensive experiments are conducted on three benchmark datasets, and our approach achieves the increase of 0.61 %–13.8 % in mean average precision (mAP) at different bit lengths compared to the state-of-the-art hashing, which demonstrates that DAGtH achieves optimal retrieval performance. The code for our DAGtH framework can be found here: https://github.com/QinLab-WFU/OUR-DAGtH.