Deep hashing image retrieval based on CNN and visual transformer network

Deep hashing technology can achieve unified representation of multimedia technology, which is widely used in fields such as smart agriculture, smart transportation, and public safety. However, the current deep hashing methods cannot achieve efficient fusion of global and local information, and the learning ability of hash code representation needs to be further strengthened. In this paper, we propose a deep hashing retrieval algorithm based on integrated CNN and visual Transformer(ICVT) network. Firstly, we propose a lightweight nonlinear spatial group enhancement (NSGE) module, which is integrated with the Transformer through a parallel architecture and introduces a self-attention mechanism to enhance the spatial distribution of semantic features through the similarity between local and global features. The CNN representation capability of feature maps is improved to enhance the spatial distribution of semantic features within each feature semantic group. Secondly, we propose a margin contrast loss function to optimize the hash code parameters, which is used to improve the retrieval accuracy of the algorithm for multi-label datasets. Finally, extensive experiments on CIFAR-10, NUS-WIDE, and ImageNet datasets demonstrate that ICVT has superior retrieval performance compared with the currently popular algorithms, especially on CIFAR-10, where the mAP performance reaches 97.5%.