Research on cross-modal hashing method based on diffusion mode

Abstract

For achieving fast and flexible retrieval across heterogeneous modalities, unsupervised is more flexible and easy to use than supervised methods, of which the unsupervised method GAN is the most popular. However, GAN has been suffering from the problems of lack of diversity in generated samples, debugging difficulties and training instability. A cross-modal hashing method based on a diffusion model is proposed in the paper. Specifically: (1) For the first time, the diffusion model is applied to the field of cross-modal retrieval, targeting three modalities for mutual retrieval. (2) The combination of adversarial network GAN and diffusion model improves the sample quality and sample diversity, and ameliorates the problems of complex GAN debugging and unstable training. The effectiveness of the proposed method is demonstrated through experiments on three datasets and comparison with state-of-the-art methods.