Video-text cross-modal retrieval algorithm based on multiple coding

Abstract

Currently, more and more video data and terminal devices accessing video resources are available to users. Video platforms such as Tiktok and Youtube are gradually rising, and the user scale and video resources are increasing day by day, which brings an urgent practical demand for video-text data cross-modal retrieval. This paper proposes a video-text cross-modal retrieval algorithm based on multiple encoding. By encoding the global features, serial features and local features of video and text, the encoded features are mapped to the common embedding space for training, loss function calculation and optimization. Through experimental verification on MASR-VTT data set and comparison with existing methods, the overall performance R@sum increased by 9.22% and 2.86% respectively, which proved the superiority of this method.