Causal Attention Transformer for Video Text Retrieval

In the metaverse, video text retrieval is an urgent and challenging need for users in social entertainment. The current attention-based video text retrieval models have not fully explored the interaction between video and text, and only brute force feature embedding. Moreover, Due to the unsupervised nature of attention weight training, existing models have weak generalization performance for dataset bias. Essentially, the model learns that false relevant information in the data is caused by confounding factors. Therefore, this article proposes a video text retrieval method based on causal attention transformer. Assuming that the confounding factors affecting the performance of video text retrieval all come from the dataset, a structural causal model that conforms to the video text retrieval task is constructed, and the impact of confounding effects during data training is reduced by adjusting the front door. In addition, we use causal attention transformer to construct a causal inference network to extract causal features between video text pairs, and replace the similarity statistical probability with causal probability in the video text retrieval framework. Experiments are conducted on the MSR-VTT, MSVD, and LSMDC datasets, which proves the effectiveness of the retrieval model proposed in this paper.