Semantic Augmentation Transformer Model for Ad-hoc Retrieval

Abstract

We propose SATM, a Semantic Augmentation Transformer Model for ad-hoc retrieval. SATM adapts contrastive learning, which improves the performance of semantic similarity and the ranking results of ad-hoc retrieval. It can also augment the semantic representation of sentence embeddings. Specifically, we first use an unsup-ervised contrastive learning augmentation module to learn query similarity so that the projection head can accurately capture query semantics. Then we use the trained encoder network to map queries and perform semantic similarity calculations and rankings with document embeddings. We use BERT to extract the contextual representation of the sentence, and use the augmentation module to enhance the semantics and eliminate the anisotropy of sentence embedding. Experimental results show that in the TrecQA data set, SATM has 4% and 1.4% improvements in MRR and MAP over Bert-base.