Building an Efficient Retrieval-based Dialogue System with Contrastive Learning

Abstract

We focus on retrieval-based dialogue systems. Such a system aims to select an appropriate response from a candidate pool for a given context. Recent methods commonly utilize powerful interaction-based pre-trained language models like BERT to achieve the goal. However, their time cost is usually not satisfying since the procedure of computing relevance scores is not efficient, especially in scenarios that require online response selection. We propose an efficient dialogue system that utilizes a representation-based BERT to address this issue, which can produce an independent representation for every response candidate and context. The relevance score can be simply calculated by the dot product. We further enhance the representation ability of this model by applying domain adaptive post-training and supervised contrastive learning fine-tuning. Experimental results on two benchmark datasets show that our method achieves competitive performance with other interaction-based models while retaining the advantage of time efficiency. We also provide an empirical and theoretical analysis of time efficiency between representation-based models and interaction-based models. The main contribution of this paper is to propose a novel methodology to build a simple but efficient dialogue system.