A centroid-based fine-tuning method for out-of-scope classification

Abstract

Accurately detecting out-of-scope queries is a challenging task in task-oriented dialog systems. Most existing research focus on adding an outlier detector after classification or designing an open world classification to identify unknown intents. There is still a major performance gap on achieving high efficiency and accuracy based on above methods. In our research, we tend to solve this problem by constructing an out-of-scope class in the classification. We propose an explainable centroid-based fine-tuning method including a modified decision metric (MDM) and a centroid-based cosine loss (CCL) on Pre-trained Transformer models for optimization. This loss function builds on Copernican structure and assigns the same margin to each in-scope class to resolve an ambiguous configuration on out-of-scope detection. Moreover, cosine similarity is utilized to remove radial variations of centroids. Experimental results show that our proposed method achieves improvement compared to other baseline methods.