A domain-aware model with multi-perspective contrastive learning for natural language understanding

Abstract

Intent detection and slot filling are core tasks in natural language understanding (NLU) for task-oriented dialogue systems. However, current models face challenges with numerous intent categories, slot types, and domain classifications, alongside a shortage of well-annotated datasets, particularly in Chinese. Therefore, we propose a domain-aware model with multi-perspective, multi-positive contrastive learning. First, we adopt a self-supervised contrastive learning with multiple perspectives and multiple positive instances, which is capable of spacing the vectors of positive and negative instances from the domain, intent, and slot perspectives, and fusing more positive instance information to increase the classification effectiveness of the model. Our proposed domain-aware model defines domain-level units at the decoding layer, allowing the model to predict intent and slot information based on domain features, which greatly reduces the search space for intent and slot. In addition, we design a dual-stage attention mechanism for capturing implicitly shared information between intents and slots. We propose a data augmentation method that adds noise to the embedding layer, applies fine-grained augmentation techniques, and filters biased samples based on a similarity threshold. Our model is applied to real task-oriented dialogue systems and compared with other NLU models. Experimental results demonstrate that our proposed model outperforms other models in terms of NLU performance.