Joint intent detection and slot filling for Turkish natural language understanding

Abstract

Intent detection and slot filling are two crucial subtasks of a text-based goal-oriented dialogue system. In a goal-oriented dialogue system, users interact with the system to complete a goal (or to fulfill their intent) and provide the necessary information (slot values) to achieve that goal. Therefore, a user?s text input includes information about the user?s intent and contains required slot values. Recently, joint models that simultaneously detect the intent and extract the slots are proposed to benefit from the interaction between the two tasks. The proposed methods are usually tested using benchmark data sets in English such as ATIS and SNIPS. Intent detection and slot filling problems are much less studied for the Turkish language mainly due to the lack of publicly available Turkish data sets. In this paper, we translate ATIS in English to Turkish and report intent detection and slot filling accuracies of several different joint models for the translated data set. We publicly share the Turkish ATIS data set to accelerate the research on the tasks. In our experiments, the best performance is obtained with the state-of-the-art bidirectional encoder representations from a transformers (BERT) based model. The BERT model is trained using a combination of intent detection and slot filling losses to jointly optimize a single model for both tasks. We achieved 96.54% intent detection accuracy and 91.56% slot filling F1 for the Turkish language. These accuracies significantly improve (7% absolute in slot filling F1) previously reported results for the same tasks in Turkish. On the other hand, we observe that the accuracy in Turkish is still slightly lower compared to the accuracy in English counterparts. This observation indicates that there is still room for improvement in the results for Turkish.