A Transformer based Multi-task Model for Domain Classification, Intent Detection and Slot-Filling

Abstract

With the ever increasing complexity of the user queries in a multi-domain based task-oriented dialogue system, it is imperative to facilitate robust Spoken Language Understanding (SLU) modules that perform multiple tasks in an unified way. In this paper, we present a novel multi-task approach for the joint modelling of three tasks together, namely, Domain Classification, Intent Detection and Slot-Filling. We hypothesize with the intuition that the cross dependencies of all these three tasks mutually help each other towards their representations and classifications which further simplify the SLU module in a multi-domain scenario. Towards this end, we propose a BERT language model based multi-task framework utilizing capsule networks and conditional random fields for addressing the classification and sequence labeling problems, respectively, for different tasks. Experimental results indicate that the proposed multi-task model outperformed several strong baselines and its single task counterparts on three benchmark datasets of different domains and attained state-of-the-art results on different tasks.