Hierarchical Multi-turn Dialogue Generation Model Based on Double-layer Decoding

Abstract

Intelligent and accurate human-machine dialogue systems can help reduce labor costs in business. Existing models of multi-turn dialogue generation, despite their successes, still suffer from lack of contextual relevance and coherence in the generated responses. In this paper, we propose a hierarchical multi-turn dialogue generation model based on double-layer decoding (HMDM-DD) to exploit the positional relationship and contextual information of the dialogues. First, we use relative position embedding to obtain the sequence of context information, then applying the self-attention mechanism to get long-distance dependencies. Finally, we use double-layer decoding to scrutinize the generated dialogue repeatedly. Experiments on two datasets demonstrate that our model has robust superiority over compared methods in generating informative and fluent dialogues.