A Novel Data-to-Text Generation Model with Transformer Planning and a Wasserstein Auto-Encoder

Abstract

Existing methods for data-to-text generation have difficulty producing diverse texts with low duplication rates. In this paper, we propose a novel data-to-text generation model with Transformer planning and a Wasserstein auto-encoder, which can convert constructed data to coherent and diverse text. This model possesses the following features: Transformer is first used to generate the data planning sequence of the target text content (each sequence is a subset of the input items that can be covered by a sentence), and then the Wasserstein Auto-Encoder(WAE) and a deep neural network are employed to establish the global latent variable space of the model. Second, text generation is performed through a hierarchical structure that takes the data planning sequence, global latent variables, and context of the generated sentences as conditions. Furthermore, to achieve diversity of text expression, a decoder is developed that combines the neural network with the WAE. The experimental results show that this model can achieve higher evaluation scores than the existing baseline models in terms of the diversity metrics of text generation and the duplication rate.