Chinese text summarization generation based on transformer and temporal convolutional network

Abstract

The recurrent neural network model based on attention mechanism has achieved good results in the text summarization generation task, but such models have problems such as insufficient parallelism and exposure bias. In order to solve the above problems, this paper proposes a two-stage Chinese text summarization generation method based on Transformer and temporal convolutional network. The first stage uses a summary generation model that fuses Transformer and a temporal convolutional network, and generates multiple candidate summaries through beam search at the decoding end. In the second stage, contrastive learning is introduced, and the candidate summaries are sorted and scored using the Roberta model to select the final summary. Through experiments on the Chinese short text summarization dataset LCSTS, ROUGE was used as the evaluation method to verify the effectiveness of the proposed method on Chinese text summarization.