DoS: Abstractive text summarization based on pretrained model with document sharing

Abstract

In this paper, an abstractive text summarization method with document sharing is proposed. It consists of a pretrained model and self-attention mechanism on multi-document. We call it DoS mechanism. By applying the mechanism to the single-document text summarization task, the model can absorb information from multiple documents, thus enhancing its effectiveness of the model. We compared the results with several models. The experimental results show that the pre-trained model with modified attention provides the best results, where the values of Rouge-l, Rouge-2, and Rouge-L are 41.3%, 27.4%, and 38.0%, respectively. Evaluations on the LCSTS demonstrate that our model outperforms the baseline model. Subsequent analysis showed that our model was able to generate higherquality summaries.