Survey on Automatic Text Summarization and Transformer Models Applicability

Abstract

This survey talks about Automatic Text Summarization. Information explosion, the problem caused by the rapid growth of the internet, increased more and more necessity of powerful summarizers. This article briefly reviews different methods and evaluation metrics. The main attention is on the applications of the latest trends, neural network-based, and pre-trained transformer language models. Pre-trained language models now are ruling the NLP field, as one of the main down-stream tasks, Automatic Text Summarization is quite an interdisciplinary task and requires more advanced techniques. But there is a limitation of input and context length results in that the whole article cannot be encoded completely. Motivated by the application of recurrent mechanism in Transformer-XL, we build an abstractive summarizer for long text and evaluate how well it performs on dataset CNN/Daily Mail. The model is under general sequence to sequence structure with a recurrent encoder and stacked Transformer decoder. The obtained ROUGE scores tell that the performance is good as expected.